Sep 11 04:39:44.018621 ip-10-0-128-18 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 04:39:44.465081 ip-10-0-128-18 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 04:39:44.465081 ip-10-0-128-18 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 04:39:44.465081 ip-10-0-128-18 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 04:39:44.465081 ip-10-0-128-18 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 04:39:44.465081 ip-10-0-128-18 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 04:39:44.465747 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.465092    2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 04:39:44.468409 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468385    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 04:39:44.468409 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468409    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 04:39:44.468409 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468413    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 04:39:44.468409 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468417    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 04:39:44.468409 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468420    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 04:39:44.468554 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468423    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 04:39:44.468554 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468426    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 04:39:44.468554 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468428    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 04:39:44.468554 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468431    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 04:39:44.468554 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468433    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 04:39:44.468554 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468436    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 04:39:44.468554 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468438    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 04:39:44.468554 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468441    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 04:39:44.468554 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468443    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 04:39:44.468554 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468446    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 04:39:44.468554 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468449    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 04:39:44.468554 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468451    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 04:39:44.468554 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468454    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 04:39:44.468554 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468456    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 04:39:44.468554 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468459    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 04:39:44.468554 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468461    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 04:39:44.468554 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468464    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 04:39:44.468554 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468466    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 04:39:44.468554 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468469    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 04:39:44.468554 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468472    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 04:39:44.469029 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468475    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 04:39:44.469029 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468477    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 04:39:44.469029 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468479    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 04:39:44.469029 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468482    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 04:39:44.469029 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468485    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 04:39:44.469029 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468487    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 04:39:44.469029 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468489    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 04:39:44.469029 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468492    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 04:39:44.469029 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468495    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 04:39:44.469029 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468497    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 04:39:44.469029 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468499    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 04:39:44.469029 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468502    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 04:39:44.469029 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468504    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 04:39:44.469029 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468507    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 04:39:44.469029 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468510    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 04:39:44.469029 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468513    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 04:39:44.469029 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468515    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 04:39:44.469029 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468518    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 04:39:44.469029 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468520    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 04:39:44.469029 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468522    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 04:39:44.469552 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468525    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 04:39:44.469552 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468528    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 04:39:44.469552 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468530    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 04:39:44.469552 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468532    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 04:39:44.469552 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468535    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 04:39:44.469552 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468537    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 04:39:44.469552 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468540    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 04:39:44.469552 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468542    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 04:39:44.469552 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468545    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 04:39:44.469552 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468547    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 04:39:44.469552 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468550    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 04:39:44.469552 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468552    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 04:39:44.469552 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468555    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 04:39:44.469552 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468557    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 04:39:44.469552 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468561    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 04:39:44.469552 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468565    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 04:39:44.469552 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468568    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 04:39:44.469552 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468571    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 04:39:44.469552 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468573    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 04:39:44.469552 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468576    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 04:39:44.470040 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468578    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 04:39:44.470040 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468581    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 04:39:44.470040 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468584    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 04:39:44.470040 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468586    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 04:39:44.470040 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468589    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 04:39:44.470040 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468592    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 04:39:44.470040 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468595    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 04:39:44.470040 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468597    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 04:39:44.470040 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468600    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 04:39:44.470040 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468602    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 04:39:44.470040 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468606    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 04:39:44.470040 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468610    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 04:39:44.470040 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468614    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 04:39:44.470040 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468617    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 04:39:44.470040 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468620    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 04:39:44.470040 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468624    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 04:39:44.470040 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468626    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 04:39:44.470040 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468629    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 04:39:44.470040 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468631    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 04:39:44.470040 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468634    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 04:39:44.470543 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468636    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 04:39:44.470543 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468639    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 04:39:44.470543 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468641    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 04:39:44.470543 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468644    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 04:39:44.470543 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468646    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 04:39:44.470543 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468649    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 04:39:44.470543 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468653    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 04:39:44.470543 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468656    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 04:39:44.470543 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468659    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 04:39:44.470543 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468662    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 04:39:44.470543 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468665    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 04:39:44.470543 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468668    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 04:39:44.470543 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468670    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 04:39:44.470543 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.468673    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 04:39:44.470543 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469353    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 04:39:44.470543 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469360    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 04:39:44.470543 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469363    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 04:39:44.470543 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469367    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 04:39:44.470543 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469370    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 04:39:44.471039 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469373    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 04:39:44.471039 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469376    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 04:39:44.471039 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469379    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 04:39:44.471039 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469381    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 04:39:44.471039 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469384    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 04:39:44.471039 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469387    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 04:39:44.471039 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469390    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 04:39:44.471039 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469393    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 04:39:44.471039 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469395    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 04:39:44.471039 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469398    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 04:39:44.471039 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469400    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 04:39:44.471039 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469403    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 04:39:44.471039 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469405    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 04:39:44.471039 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469408    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 04:39:44.471039 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469411    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 04:39:44.471039 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469413    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 04:39:44.471039 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469416    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 04:39:44.471039 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469418    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 04:39:44.471039 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469420    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 04:39:44.471039 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469423    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 04:39:44.471560 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469425    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 04:39:44.471560 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469428    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 04:39:44.471560 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469430    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 04:39:44.471560 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469432    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 04:39:44.471560 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469435    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 04:39:44.471560 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469438    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 04:39:44.471560 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469440    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 04:39:44.471560 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469443    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 04:39:44.471560 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469445    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 04:39:44.471560 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469448    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 04:39:44.471560 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469451    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 04:39:44.471560 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469454    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 04:39:44.471560 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469456    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 04:39:44.471560 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469459    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 04:39:44.471560 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469461    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 04:39:44.471560 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469464    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 04:39:44.471560 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469466    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 04:39:44.471560 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469469    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 04:39:44.471560 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469472    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 04:39:44.471560 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469475    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 04:39:44.472053 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469477    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 04:39:44.472053 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469480    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 04:39:44.472053 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469482    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 04:39:44.472053 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469485    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 04:39:44.472053 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469487    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 04:39:44.472053 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469490    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 04:39:44.472053 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469492    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 04:39:44.472053 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469503    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 04:39:44.472053 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469506    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 04:39:44.472053 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469509    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 04:39:44.472053 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469511    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 04:39:44.472053 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469514    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 04:39:44.472053 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469516    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 04:39:44.472053 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469519    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 04:39:44.472053 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469521    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 04:39:44.472053 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469524    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 04:39:44.472053 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469527    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 04:39:44.472053 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469530    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 04:39:44.472053 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469532    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 04:39:44.472053 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469535    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 04:39:44.472555 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469538    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 04:39:44.472555 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469540    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 04:39:44.472555 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469543    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 04:39:44.472555 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469546    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 04:39:44.472555 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469548    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 04:39:44.472555 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469551    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 04:39:44.472555 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469553    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 04:39:44.472555 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469556    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 04:39:44.472555 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469559    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 04:39:44.472555 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469564    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 04:39:44.472555 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469568    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 04:39:44.472555 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469571    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 04:39:44.472555 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469575    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 04:39:44.472555 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469578    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 04:39:44.472555 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469581    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 04:39:44.472555 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469583    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 04:39:44.472555 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469586    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 04:39:44.472555 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469589    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 04:39:44.472555 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469592    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 04:39:44.473019 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469594    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 04:39:44.473019 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469597    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 04:39:44.473019 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469601    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 04:39:44.473019 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469605    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 04:39:44.473019 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469609    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 04:39:44.473019 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469612    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 04:39:44.473019 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469614    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 04:39:44.473019 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469617    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 04:39:44.473019 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469620    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 04:39:44.473019 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469623    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 04:39:44.473019 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469626    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 04:39:44.473019 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469629    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 04:39:44.473019 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469631    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 04:39:44.473019 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469634    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 04:39:44.473019 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469637    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 04:39:44.473019 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469988    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 04:39:44.473019 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469993    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 04:39:44.473019 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469996    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 04:39:44.473019 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.469999    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 04:39:44.473513 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470002    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 04:39:44.473513 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470004    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 04:39:44.473513 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470007    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 04:39:44.473513 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470009    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 04:39:44.473513 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470012    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 04:39:44.473513 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470014    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 04:39:44.473513 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470017    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 04:39:44.473513 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470020    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 04:39:44.473513 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470022    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 04:39:44.473513 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470025    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 04:39:44.473513 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470027    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 04:39:44.473513 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470030    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 04:39:44.473513 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470033    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 04:39:44.473513 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470036    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 04:39:44.473513 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470038    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 04:39:44.473513 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470041    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 04:39:44.473513 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470044    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 04:39:44.473513 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470046    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 04:39:44.473513 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470049    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 04:39:44.473513 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470051    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 04:39:44.474000 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470054    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 04:39:44.474000 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470057    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 04:39:44.474000 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470060    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 04:39:44.474000 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470063    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 04:39:44.474000 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470065    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 04:39:44.474000 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470067    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 04:39:44.474000 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470070    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 04:39:44.474000 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470072    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 04:39:44.474000 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470075    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 04:39:44.474000 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470078    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 04:39:44.474000 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470080    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 04:39:44.474000 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470082    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 04:39:44.474000 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470085    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 04:39:44.474000 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470088    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 04:39:44.474000 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470090    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 04:39:44.474000 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470092    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 04:39:44.474000 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470095    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 04:39:44.474000 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470097    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 04:39:44.474000 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470099    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 04:39:44.474570 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470105    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 04:39:44.474570 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470107    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 04:39:44.474570 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470109    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 04:39:44.474570 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470112    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 04:39:44.474570 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470114    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 04:39:44.474570 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470118    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 04:39:44.474570 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470122    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 04:39:44.474570 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470124    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 04:39:44.474570 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470126    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 04:39:44.474570 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470129    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 04:39:44.474570 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470131    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 04:39:44.474570 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470133    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 04:39:44.474570 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470136    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 04:39:44.474570 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470139    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 04:39:44.474570 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470141    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 04:39:44.474570 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470144    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 04:39:44.474570 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470146    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 04:39:44.474570 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470148    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 04:39:44.474570 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470151    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 04:39:44.474570 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470153    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 04:39:44.475045 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470156    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 04:39:44.475045 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470158    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 04:39:44.475045 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470161    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 04:39:44.475045 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470163    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 04:39:44.475045 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470166    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 04:39:44.475045 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470168    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 04:39:44.475045 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470170    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 04:39:44.475045 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470174    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 04:39:44.475045 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470177    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 04:39:44.475045 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470181    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 04:39:44.475045 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470184    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 04:39:44.475045 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470186    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 04:39:44.475045 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470192    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 04:39:44.475045 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470195    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 04:39:44.475045 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470198    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 04:39:44.475045 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470200    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 04:39:44.475045 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470202    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 04:39:44.475045 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470205    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 04:39:44.475045 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470207    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 04:39:44.475045 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470210    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 04:39:44.475577 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470212    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 04:39:44.475577 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470214    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 04:39:44.475577 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470216    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 04:39:44.475577 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470219    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 04:39:44.475577 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470221    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 04:39:44.475577 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470224    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 04:39:44.475577 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470226    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 04:39:44.475577 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470229    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 04:39:44.475577 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470231    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 04:39:44.475577 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470235    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 04:39:44.475577 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470238    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 04:39:44.475577 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470241    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 04:39:44.475577 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470244    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 04:39:44.475577 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470247    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 04:39:44.475577 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470249    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 04:39:44.475577 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470252    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 04:39:44.475577 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470324    2567 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 04:39:44.475577 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470373    2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 04:39:44.475577 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470381    2567 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 04:39:44.475577 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470386    2567 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 04:39:44.475577 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470392    2567 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470395    2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470399    2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470403    2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470408    2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470410    2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470414    2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470418    2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470421    2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470424    2567 flags.go:64] FLAG: --cgroup-root=""
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470427    2567 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470430    2567 flags.go:64] FLAG: --client-ca-file=""
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470432    2567 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470435    2567 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470439    2567 flags.go:64] FLAG: --cluster-domain=""
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470442    2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470445    2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470448    2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470451    2567 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470454    2567 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470457    2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470460    2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470463    2567 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470466    2567 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 04:39:44.476115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470469    2567 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470471    2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470474    2567 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470478    2567 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470482    2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470484    2567 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470487    2567 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470490    2567 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470493    2567 flags.go:64] FLAG: --enable-server="true"
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470496    2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470500    2567 flags.go:64] FLAG: --event-burst="100"
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470503    2567 flags.go:64] FLAG: --event-qps="50"
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470507    2567 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470510    2567 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470513    2567 flags.go:64] FLAG: --eviction-hard=""
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470517    2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470520    2567 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470523    2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470526    2567 flags.go:64] FLAG: --eviction-soft=""
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470529    2567 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470531    2567 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470534    2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470537    2567 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470539    2567 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470542    2567 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 04:39:44.476705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470545    2567 flags.go:64] FLAG: --feature-gates=""
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470549    2567 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470552    2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470555    2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470558    2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470561    2567 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470564    2567 flags.go:64] FLAG: --help="false"
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470567    2567 flags.go:64] FLAG: --hostname-override="ip-10-0-128-18.ec2.internal"
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470570    2567 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470572    2567 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470575    2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470578    2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470582    2567 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470584    2567 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470587    2567 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470590    2567 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470593    2567 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470596    2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470599    2567 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470603    2567 flags.go:64] FLAG: --kube-reserved=""
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470606    2567 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470608    2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470611    2567 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470614    2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 04:39:44.477350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470619    2567 flags.go:64] FLAG: --lock-file=""
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470622    2567 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470624    2567 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470627    2567 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470632    2567 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470635    2567 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470638    2567 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470641    2567 flags.go:64] FLAG: --logging-format="text"
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470643    2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470647    2567 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470649    2567 flags.go:64] FLAG: --manifest-url=""
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470652    2567 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470656    2567 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470659    2567 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470663    2567 flags.go:64] FLAG: --max-pods="110"
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470666    2567 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470669    2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470671    2567 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470674    2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470677    2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470680    2567 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470683    2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470690    2567 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470693    2567 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470696    2567 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 04:39:44.477978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470699    2567 flags.go:64] FLAG: --pod-cidr=""
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470702    2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470707    2567 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470711    2567 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470714    2567 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470717    2567 flags.go:64] FLAG: --port="10250"
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470720    2567 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470725    2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01518d5d26946c8d4"
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470728    2567 flags.go:64] FLAG: --qos-reserved=""
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470731    2567 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470733    2567 flags.go:64] FLAG: --register-node="true"
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470736    2567 flags.go:64] FLAG: --register-with-taints=""
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470740    2567 flags.go:64] FLAG: --registry-burst="10"
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470743    2567 flags.go:64] FLAG: --registry-qps="5"
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470745    2567 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470748    2567 flags.go:64] FLAG: --reserved-memory=""
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470751    2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470754    2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470758    2567 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470760    2567 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470763    2567 flags.go:64] FLAG: --runonce="false"
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470766    2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470769    2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470772    2567 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470774    2567 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 04:39:44.478618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470777    2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470780    2567 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470783    2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470786    2567 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470789    2567 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470791    2567 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470794    2567 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470796    2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470799    2567 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470802    2567 flags.go:64] FLAG: --system-cgroups=""
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470806    2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470812    2567 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470815    2567 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470818    2567 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470824    2567 flags.go:64] FLAG: --tls-min-version=""
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470827    2567 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470830    2567 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470834    2567 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470836    2567 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470839    2567 flags.go:64] FLAG: --v="2"
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470843    2567 flags.go:64] FLAG: --version="false"
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470847    2567 flags.go:64] FLAG: --vmodule=""
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470850    2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.470853    2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 04:39:44.479253 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470959    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 04:39:44.479844 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470962    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 04:39:44.479844 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470965    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 04:39:44.479844 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470968    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 04:39:44.479844 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470971    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 04:39:44.479844 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470973    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 04:39:44.479844 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470976    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 04:39:44.479844 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470979    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 04:39:44.479844 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470981    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 04:39:44.479844 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470984    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 04:39:44.479844 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470987    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 04:39:44.479844 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470989    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 04:39:44.479844 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470992    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 04:39:44.479844 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470994    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 04:39:44.479844 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470997    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 04:39:44.479844 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.470999    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 04:39:44.479844 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471002    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 04:39:44.479844 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471004    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 04:39:44.479844 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471008    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 04:39:44.479844 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471011    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 04:39:44.479844 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471013    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 04:39:44.480351 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471017    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 04:39:44.480351 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471022    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 04:39:44.480351 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471025    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 04:39:44.480351 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471028    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 04:39:44.480351 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471031    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 04:39:44.480351 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471034    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 04:39:44.480351 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471036    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 04:39:44.480351 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471040    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 04:39:44.480351 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471043    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 04:39:44.480351 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471046    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 04:39:44.480351 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471050    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 04:39:44.480351 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471053    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 04:39:44.480351 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471056    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 04:39:44.480351 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471058    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 04:39:44.480351 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471061    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 04:39:44.480351 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471063    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 04:39:44.480351 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471066    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 04:39:44.480351 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471068    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 04:39:44.480351 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471071    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 04:39:44.480830 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471073    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 04:39:44.480830 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471076    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 04:39:44.480830 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471078    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 04:39:44.480830 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471081    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 04:39:44.480830 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471083    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 04:39:44.480830 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471086    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 04:39:44.480830 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471088    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 04:39:44.480830 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471091    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 04:39:44.480830 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471093    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 04:39:44.480830 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471095    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 04:39:44.480830 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471100    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 04:39:44.480830 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471102    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 04:39:44.480830 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471105    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 04:39:44.480830 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471107    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 04:39:44.480830 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471111    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 04:39:44.480830 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471113    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 04:39:44.480830 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471116    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 04:39:44.480830 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471119    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 04:39:44.480830 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471121    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 04:39:44.480830 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471124    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 04:39:44.480830 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471126    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 04:39:44.481365 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471129    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 04:39:44.481365 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471131    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 04:39:44.481365 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471134    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 04:39:44.481365 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471136    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 04:39:44.481365 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471138    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 04:39:44.481365 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471141    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 04:39:44.481365 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471143    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 04:39:44.481365 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471146    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 04:39:44.481365 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471148    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 04:39:44.481365 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471151    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 04:39:44.481365 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471153    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 04:39:44.481365 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471156    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 04:39:44.481365 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471159    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 04:39:44.481365 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471163    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 04:39:44.481365 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471166    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 04:39:44.481365 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471169    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 04:39:44.481365 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471171    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 04:39:44.481365 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471174    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 04:39:44.481365 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471176    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 04:39:44.481846 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471178    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 04:39:44.481846 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471181    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 04:39:44.481846 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471185    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 04:39:44.481846 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471187    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 04:39:44.481846 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471190    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 04:39:44.481846 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471192    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 04:39:44.481846 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471196    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 04:39:44.481846 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471198    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 04:39:44.481846 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471201    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 04:39:44.481846 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471203    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 04:39:44.481846 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471206    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 04:39:44.481846 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471209    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 04:39:44.481846 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471211    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 04:39:44.481846 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471214    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 04:39:44.481846 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471216    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 04:39:44.481846 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471219    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 04:39:44.481846 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471221    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 04:39:44.481846 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471224    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 04:39:44.481846 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.471226    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 04:39:44.482319 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.471231    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 04:39:44.482319 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.479149    2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 04:39:44.482319 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.479163    2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 04:39:44.482319 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.481077    2567 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 04:39:44.482319 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.481086    2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 04:39:44.482319 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481182    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 04:39:44.482319 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481187    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 04:39:44.482319 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481190    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 04:39:44.482319 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481193    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 04:39:44.482319 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481196    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 04:39:44.482319 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481198    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 04:39:44.482319 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481201    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 04:39:44.482319 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481204    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 04:39:44.482319 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481206    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 04:39:44.482766 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481209    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 04:39:44.482766 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481212    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 04:39:44.482766 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481214    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 04:39:44.482766 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481217    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 04:39:44.482766 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481219    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 04:39:44.482766 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481221    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 04:39:44.482766 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481224    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 04:39:44.482766 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481226    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 04:39:44.482766 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481229    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 04:39:44.482766 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481232    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 04:39:44.482766 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481235    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 04:39:44.482766 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481237    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 04:39:44.482766 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481240    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 04:39:44.482766 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481242    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 04:39:44.482766 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481245    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 04:39:44.482766 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481248    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 04:39:44.482766 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481250    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 04:39:44.482766 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481253    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 04:39:44.482766 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481255    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 04:39:44.482766 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481257    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 04:39:44.483259 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481261    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 04:39:44.483259 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481264    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 04:39:44.483259 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481266    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 04:39:44.483259 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481269    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 04:39:44.483259 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481271    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 04:39:44.483259 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481274    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 04:39:44.483259 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481277    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 04:39:44.483259 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481279    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 04:39:44.483259 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481282    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 04:39:44.483259 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481284    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 04:39:44.483259 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481288    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 04:39:44.483259 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481293    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 04:39:44.483259 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481295    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 04:39:44.483259 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481298    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 04:39:44.483259 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481300    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 04:39:44.483259 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481303    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 04:39:44.483259 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481305    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 04:39:44.483259 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481307    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 04:39:44.483259 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481310    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 04:39:44.483259 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481312    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 04:39:44.483763 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481315    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 04:39:44.483763 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481318    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 04:39:44.483763 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481328    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 04:39:44.483763 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481331    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 04:39:44.483763 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481353    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 04:39:44.483763 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481358    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 04:39:44.483763 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481363    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 04:39:44.483763 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481366    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 04:39:44.483763 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481370    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 04:39:44.483763 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481373    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 04:39:44.483763 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481376    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 04:39:44.483763 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481379    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 04:39:44.483763 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481382    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 04:39:44.483763 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481385    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 04:39:44.483763 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481388    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 04:39:44.483763 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481390    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 04:39:44.483763 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481393    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 04:39:44.483763 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481395    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 04:39:44.484205 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481398    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 04:39:44.484205 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481400    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 04:39:44.484205 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481403    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 04:39:44.484205 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481405    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 04:39:44.484205 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481408    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 04:39:44.484205 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481415    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 04:39:44.484205 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481417    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 04:39:44.484205 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481420    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 04:39:44.484205 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481422    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 04:39:44.484205 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481425    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 04:39:44.484205 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481427    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 04:39:44.484205 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481430    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 04:39:44.484205 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481432    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 04:39:44.484205 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481435    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 04:39:44.484205 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481437    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 04:39:44.484205 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481440    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 04:39:44.484205 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481442    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 04:39:44.484205 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481450    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 04:39:44.484205 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481452    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 04:39:44.484205 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481455    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 04:39:44.484713 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481457    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 04:39:44.484713 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481460    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 04:39:44.484713 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481462    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 04:39:44.484713 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481464    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 04:39:44.484713 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481467    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 04:39:44.484713 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481470    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 04:39:44.484713 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481473    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 04:39:44.484713 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481481    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 04:39:44.484713 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481483    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 04:39:44.484713 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481486    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 04:39:44.484713 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481488    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 04:39:44.484713 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481491    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 04:39:44.484713 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.481502    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 04:39:44.484713 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481643    2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 04:39:44.484713 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481648    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 04:39:44.485088 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481652    2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 04:39:44.485088 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481654    2567 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 04:39:44.485088 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481657    2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 04:39:44.485088 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481659    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 04:39:44.485088 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481662    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 04:39:44.485088 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481664    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 04:39:44.485088 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481667    2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 04:39:44.485088 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481669    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 04:39:44.485088 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481672    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 04:39:44.485088 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481674    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 04:39:44.485088 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481677    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 04:39:44.485088 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481680    2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 04:39:44.485088 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481683    2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 04:39:44.485088 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481686    2567 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 04:39:44.485088 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481690    2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 04:39:44.485088 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481698    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 04:39:44.485088 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481701    2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 04:39:44.485088 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481704    2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 04:39:44.485088 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481707    2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 04:39:44.485088 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481709    2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 04:39:44.485735 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481712    2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 04:39:44.485735 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481714    2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 04:39:44.485735 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481717    2567 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 04:39:44.485735 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481720    2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 04:39:44.485735 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481722    2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 04:39:44.485735 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481725    2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 04:39:44.485735 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481727    2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 04:39:44.485735 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481729    2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 04:39:44.485735 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481732    2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 04:39:44.485735 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481735    2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 04:39:44.485735 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481739    2567 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 04:39:44.485735 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481741    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 04:39:44.485735 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481744    2567 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 04:39:44.485735 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481746    2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 04:39:44.485735 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481749    2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 04:39:44.485735 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481751    2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 04:39:44.485735 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481754    2567 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 04:39:44.485735 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481756    2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 04:39:44.485735 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481759    2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 04:39:44.485735 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481761    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 04:39:44.486216 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481764    2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 04:39:44.486216 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481766    2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 04:39:44.486216 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481769    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 04:39:44.486216 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481771    2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 04:39:44.486216 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481773    2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 04:39:44.486216 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481776    2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 04:39:44.486216 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481778    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 04:39:44.486216 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481781    2567 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 04:39:44.486216 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481789    2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 04:39:44.486216 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481792    2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 04:39:44.486216 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481794    2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 04:39:44.486216 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481797    2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 04:39:44.486216 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481799    2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 04:39:44.486216 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481802    2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 04:39:44.486216 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481804    2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 04:39:44.486216 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481806    2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 04:39:44.486216 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481809    2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 04:39:44.486216 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481811    2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 04:39:44.486216 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481814    2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 04:39:44.486216 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481816    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 04:39:44.486723 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481818    2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 04:39:44.486723 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481821    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 04:39:44.486723 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481823    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 04:39:44.486723 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481826    2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 04:39:44.486723 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481828    2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 04:39:44.486723 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481831    2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 04:39:44.486723 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481833    2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 04:39:44.486723 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481835    2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 04:39:44.486723 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481838    2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 04:39:44.486723 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481840    2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 04:39:44.486723 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481842    2567 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 04:39:44.486723 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481845    2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 04:39:44.486723 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481847    2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 04:39:44.486723 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481850    2567 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 04:39:44.486723 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481852    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 04:39:44.486723 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481855    2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 04:39:44.486723 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481858    2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 04:39:44.486723 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481860    2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 04:39:44.486723 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481863    2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 04:39:44.486723 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481865    2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 04:39:44.487199 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481868    2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 04:39:44.487199 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481876    2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 04:39:44.487199 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481878    2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 04:39:44.487199 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481881    2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 04:39:44.487199 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481883    2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 04:39:44.487199 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481886    2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 04:39:44.487199 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481888    2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 04:39:44.487199 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481891    2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 04:39:44.487199 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481893    2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 04:39:44.487199 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481895    2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 04:39:44.487199 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481898    2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 04:39:44.487199 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481900    2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 04:39:44.487199 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481902    2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 04:39:44.487199 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481905    2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 04:39:44.487199 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481907    2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 04:39:44.487199 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481910    2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 04:39:44.487199 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:44.481912    2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 04:39:44.487685 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.481917    2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 04:39:44.487685 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.482017    2567 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 04:39:44.487685 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.484602    2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 04:39:44.487685 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.485686    2567 server.go:1019] "Starting client certificate rotation"
Sep 11 04:39:44.487685 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.485803    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 04:39:44.487685 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.485846    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 04:39:44.489226 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.489213    2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 04:39:44.491867 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.491837    2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 04:39:44.504878 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.504859    2567 log.go:25] "Validated CRI v1 runtime API"
Sep 11 04:39:44.508414 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.508397    2567 log.go:25] "Validated CRI v1 image API"
Sep 11 04:39:44.509250 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.509230    2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 04:39:44.513385 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.513361    2567 fs.go:135] Filesystem UUIDs: map[3c7b13e8-eba2-465b-9149-d17541b37c54:/dev/nvme0n1p3 59c5bcb4-e57b-42ea-9847-80fab29684fa:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 11 04:39:44.513440 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.513385    2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 04:39:44.519545 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.519407    2567 manager.go:216] Machine: {Timestamp:2026-09-11 04:39:44.518096599 +0000 UTC m=+0.374289168 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101055 MemoryCapacity:32812158976 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2623d37e1330c72a026271aaa2ea77 SystemUUID:ec2623d3-7e13-30c7-2a02-6271aaa2ea77 BootID:84eb9d4d-3272-4f81-b736-e273225f6d95 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:79:52:84:8d:4f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:79:52:84:8d:4f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:b3:51:fd:27:ee Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812158976 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 04:39:44.519545 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.519535    2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 04:39:44.519675 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.519609    2567 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 04:39:44.519915 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.519902    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 04:39:44.523043 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.523013    2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 04:39:44.523171 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.523046    2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-18.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 04:39:44.523902 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.523893    2567 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 04:39:44.523940 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.523904    2567 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 04:39:44.523940 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.523912    2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 04:39:44.524798 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.524788    2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 04:39:44.526040 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.526030    2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 04:39:44.531582 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.531570    2567 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 04:39:44.531696 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.531687    2567 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 04:39:44.531730 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.531715    2567 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 04:39:44.531760 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.531730    2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 04:39:44.531760 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.531745    2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 04:39:44.531760 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.531753    2567 kubelet.go:398] "Adding apiserver pod source"
Sep 11 04:39:44.531837 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.531763    2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 04:39:44.532116 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.532104    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 04:39:44.532150 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.532123    2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 04:39:44.534499 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.534484    2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 04:39:44.536856 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.536842    2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 04:39:44.536900 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.536874    2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 04:39:44.538842 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.538830    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 04:39:44.538912 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.538847    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 04:39:44.538912 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.538853    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 04:39:44.538912 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.538859    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 04:39:44.538912 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.538867    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 04:39:44.538912 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.538872    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 04:39:44.538912 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.538885    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 04:39:44.538912 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.538890    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 04:39:44.538912 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.538896    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 04:39:44.538912 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.538901    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 04:39:44.538912 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.538914    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 04:39:44.539166 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.538919    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 04:39:44.539850 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.539841    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 04:39:44.539850 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.539850    2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 04:39:44.543607 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.543594    2567 server.go:1268] "Started kubelet"
Sep 11 04:39:44.543744 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.543673    2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 04:39:44.543855 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:44.543798    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 04:39:44.543855 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.543846    2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-18.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 04:39:44.543990 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.543845    2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 04:39:44.543990 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.543905    2567 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 04:39:44.544138 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:44.543966    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-18.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 04:39:44.544490 ip-10-0-128-18 systemd[1]: Started Kubernetes Kubelet.
Sep 11 04:39:44.547011 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.546986    2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 04:39:44.548518 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:44.548477    2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 04:39:44.550394 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.550379    2567 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 04:39:44.552021 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:44.550882    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-18.ec2.internal.18d42a697dcee9d0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-18.ec2.internal,UID:ip-10-0-128-18.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-18.ec2.internal,},FirstTimestamp:2026-09-11 04:39:44.54357448 +0000 UTC m=+0.399767050,LastTimestamp:2026-09-11 04:39:44.54357448 +0000 UTC m=+0.399767050,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-18.ec2.internal,}"
Sep 11 04:39:44.552937 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.552911    2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 04:39:44.552937 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.552923    2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 04:39:44.553066 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:44.552979    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-18.ec2.internal\" not found"
Sep 11 04:39:44.553690 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.553669    2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 04:39:44.553690 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.553671    2567 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 04:39:44.553805 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.553704    2567 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 04:39:44.553805 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.553751    2567 factory.go:55] Registering systemd factory
Sep 11 04:39:44.553805 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.553764    2567 factory.go:223] Registration of the systemd container factory successfully
Sep 11 04:39:44.553805 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.553796    2567 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 04:39:44.553805 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.553807    2567 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 04:39:44.554005 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.553992    2567 factory.go:153] Registering CRI-O factory
Sep 11 04:39:44.554048 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.554011    2567 factory.go:223] Registration of the crio container factory successfully
Sep 11 04:39:44.554093 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.554062    2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 04:39:44.554093 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.554079    2567 factory.go:103] Registering Raw factory
Sep 11 04:39:44.554093 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.554092    2567 manager.go:1196] Started watching for new ooms in manager
Sep 11 04:39:44.554575 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.554562    2567 manager.go:349] Starting recovery of all containers
Sep 11 04:39:44.561780 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:44.561751    2567 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 04:39:44.561780 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:44.561772    2567 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-18.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 04:39:44.565784 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.565691    2567 manager.go:354] Recovery completed
Sep 11 04:39:44.568288 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.568268    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hsjpb"
Sep 11 04:39:44.569739 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.569727    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 04:39:44.572009 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.571995    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-18.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 04:39:44.572072 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.572021    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-18.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 04:39:44.572072 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.572031    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-18.ec2.internal" event="NodeHasSufficientPID"
Sep 11 04:39:44.572611 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.572597    2567 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 04:39:44.572611 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.572610    2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 04:39:44.572736 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.572616    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 04:39:44.572736 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.572629    2567 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 04:39:44.573415 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.573398    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-18.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 04:39:44.573466 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.573425    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-18.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 04:39:44.573466 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.573434    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-18.ec2.internal" event="NodeHasSufficientPID"
Sep 11 04:39:44.574877 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.574861    2567 policy_none.go:49] "None policy: Start"
Sep 11 04:39:44.574955 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.574881    2567 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 04:39:44.574955 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.574893    2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 04:39:44.575903 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.575886    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hsjpb"
Sep 11 04:39:44.577208 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.577194    2567 policy_none.go:47] "Start"
Sep 11 04:39:44.581331 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:44.581238    2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-18.ec2.internal.18d42a697f80c8aa  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-18.ec2.internal,UID:ip-10-0-128-18.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-18.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-18.ec2.internal,},FirstTimestamp:2026-09-11 04:39:44.572008618 +0000 UTC m=+0.428201188,LastTimestamp:2026-09-11 04:39:44.572008618 +0000 UTC m=+0.428201188,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-18.ec2.internal,}"
Sep 11 04:39:44.610970 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.610954    2567 manager.go:337] "Starting Device Plugin manager"
Sep 11 04:39:44.635640 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:44.610996    2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 04:39:44.635640 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.611005    2567 server.go:87] "Starting device plugin registration server"
Sep 11 04:39:44.635640 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.611298    2567 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 04:39:44.635640 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.611313    2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 04:39:44.635640 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.611413    2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 04:39:44.635640 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.611478    2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 04:39:44.635640 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.611486    2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 04:39:44.635640 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:44.612093    2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 04:39:44.635640 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:44.612135    2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-18.ec2.internal\" not found"
Sep 11 04:39:44.635640 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.624732    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 04:39:44.635640 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.626048    2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 04:39:44.635640 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.626089    2567 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 04:39:44.635640 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.626111    2567 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 04:39:44.635640 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:44.626146    2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 04:39:44.635640 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.627788    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 04:39:44.711624 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.711603    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 04:39:44.712545 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.712530    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-18.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 04:39:44.712603 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.712559    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-18.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 04:39:44.712603 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.712570    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-18.ec2.internal" event="NodeHasSufficientPID"
Sep 11 04:39:44.712603 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.712592    2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-18.ec2.internal"
Sep 11 04:39:44.721892 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.721826    2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-18.ec2.internal"
Sep 11 04:39:44.721892 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:44.721848    2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-18.ec2.internal\": node \"ip-10-0-128-18.ec2.internal\" not found"
Sep 11 04:39:44.726469 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.726454    2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-18.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-18.ec2.internal"]
Sep 11 04:39:44.726525 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.726514    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 04:39:44.727902 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.727878    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-18.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 04:39:44.727964 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.727907    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-18.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 04:39:44.727964 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.727916    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-18.ec2.internal" event="NodeHasSufficientPID"
Sep 11 04:39:44.730199 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.730187    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 04:39:44.730313 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.730300    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-18.ec2.internal"
Sep 11 04:39:44.730371 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.730327    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 04:39:44.730836 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.730822    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-18.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 04:39:44.730886 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.730842    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-18.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 04:39:44.730886 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.730850    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-18.ec2.internal" event="NodeHasSufficientPID"
Sep 11 04:39:44.730953 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.730919    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-18.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 04:39:44.730953 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.730937    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-18.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 04:39:44.730953 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.730946    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-18.ec2.internal" event="NodeHasSufficientPID"
Sep 11 04:39:44.733031 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.733018    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-18.ec2.internal"
Sep 11 04:39:44.733087 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.733040    2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 04:39:44.733702 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.733688    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-18.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 04:39:44.733775 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.733716    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-18.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 04:39:44.733775 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.733731    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-18.ec2.internal" event="NodeHasSufficientPID"
Sep 11 04:39:44.752878 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:44.752858    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-18.ec2.internal\" not found" node="ip-10-0-128-18.ec2.internal"
Sep 11 04:39:44.754621 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.754519    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/26845c0b3b67ab14283ca3124672441d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-18.ec2.internal\" (UID: \"26845c0b3b67ab14283ca3124672441d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-18.ec2.internal"
Sep 11 04:39:44.754621 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.754541    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26845c0b3b67ab14283ca3124672441d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-18.ec2.internal\" (UID: \"26845c0b3b67ab14283ca3124672441d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-18.ec2.internal"
Sep 11 04:39:44.754621 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.754557    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/81eb68d3a2c532023a6d1abc2053e6b2-config\") pod \"kube-apiserver-proxy-ip-10-0-128-18.ec2.internal\" (UID: \"81eb68d3a2c532023a6d1abc2053e6b2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-18.ec2.internal"
Sep 11 04:39:44.755986 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:44.755961    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-18.ec2.internal\" not found"
Sep 11 04:39:44.756261 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:44.756245    2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-18.ec2.internal\" not found" node="ip-10-0-128-18.ec2.internal"
Sep 11 04:39:44.855651 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.855614    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26845c0b3b67ab14283ca3124672441d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-18.ec2.internal\" (UID: \"26845c0b3b67ab14283ca3124672441d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-18.ec2.internal"
Sep 11 04:39:44.855651 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.855646    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/81eb68d3a2c532023a6d1abc2053e6b2-config\") pod \"kube-apiserver-proxy-ip-10-0-128-18.ec2.internal\" (UID: \"81eb68d3a2c532023a6d1abc2053e6b2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-18.ec2.internal"
Sep 11 04:39:44.855833 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.855664    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/26845c0b3b67ab14283ca3124672441d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-18.ec2.internal\" (UID: \"26845c0b3b67ab14283ca3124672441d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-18.ec2.internal"
Sep 11 04:39:44.855833 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.855713    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/81eb68d3a2c532023a6d1abc2053e6b2-config\") pod \"kube-apiserver-proxy-ip-10-0-128-18.ec2.internal\" (UID: \"81eb68d3a2c532023a6d1abc2053e6b2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-18.ec2.internal"
Sep 11 04:39:44.855833 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.855718    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/26845c0b3b67ab14283ca3124672441d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-18.ec2.internal\" (UID: \"26845c0b3b67ab14283ca3124672441d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-18.ec2.internal"
Sep 11 04:39:44.855833 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:44.855752    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/26845c0b3b67ab14283ca3124672441d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-18.ec2.internal\" (UID: \"26845c0b3b67ab14283ca3124672441d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-18.ec2.internal"
Sep 11 04:39:44.856657 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:44.856640    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-18.ec2.internal\" not found"
Sep 11 04:39:44.957105 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:44.957076    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-18.ec2.internal\" not found"
Sep 11 04:39:45.055697 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.055631    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-18.ec2.internal"
Sep 11 04:39:45.057778 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:45.057760    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-18.ec2.internal\" not found"
Sep 11 04:39:45.061030 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.061010    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-18.ec2.internal"
Sep 11 04:39:45.158738 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:45.158708    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-18.ec2.internal\" not found"
Sep 11 04:39:45.259124 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:45.259102    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-18.ec2.internal\" not found"
Sep 11 04:39:45.359654 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:45.359590    2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-18.ec2.internal\" not found"
Sep 11 04:39:45.381170 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.381151    2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 04:39:45.453201 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.453171    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-18.ec2.internal"
Sep 11 04:39:45.463023 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.463006    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 04:39:45.464984 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.464970    2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-18.ec2.internal"
Sep 11 04:39:45.475519 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.475501    2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 04:39:45.485637 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.485621    2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 04:39:45.485739 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.485722    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 04:39:45.485789 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.485771    2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 04:39:45.532935 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.532913    2567 apiserver.go:52] "Watching apiserver"
Sep 11 04:39:45.551830 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.551802    2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 04:39:45.552251 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.552218    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-7thkg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-18.ec2.internal","openshift-multus/multus-zs45f","openshift-multus/network-metrics-daemon-dfv69","kube-system/global-pull-secret-syncer-9zqb2","kube-system/konnectivity-agent-2lz5t","openshift-multus/multus-additional-cni-plugins-p8wsn","openshift-network-diagnostics/network-check-target-nbqjz","openshift-network-operator/iptables-alerter-4d47f","openshift-ovn-kubernetes/ovnkube-node-x8zh9","kube-system/kube-apiserver-proxy-ip-10-0-128-18.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz","openshift-cluster-node-tuning-operator/tuned-hjkmf","openshift-dns/node-resolver-clstx"]
Sep 11 04:39:45.553109 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.553087    2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 04:39:45.554708 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.554688    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7thkg"
Sep 11 04:39:45.556828 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.556805    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.557022 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.556935    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9mqhq\""
Sep 11 04:39:45.557022 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.556953    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 04:39:45.557352 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.556947    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 04:39:45.559155 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.557995    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 04:39:45.559285 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.559244    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:39:45.560057 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.559870    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57e4b5b7-b3cc-4052-be3c-07a138c81e91-host\") pod \"node-ca-7thkg\" (UID: \"57e4b5b7-b3cc-4052-be3c-07a138c81e91\") " pod="openshift-image-registry/node-ca-7thkg"
Sep 11 04:39:45.560057 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:45.559545    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfv69" podUID="0dc2bea8-03da-45f3-ac2b-d428b89a7542"
Sep 11 04:39:45.560057 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.559913    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/57e4b5b7-b3cc-4052-be3c-07a138c81e91-serviceca\") pod \"node-ca-7thkg\" (UID: \"57e4b5b7-b3cc-4052-be3c-07a138c81e91\") " pod="openshift-image-registry/node-ca-7thkg"
Sep 11 04:39:45.560057 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.559575    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 04:39:45.560057 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.559938    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whvgt\" (UniqueName: \"kubernetes.io/projected/57e4b5b7-b3cc-4052-be3c-07a138c81e91-kube-api-access-whvgt\") pod \"node-ca-7thkg\" (UID: \"57e4b5b7-b3cc-4052-be3c-07a138c81e91\") " pod="openshift-image-registry/node-ca-7thkg"
Sep 11 04:39:45.560057 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.559928    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 04:39:45.560057 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.560044    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 04:39:45.560057 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.560052    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mk4mp\""
Sep 11 04:39:45.560445 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.560077    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 04:39:45.561263 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.561244    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:39:45.561365 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:45.561319    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zqb2" podUID="dbc886e3-0aab-4149-87f3-1d4ec91672f9"
Sep 11 04:39:45.562901 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.562879    2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 04:39:45.563418 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.563402    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2lz5t"
Sep 11 04:39:45.565621 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.565602    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 04:39:45.565853 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.565836    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p8wsn"
Sep 11 04:39:45.566135 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.566120    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 04:39:45.566252 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.566236    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mcmpx\""
Sep 11 04:39:45.567989 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.567972    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:39:45.568059 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:45.568033    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbqjz" podUID="ae3ae47e-fe4b-4298-a2fd-5935514af075"
Sep 11 04:39:45.568502 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.568485    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 04:39:45.568727 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.568713    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pbb6b\""
Sep 11 04:39:45.569636 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.569447    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 04:39:45.570748 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.570731    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4d47f"
Sep 11 04:39:45.572841 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.572823    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 04:39:45.572956 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.572868    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 04:39:45.573024 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.573003    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-l95dm\""
Sep 11 04:39:45.573150 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.573117    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.573220 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.573174    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 04:39:45.575406 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.575392    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz"
Sep 11 04:39:45.575919 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.575899    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 04:39:45.575919 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.575909    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wtcd4\""
Sep 11 04:39:45.576448 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.576431    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 04:39:45.576554 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.576510    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 04:39:45.576621 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.576563    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 04:39:45.576621 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.576574    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 04:39:45.576621 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.576611    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 04:39:45.577867 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.577819    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 04:39:45.577867 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.577846    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 04:39:45.578074 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.578044    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gc6gm\""
Sep 11 04:39:45.578168 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.578154    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.579352 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.579317    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 04:39:45.580518 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.580499    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-clstx"
Sep 11 04:39:45.580625 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.580559    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 04:39:45.580625 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.580599    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-m7hp2\""
Sep 11 04:39:45.580757 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.580725    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 04:39:45.582603 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.582537    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 04:34:45 +0000 UTC" deadline="2028-05-21 08:20:52.284731158 +0000 UTC"
Sep 11 04:39:45.582603 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.582585    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14835h41m6.702150504s"
Sep 11 04:39:45.583110 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.583093    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 04:39:45.583214 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.583138    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 04:39:45.583293 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.583276    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qlqh8\""
Sep 11 04:39:45.596645 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.596629    2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8b88n"
Sep 11 04:39:45.605787 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.605766    2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8b88n"
Sep 11 04:39:45.611626 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.611580    2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 04:39:45.654514 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:45.654488    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81eb68d3a2c532023a6d1abc2053e6b2.slice/crio-894d7cbb1d63f621dbe51215144c1ae5be42a077d9df8590f4382b0ae80a8e8c WatchSource:0}: Error finding container 894d7cbb1d63f621dbe51215144c1ae5be42a077d9df8590f4382b0ae80a8e8c: Status 404 returned error can't find the container with id 894d7cbb1d63f621dbe51215144c1ae5be42a077d9df8590f4382b0ae80a8e8c
Sep 11 04:39:45.654890 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.654873    2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 04:39:45.659484 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.659468    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 04:39:45.660375 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.660354    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-etc-kubernetes\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.660452 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.660386    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flls2\" (UniqueName: \"kubernetes.io/projected/078c1f6d-dc41-44ae-9828-96baf9124176-kube-api-access-flls2\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.660452 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.660403    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-sys\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.660452 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.660418    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-var-lib-kubelet\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.660452 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.660432    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53a003c6-2a55-4b7c-9242-766116f41b2a-tmp-dir\") pod \"node-resolver-clstx\" (UID: \"53a003c6-2a55-4b7c-9242-766116f41b2a\") " pod="openshift-dns/node-resolver-clstx"
Sep 11 04:39:45.660608 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.660515    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dbc886e3-0aab-4149-87f3-1d4ec91672f9-kubelet-config\") pod \"global-pull-secret-syncer-9zqb2\" (UID: \"dbc886e3-0aab-4149-87f3-1d4ec91672f9\") " pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:39:45.660608 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.660555    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-hostroot\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.660608 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.660580    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-multus-conf-dir\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.660724 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.660608    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-run-systemd\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.660724 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.660637    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-ovn-node-metrics-cert\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.660724 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.660660    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-etc-modprobe-d\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.660724 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.660684    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d5d3232f-f007-4c0d-976f-80397eb71a4b-agent-certs\") pod \"konnectivity-agent-2lz5t\" (UID: \"d5d3232f-f007-4c0d-976f-80397eb71a4b\") " pod="kube-system/konnectivity-agent-2lz5t"
Sep 11 04:39:45.660724 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.660707    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-host-var-lib-kubelet\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.660960 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.660735    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57e4b5b7-b3cc-4052-be3c-07a138c81e91-host\") pod \"node-ca-7thkg\" (UID: \"57e4b5b7-b3cc-4052-be3c-07a138c81e91\") " pod="openshift-image-registry/node-ca-7thkg"
Sep 11 04:39:45.660960 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.660782    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-etc-sysctl-d\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.660960 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.660794    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57e4b5b7-b3cc-4052-be3c-07a138c81e91-host\") pod \"node-ca-7thkg\" (UID: \"57e4b5b7-b3cc-4052-be3c-07a138c81e91\") " pod="openshift-image-registry/node-ca-7thkg"
Sep 11 04:39:45.660960 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.660823    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d07dca5c-5fab-4523-bc76-884d85f79afe-tmp\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.660960 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.660847    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7b76\" (UniqueName: \"kubernetes.io/projected/53a003c6-2a55-4b7c-9242-766116f41b2a-kube-api-access-p7b76\") pod \"node-resolver-clstx\" (UID: \"53a003c6-2a55-4b7c-9242-766116f41b2a\") " pod="openshift-dns/node-resolver-clstx"
Sep 11 04:39:45.660960 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.660873    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh7kw\" (UniqueName: \"kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw\") pod \"network-check-target-nbqjz\" (UID: \"ae3ae47e-fe4b-4298-a2fd-5935514af075\") " pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:39:45.660960 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.660897    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2133f096-0741-459c-a579-1742c273f6f0-host-slash\") pod \"iptables-alerter-4d47f\" (UID: \"2133f096-0741-459c-a579-1742c273f6f0\") " pod="openshift-network-operator/iptables-alerter-4d47f"
Sep 11 04:39:45.660960 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.660929    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-system-cni-dir\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.661320 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.660967    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-ovnkube-script-lib\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.661320 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661002    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dbc886e3-0aab-4149-87f3-1d4ec91672f9-dbus\") pod \"global-pull-secret-syncer-9zqb2\" (UID: \"dbc886e3-0aab-4149-87f3-1d4ec91672f9\") " pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:39:45.661320 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661029    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/078c1f6d-dc41-44ae-9828-96baf9124176-multus-daemon-config\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.661320 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661055    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-ovnkube-config\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.661320 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661083    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5232e65a-4484-4196-bb63-6fd0840e83e0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p8wsn\" (UID: \"5232e65a-4484-4196-bb63-6fd0840e83e0\") " pod="openshift-multus/multus-additional-cni-plugins-p8wsn"
Sep 11 04:39:45.661320 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661111    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs\") pod \"network-metrics-daemon-dfv69\" (UID: \"0dc2bea8-03da-45f3-ac2b-d428b89a7542\") " pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:39:45.661320 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661135    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-host-cni-netd\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.661320 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661159    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-env-overrides\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.661320 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661182    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-lib-modules\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.661320 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661205    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d07dca5c-5fab-4523-bc76-884d85f79afe-etc-tuned\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.661320 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661226    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/53a003c6-2a55-4b7c-9242-766116f41b2a-hosts-file\") pod \"node-resolver-clstx\" (UID: \"53a003c6-2a55-4b7c-9242-766116f41b2a\") " pod="openshift-dns/node-resolver-clstx"
Sep 11 04:39:45.661320 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661249    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5232e65a-4484-4196-bb63-6fd0840e83e0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p8wsn\" (UID: \"5232e65a-4484-4196-bb63-6fd0840e83e0\") " pod="openshift-multus/multus-additional-cni-plugins-p8wsn"
Sep 11 04:39:45.661320 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661278    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7b5102b3-adf5-4fce-a49a-474eec1ad621-registration-dir\") pod \"aws-ebs-csi-driver-node-l8kvz\" (UID: \"7b5102b3-adf5-4fce-a49a-474eec1ad621\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz"
Sep 11 04:39:45.661320 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661301    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-host-var-lib-cni-bin\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.661908 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661327    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.661908 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661372    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-etc-kubernetes\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.661908 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661408    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2133f096-0741-459c-a579-1742c273f6f0-iptables-alerter-script\") pod \"iptables-alerter-4d47f\" (UID: \"2133f096-0741-459c-a579-1742c273f6f0\") " pod="openshift-network-operator/iptables-alerter-4d47f"
Sep 11 04:39:45.661908 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661432    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7b5102b3-adf5-4fce-a49a-474eec1ad621-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l8kvz\" (UID: \"7b5102b3-adf5-4fce-a49a-474eec1ad621\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz"
Sep 11 04:39:45.661908 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661456    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7b5102b3-adf5-4fce-a49a-474eec1ad621-etc-selinux\") pod \"aws-ebs-csi-driver-node-l8kvz\" (UID: \"7b5102b3-adf5-4fce-a49a-474eec1ad621\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz"
Sep 11 04:39:45.661908 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661486    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/078c1f6d-dc41-44ae-9828-96baf9124176-cni-binary-copy\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.661908 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661520    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-node-log\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.661908 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661545    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-host-cni-bin\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.661908 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661569    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-etc-sysconfig\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.661908 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661593    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7b5102b3-adf5-4fce-a49a-474eec1ad621-socket-dir\") pod \"aws-ebs-csi-driver-node-l8kvz\" (UID: \"7b5102b3-adf5-4fce-a49a-474eec1ad621\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz"
Sep 11 04:39:45.661908 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661621    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7b5102b3-adf5-4fce-a49a-474eec1ad621-sys-fs\") pod \"aws-ebs-csi-driver-node-l8kvz\" (UID: \"7b5102b3-adf5-4fce-a49a-474eec1ad621\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz"
Sep 11 04:39:45.661908 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661660    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-host-run-k8s-cni-cncf-io\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.661908 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661759    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-host-run-ovn-kubernetes\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.661908 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661790    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5232e65a-4484-4196-bb63-6fd0840e83e0-cnibin\") pod \"multus-additional-cni-plugins-p8wsn\" (UID: \"5232e65a-4484-4196-bb63-6fd0840e83e0\") " pod="openshift-multus/multus-additional-cni-plugins-p8wsn"
Sep 11 04:39:45.661908 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661820    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5232e65a-4484-4196-bb63-6fd0840e83e0-cni-binary-copy\") pod \"multus-additional-cni-plugins-p8wsn\" (UID: \"5232e65a-4484-4196-bb63-6fd0840e83e0\") " pod="openshift-multus/multus-additional-cni-plugins-p8wsn"
Sep 11 04:39:45.661908 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661843    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qzxj\" (UniqueName: \"kubernetes.io/projected/5232e65a-4484-4196-bb63-6fd0840e83e0-kube-api-access-7qzxj\") pod \"multus-additional-cni-plugins-p8wsn\" (UID: \"5232e65a-4484-4196-bb63-6fd0840e83e0\") " pod="openshift-multus/multus-additional-cni-plugins-p8wsn"
Sep 11 04:39:45.662396 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661864    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4nwh\" (UniqueName: \"kubernetes.io/projected/7b5102b3-adf5-4fce-a49a-474eec1ad621-kube-api-access-j4nwh\") pod \"aws-ebs-csi-driver-node-l8kvz\" (UID: \"7b5102b3-adf5-4fce-a49a-474eec1ad621\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz"
Sep 11 04:39:45.662396 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661886    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-multus-cni-dir\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.662396 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661928    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-host-kubelet\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.662396 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661966    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-etc-sysctl-conf\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.662396 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.661996    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-etc-systemd\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.662396 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662034    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q46fp\" (UniqueName: \"kubernetes.io/projected/2133f096-0741-459c-a579-1742c273f6f0-kube-api-access-q46fp\") pod \"iptables-alerter-4d47f\" (UID: \"2133f096-0741-459c-a579-1742c273f6f0\") " pod="openshift-network-operator/iptables-alerter-4d47f"
Sep 11 04:39:45.662396 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662070    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7b5102b3-adf5-4fce-a49a-474eec1ad621-device-dir\") pod \"aws-ebs-csi-driver-node-l8kvz\" (UID: \"7b5102b3-adf5-4fce-a49a-474eec1ad621\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz"
Sep 11 04:39:45.662396 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662100    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-multus-socket-dir-parent\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.662396 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662123    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-run-ovn\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.662396 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662147    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5vqh\" (UniqueName: \"kubernetes.io/projected/d07dca5c-5fab-4523-bc76-884d85f79afe-kube-api-access-h5vqh\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.662396 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662188    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-cnibin\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.662396 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662211    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-host-run-multus-certs\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.662396 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662242    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-systemd-units\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.662396 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662274    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret\") pod \"global-pull-secret-syncer-9zqb2\" (UID: \"dbc886e3-0aab-4149-87f3-1d4ec91672f9\") " pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:39:45.662396 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662300    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5232e65a-4484-4196-bb63-6fd0840e83e0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p8wsn\" (UID: \"5232e65a-4484-4196-bb63-6fd0840e83e0\") " pod="openshift-multus/multus-additional-cni-plugins-p8wsn"
Sep 11 04:39:45.662396 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662333    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-host-slash\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.662939 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662375    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-host-run-netns\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.662939 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662389    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-var-lib-openvswitch\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.662939 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662403    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-run-openvswitch\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.662939 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662420    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whvgt\" (UniqueName: \"kubernetes.io/projected/57e4b5b7-b3cc-4052-be3c-07a138c81e91-kube-api-access-whvgt\") pod \"node-ca-7thkg\" (UID: \"57e4b5b7-b3cc-4052-be3c-07a138c81e91\") " pod="openshift-image-registry/node-ca-7thkg"
Sep 11 04:39:45.662939 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662434    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49725\" (UniqueName: \"kubernetes.io/projected/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-kube-api-access-49725\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.662939 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662470    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d5d3232f-f007-4c0d-976f-80397eb71a4b-konnectivity-ca\") pod \"konnectivity-agent-2lz5t\" (UID: \"d5d3232f-f007-4c0d-976f-80397eb71a4b\") " pod="kube-system/konnectivity-agent-2lz5t"
Sep 11 04:39:45.662939 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662495    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-host-var-lib-cni-multus\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.662939 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662520    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9fhf\" (UniqueName: \"kubernetes.io/projected/0dc2bea8-03da-45f3-ac2b-d428b89a7542-kube-api-access-g9fhf\") pod \"network-metrics-daemon-dfv69\" (UID: \"0dc2bea8-03da-45f3-ac2b-d428b89a7542\") " pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:39:45.662939 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662545    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-run\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.662939 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662560    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-host\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.662939 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662573    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-etc-openvswitch\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.662939 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662586    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-log-socket\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.662939 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662606    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5232e65a-4484-4196-bb63-6fd0840e83e0-system-cni-dir\") pod \"multus-additional-cni-plugins-p8wsn\" (UID: \"5232e65a-4484-4196-bb63-6fd0840e83e0\") " pod="openshift-multus/multus-additional-cni-plugins-p8wsn"
Sep 11 04:39:45.662939 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662634    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/57e4b5b7-b3cc-4052-be3c-07a138c81e91-serviceca\") pod \"node-ca-7thkg\" (UID: \"57e4b5b7-b3cc-4052-be3c-07a138c81e91\") " pod="openshift-image-registry/node-ca-7thkg"
Sep 11 04:39:45.662939 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.662659    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-host-run-netns\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.663521 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.663099    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/57e4b5b7-b3cc-4052-be3c-07a138c81e91-serviceca\") pod \"node-ca-7thkg\" (UID: \"57e4b5b7-b3cc-4052-be3c-07a138c81e91\") " pod="openshift-image-registry/node-ca-7thkg"
Sep 11 04:39:45.669530 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.669510    2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 04:39:45.673220 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.673197    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whvgt\" (UniqueName: \"kubernetes.io/projected/57e4b5b7-b3cc-4052-be3c-07a138c81e91-kube-api-access-whvgt\") pod \"node-ca-7thkg\" (UID: \"57e4b5b7-b3cc-4052-be3c-07a138c81e91\") " pod="openshift-image-registry/node-ca-7thkg"
Sep 11 04:39:45.702226 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:45.702044    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26845c0b3b67ab14283ca3124672441d.slice/crio-60b551fcb1da64aa0a364f28a2bfd29b36cc4c7cd55298bbc65786c917a8dba6 WatchSource:0}: Error finding container 60b551fcb1da64aa0a364f28a2bfd29b36cc4c7cd55298bbc65786c917a8dba6: Status 404 returned error can't find the container with id 60b551fcb1da64aa0a364f28a2bfd29b36cc4c7cd55298bbc65786c917a8dba6
Sep 11 04:39:45.763052 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763017    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-host-cni-bin\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.763052 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763051    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-etc-sysconfig\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.763188 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763068    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7b5102b3-adf5-4fce-a49a-474eec1ad621-socket-dir\") pod \"aws-ebs-csi-driver-node-l8kvz\" (UID: \"7b5102b3-adf5-4fce-a49a-474eec1ad621\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz"
Sep 11 04:39:45.763188 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763091    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7b5102b3-adf5-4fce-a49a-474eec1ad621-sys-fs\") pod \"aws-ebs-csi-driver-node-l8kvz\" (UID: \"7b5102b3-adf5-4fce-a49a-474eec1ad621\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz"
Sep 11 04:39:45.763188 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763106    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-host-run-k8s-cni-cncf-io\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.763188 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763110    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-host-cni-bin\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.763188 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763157    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-host-run-k8s-cni-cncf-io\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.763188 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763159    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-etc-sysconfig\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.763188 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763174    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7b5102b3-adf5-4fce-a49a-474eec1ad621-sys-fs\") pod \"aws-ebs-csi-driver-node-l8kvz\" (UID: \"7b5102b3-adf5-4fce-a49a-474eec1ad621\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz"
Sep 11 04:39:45.763522 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763218    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-host-run-ovn-kubernetes\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.763522 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763245    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7b5102b3-adf5-4fce-a49a-474eec1ad621-socket-dir\") pod \"aws-ebs-csi-driver-node-l8kvz\" (UID: \"7b5102b3-adf5-4fce-a49a-474eec1ad621\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz"
Sep 11 04:39:45.763522 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763253    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5232e65a-4484-4196-bb63-6fd0840e83e0-cnibin\") pod \"multus-additional-cni-plugins-p8wsn\" (UID: \"5232e65a-4484-4196-bb63-6fd0840e83e0\") " pod="openshift-multus/multus-additional-cni-plugins-p8wsn"
Sep 11 04:39:45.763522 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763256    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-host-run-ovn-kubernetes\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.763522 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763280    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5232e65a-4484-4196-bb63-6fd0840e83e0-cni-binary-copy\") pod \"multus-additional-cni-plugins-p8wsn\" (UID: \"5232e65a-4484-4196-bb63-6fd0840e83e0\") " pod="openshift-multus/multus-additional-cni-plugins-p8wsn"
Sep 11 04:39:45.763522 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763282    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5232e65a-4484-4196-bb63-6fd0840e83e0-cnibin\") pod \"multus-additional-cni-plugins-p8wsn\" (UID: \"5232e65a-4484-4196-bb63-6fd0840e83e0\") " pod="openshift-multus/multus-additional-cni-plugins-p8wsn"
Sep 11 04:39:45.763522 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763375    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qzxj\" (UniqueName: \"kubernetes.io/projected/5232e65a-4484-4196-bb63-6fd0840e83e0-kube-api-access-7qzxj\") pod \"multus-additional-cni-plugins-p8wsn\" (UID: \"5232e65a-4484-4196-bb63-6fd0840e83e0\") " pod="openshift-multus/multus-additional-cni-plugins-p8wsn"
Sep 11 04:39:45.763522 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763415    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4nwh\" (UniqueName: \"kubernetes.io/projected/7b5102b3-adf5-4fce-a49a-474eec1ad621-kube-api-access-j4nwh\") pod \"aws-ebs-csi-driver-node-l8kvz\" (UID: \"7b5102b3-adf5-4fce-a49a-474eec1ad621\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz"
Sep 11 04:39:45.763522 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763447    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-multus-cni-dir\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.763522 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763472    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-host-kubelet\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.763522 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763497    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-etc-sysctl-conf\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.763927 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763524    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-etc-systemd\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.763927 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763835    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-host-kubelet\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.763927 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763862    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q46fp\" (UniqueName: \"kubernetes.io/projected/2133f096-0741-459c-a579-1742c273f6f0-kube-api-access-q46fp\") pod \"iptables-alerter-4d47f\" (UID: \"2133f096-0741-459c-a579-1742c273f6f0\") " pod="openshift-network-operator/iptables-alerter-4d47f"
Sep 11 04:39:45.763927 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763904    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7b5102b3-adf5-4fce-a49a-474eec1ad621-device-dir\") pod \"aws-ebs-csi-driver-node-l8kvz\" (UID: \"7b5102b3-adf5-4fce-a49a-474eec1ad621\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz"
Sep 11 04:39:45.764151 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763938    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-multus-socket-dir-parent\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.764151 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763950    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-multus-cni-dir\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.764151 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763969    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-run-ovn\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.764151 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.763995    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5vqh\" (UniqueName: \"kubernetes.io/projected/d07dca5c-5fab-4523-bc76-884d85f79afe-kube-api-access-h5vqh\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.764151 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764027    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7b5102b3-adf5-4fce-a49a-474eec1ad621-device-dir\") pod \"aws-ebs-csi-driver-node-l8kvz\" (UID: \"7b5102b3-adf5-4fce-a49a-474eec1ad621\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz"
Sep 11 04:39:45.764151 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764040    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-cnibin\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.764151 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764078    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-host-run-multus-certs\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.764151 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764111    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-systemd-units\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.764151 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764143    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret\") pod \"global-pull-secret-syncer-9zqb2\" (UID: \"dbc886e3-0aab-4149-87f3-1d4ec91672f9\") " pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:39:45.764626 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764176    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5232e65a-4484-4196-bb63-6fd0840e83e0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p8wsn\" (UID: \"5232e65a-4484-4196-bb63-6fd0840e83e0\") " pod="openshift-multus/multus-additional-cni-plugins-p8wsn"
Sep 11 04:39:45.764626 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764201    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-host-slash\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.764626 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764206    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-run-ovn\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.764626 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764230    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-host-run-netns\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.764626 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764261    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-var-lib-openvswitch\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.764626 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764263    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5232e65a-4484-4196-bb63-6fd0840e83e0-cni-binary-copy\") pod \"multus-additional-cni-plugins-p8wsn\" (UID: \"5232e65a-4484-4196-bb63-6fd0840e83e0\") " pod="openshift-multus/multus-additional-cni-plugins-p8wsn"
Sep 11 04:39:45.764626 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764276    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-multus-socket-dir-parent\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.764626 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764291    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-run-openvswitch\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.764626 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764377    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-etc-sysctl-conf\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.764626 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764387    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49725\" (UniqueName: \"kubernetes.io/projected/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-kube-api-access-49725\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.764626 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764430    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d5d3232f-f007-4c0d-976f-80397eb71a4b-konnectivity-ca\") pod \"konnectivity-agent-2lz5t\" (UID: \"d5d3232f-f007-4c0d-976f-80397eb71a4b\") " pod="kube-system/konnectivity-agent-2lz5t"
Sep 11 04:39:45.764626 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764465    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-host-var-lib-cni-multus\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.764626 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764503    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9fhf\" (UniqueName: \"kubernetes.io/projected/0dc2bea8-03da-45f3-ac2b-d428b89a7542-kube-api-access-g9fhf\") pod \"network-metrics-daemon-dfv69\" (UID: \"0dc2bea8-03da-45f3-ac2b-d428b89a7542\") " pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:39:45.764626 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:45.764514    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 04:39:45.764626 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764528    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-run-openvswitch\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.764626 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764532    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-run\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.764626 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764579    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-host-slash\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.764626 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764585    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-run\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.765468 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:45.764612    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret podName:dbc886e3-0aab-4149-87f3-1d4ec91672f9 nodeName:}" failed. No retries permitted until 2026-09-11 04:39:46.26456698 +0000 UTC m=+2.120759560 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret") pod "global-pull-secret-syncer-9zqb2" (UID: "dbc886e3-0aab-4149-87f3-1d4ec91672f9") : object "kube-system"/"original-pull-secret" not registered
Sep 11 04:39:45.765468 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764641    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-host-run-netns\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.765468 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764710    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-var-lib-openvswitch\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.765468 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764758    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5232e65a-4484-4196-bb63-6fd0840e83e0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p8wsn\" (UID: \"5232e65a-4484-4196-bb63-6fd0840e83e0\") " pod="openshift-multus/multus-additional-cni-plugins-p8wsn"
Sep 11 04:39:45.765468 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764760    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-host-run-multus-certs\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.765468 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764792    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-host-var-lib-cni-multus\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.765468 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764818    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-host\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.765468 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764857    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-etc-openvswitch\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.765468 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764860    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-systemd-units\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.765468 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764878    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-log-socket\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.765468 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764922    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5232e65a-4484-4196-bb63-6fd0840e83e0-system-cni-dir\") pod \"multus-additional-cni-plugins-p8wsn\" (UID: \"5232e65a-4484-4196-bb63-6fd0840e83e0\") " pod="openshift-multus/multus-additional-cni-plugins-p8wsn"
Sep 11 04:39:45.765468 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764940    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-host\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.765468 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764999    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-log-socket\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.765468 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765027    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-etc-systemd\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.765468 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.764951    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-host-run-netns\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.765468 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765068    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-cnibin\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.765468 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765067    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-etc-openvswitch\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.766317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765095    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-etc-kubernetes\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.766317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765101    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-host-run-netns\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.766317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765116    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5232e65a-4484-4196-bb63-6fd0840e83e0-system-cni-dir\") pod \"multus-additional-cni-plugins-p8wsn\" (UID: \"5232e65a-4484-4196-bb63-6fd0840e83e0\") " pod="openshift-multus/multus-additional-cni-plugins-p8wsn"
Sep 11 04:39:45.766317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765138    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flls2\" (UniqueName: \"kubernetes.io/projected/078c1f6d-dc41-44ae-9828-96baf9124176-kube-api-access-flls2\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.766317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765154    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-etc-kubernetes\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.766317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765159    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d5d3232f-f007-4c0d-976f-80397eb71a4b-konnectivity-ca\") pod \"konnectivity-agent-2lz5t\" (UID: \"d5d3232f-f007-4c0d-976f-80397eb71a4b\") " pod="kube-system/konnectivity-agent-2lz5t"
Sep 11 04:39:45.766317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765178    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-sys\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.766317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765216    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-var-lib-kubelet\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.766317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765244    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53a003c6-2a55-4b7c-9242-766116f41b2a-tmp-dir\") pod \"node-resolver-clstx\" (UID: \"53a003c6-2a55-4b7c-9242-766116f41b2a\") " pod="openshift-dns/node-resolver-clstx"
Sep 11 04:39:45.766317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765274    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dbc886e3-0aab-4149-87f3-1d4ec91672f9-kubelet-config\") pod \"global-pull-secret-syncer-9zqb2\" (UID: \"dbc886e3-0aab-4149-87f3-1d4ec91672f9\") " pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:39:45.766317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765298    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-hostroot\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.766317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765327    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-multus-conf-dir\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.766317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765378    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-run-systemd\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.766317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765411    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-ovn-node-metrics-cert\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.766317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765440    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-etc-modprobe-d\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.766317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765464    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d5d3232f-f007-4c0d-976f-80397eb71a4b-agent-certs\") pod \"konnectivity-agent-2lz5t\" (UID: \"d5d3232f-f007-4c0d-976f-80397eb71a4b\") " pod="kube-system/konnectivity-agent-2lz5t"
Sep 11 04:39:45.766317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765472    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-sys\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.766317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765494    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-host-var-lib-kubelet\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.767171 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765526    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-var-lib-kubelet\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.767171 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765530    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-etc-sysctl-d\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.767171 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765558    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d07dca5c-5fab-4523-bc76-884d85f79afe-tmp\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.767171 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765580    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7b76\" (UniqueName: \"kubernetes.io/projected/53a003c6-2a55-4b7c-9242-766116f41b2a-kube-api-access-p7b76\") pod \"node-resolver-clstx\" (UID: \"53a003c6-2a55-4b7c-9242-766116f41b2a\") " pod="openshift-dns/node-resolver-clstx"
Sep 11 04:39:45.767171 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765608    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh7kw\" (UniqueName: \"kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw\") pod \"network-check-target-nbqjz\" (UID: \"ae3ae47e-fe4b-4298-a2fd-5935514af075\") " pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:39:45.767171 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765638    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2133f096-0741-459c-a579-1742c273f6f0-host-slash\") pod \"iptables-alerter-4d47f\" (UID: \"2133f096-0741-459c-a579-1742c273f6f0\") " pod="openshift-network-operator/iptables-alerter-4d47f"
Sep 11 04:39:45.767171 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765644    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-multus-conf-dir\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.767171 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765667    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-system-cni-dir\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.767171 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765696    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-ovnkube-script-lib\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.767171 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765719    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dbc886e3-0aab-4149-87f3-1d4ec91672f9-dbus\") pod \"global-pull-secret-syncer-9zqb2\" (UID: \"dbc886e3-0aab-4149-87f3-1d4ec91672f9\") " pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:39:45.767171 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765747    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/078c1f6d-dc41-44ae-9828-96baf9124176-multus-daemon-config\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.767171 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765776    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-ovnkube-config\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.767171 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765805    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5232e65a-4484-4196-bb63-6fd0840e83e0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p8wsn\" (UID: \"5232e65a-4484-4196-bb63-6fd0840e83e0\") " pod="openshift-multus/multus-additional-cni-plugins-p8wsn"
Sep 11 04:39:45.767171 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765823    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-hostroot\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.767171 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765833    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs\") pod \"network-metrics-daemon-dfv69\" (UID: \"0dc2bea8-03da-45f3-ac2b-d428b89a7542\") " pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:39:45.767171 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765880    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-host-cni-netd\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.767171 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765909    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-env-overrides\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.768211 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765960    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-lib-modules\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.768211 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.765981    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2133f096-0741-459c-a579-1742c273f6f0-host-slash\") pod \"iptables-alerter-4d47f\" (UID: \"2133f096-0741-459c-a579-1742c273f6f0\") " pod="openshift-network-operator/iptables-alerter-4d47f"
Sep 11 04:39:45.768211 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.766000    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d07dca5c-5fab-4523-bc76-884d85f79afe-etc-tuned\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.768211 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.766030    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/53a003c6-2a55-4b7c-9242-766116f41b2a-hosts-file\") pod \"node-resolver-clstx\" (UID: \"53a003c6-2a55-4b7c-9242-766116f41b2a\") " pod="openshift-dns/node-resolver-clstx"
Sep 11 04:39:45.768211 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.766064    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5232e65a-4484-4196-bb63-6fd0840e83e0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p8wsn\" (UID: \"5232e65a-4484-4196-bb63-6fd0840e83e0\") " pod="openshift-multus/multus-additional-cni-plugins-p8wsn"
Sep 11 04:39:45.768211 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.766081    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-system-cni-dir\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.768211 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.766116    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7b5102b3-adf5-4fce-a49a-474eec1ad621-registration-dir\") pod \"aws-ebs-csi-driver-node-l8kvz\" (UID: \"7b5102b3-adf5-4fce-a49a-474eec1ad621\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz"
Sep 11 04:39:45.768211 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.766149    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-host-var-lib-cni-bin\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.768211 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.766160    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-etc-modprobe-d\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.768211 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.766185    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.768211 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.766215    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-etc-kubernetes\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.768211 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.766245    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2133f096-0741-459c-a579-1742c273f6f0-iptables-alerter-script\") pod \"iptables-alerter-4d47f\" (UID: \"2133f096-0741-459c-a579-1742c273f6f0\") " pod="openshift-network-operator/iptables-alerter-4d47f"
Sep 11 04:39:45.768211 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.766279    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7b5102b3-adf5-4fce-a49a-474eec1ad621-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l8kvz\" (UID: \"7b5102b3-adf5-4fce-a49a-474eec1ad621\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz"
Sep 11 04:39:45.768211 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.766309    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7b5102b3-adf5-4fce-a49a-474eec1ad621-etc-selinux\") pod \"aws-ebs-csi-driver-node-l8kvz\" (UID: \"7b5102b3-adf5-4fce-a49a-474eec1ad621\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz"
Sep 11 04:39:45.768211 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.766333    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/078c1f6d-dc41-44ae-9828-96baf9124176-cni-binary-copy\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.768211 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.766392    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-node-log\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.768211 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.766454    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/53a003c6-2a55-4b7c-9242-766116f41b2a-tmp-dir\") pod \"node-resolver-clstx\" (UID: \"53a003c6-2a55-4b7c-9242-766116f41b2a\") " pod="openshift-dns/node-resolver-clstx"
Sep 11 04:39:45.769020 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.766517    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-run-systemd\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.769020 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.766578    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-node-log\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.769020 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.766605    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5232e65a-4484-4196-bb63-6fd0840e83e0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p8wsn\" (UID: \"5232e65a-4484-4196-bb63-6fd0840e83e0\") " pod="openshift-multus/multus-additional-cni-plugins-p8wsn"
Sep 11 04:39:45.769020 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.766663    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-lib-modules\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.769020 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.766667    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dbc886e3-0aab-4149-87f3-1d4ec91672f9-dbus\") pod \"global-pull-secret-syncer-9zqb2\" (UID: \"dbc886e3-0aab-4149-87f3-1d4ec91672f9\") " pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:39:45.769020 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.766693    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-host-var-lib-cni-bin\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.769020 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.766728    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/53a003c6-2a55-4b7c-9242-766116f41b2a-hosts-file\") pod \"node-resolver-clstx\" (UID: \"53a003c6-2a55-4b7c-9242-766116f41b2a\") " pod="openshift-dns/node-resolver-clstx"
Sep 11 04:39:45.769020 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.766805    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7b5102b3-adf5-4fce-a49a-474eec1ad621-registration-dir\") pod \"aws-ebs-csi-driver-node-l8kvz\" (UID: \"7b5102b3-adf5-4fce-a49a-474eec1ad621\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz"
Sep 11 04:39:45.769020 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:45.766895    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:39:45.769020 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:45.766955    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs podName:0dc2bea8-03da-45f3-ac2b-d428b89a7542 nodeName:}" failed. No retries permitted until 2026-09-11 04:39:46.26693356 +0000 UTC m=+2.123126121 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs") pod "network-metrics-daemon-dfv69" (UID: "0dc2bea8-03da-45f3-ac2b-d428b89a7542") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:39:45.769020 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.767049    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7b5102b3-adf5-4fce-a49a-474eec1ad621-etc-selinux\") pod \"aws-ebs-csi-driver-node-l8kvz\" (UID: \"7b5102b3-adf5-4fce-a49a-474eec1ad621\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz"
Sep 11 04:39:45.769020 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.767104    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7b5102b3-adf5-4fce-a49a-474eec1ad621-kubelet-dir\") pod \"aws-ebs-csi-driver-node-l8kvz\" (UID: \"7b5102b3-adf5-4fce-a49a-474eec1ad621\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz"
Sep 11 04:39:45.769020 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.767128    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/078c1f6d-dc41-44ae-9828-96baf9124176-multus-daemon-config\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.769020 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.767193    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-etc-sysctl-d\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.769020 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.767193    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2133f096-0741-459c-a579-1742c273f6f0-iptables-alerter-script\") pod \"iptables-alerter-4d47f\" (UID: \"2133f096-0741-459c-a579-1742c273f6f0\") " pod="openshift-network-operator/iptables-alerter-4d47f"
Sep 11 04:39:45.769020 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.767261    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/078c1f6d-dc41-44ae-9828-96baf9124176-host-var-lib-kubelet\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.769020 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.767280    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5232e65a-4484-4196-bb63-6fd0840e83e0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p8wsn\" (UID: \"5232e65a-4484-4196-bb63-6fd0840e83e0\") " pod="openshift-multus/multus-additional-cni-plugins-p8wsn"
Sep 11 04:39:45.769968 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.767372    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-host-cni-netd\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.769968 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.767668    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-ovnkube-script-lib\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.769968 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.767778    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-env-overrides\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.769968 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.767850    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.769968 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.767916    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/078c1f6d-dc41-44ae-9828-96baf9124176-cni-binary-copy\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.769968 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.767931    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d07dca5c-5fab-4523-bc76-884d85f79afe-etc-kubernetes\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.769968 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.768056    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dbc886e3-0aab-4149-87f3-1d4ec91672f9-kubelet-config\") pod \"global-pull-secret-syncer-9zqb2\" (UID: \"dbc886e3-0aab-4149-87f3-1d4ec91672f9\") " pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:39:45.769968 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.768108    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-ovnkube-config\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.769968 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.768552    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-ovn-node-metrics-cert\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.769968 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.769799    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d07dca5c-5fab-4523-bc76-884d85f79afe-tmp\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.770715 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.770419    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d07dca5c-5fab-4523-bc76-884d85f79afe-etc-tuned\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.771418 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.771394    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d5d3232f-f007-4c0d-976f-80397eb71a4b-agent-certs\") pod \"konnectivity-agent-2lz5t\" (UID: \"d5d3232f-f007-4c0d-976f-80397eb71a4b\") " pod="kube-system/konnectivity-agent-2lz5t"
Sep 11 04:39:45.772626 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.772599    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q46fp\" (UniqueName: \"kubernetes.io/projected/2133f096-0741-459c-a579-1742c273f6f0-kube-api-access-q46fp\") pod \"iptables-alerter-4d47f\" (UID: \"2133f096-0741-459c-a579-1742c273f6f0\") " pod="openshift-network-operator/iptables-alerter-4d47f"
Sep 11 04:39:45.772724 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.772661    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5vqh\" (UniqueName: \"kubernetes.io/projected/d07dca5c-5fab-4523-bc76-884d85f79afe-kube-api-access-h5vqh\") pod \"tuned-hjkmf\" (UID: \"d07dca5c-5fab-4523-bc76-884d85f79afe\") " pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.772783 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.772764    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4nwh\" (UniqueName: \"kubernetes.io/projected/7b5102b3-adf5-4fce-a49a-474eec1ad621-kube-api-access-j4nwh\") pod \"aws-ebs-csi-driver-node-l8kvz\" (UID: \"7b5102b3-adf5-4fce-a49a-474eec1ad621\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz"
Sep 11 04:39:45.778485 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.778468    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49725\" (UniqueName: \"kubernetes.io/projected/4ffd3e8f-56a8-4b04-96f3-ca0921820cbe-kube-api-access-49725\") pod \"ovnkube-node-x8zh9\" (UID: \"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe\") " pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.778863 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.778848    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qzxj\" (UniqueName: \"kubernetes.io/projected/5232e65a-4484-4196-bb63-6fd0840e83e0-kube-api-access-7qzxj\") pod \"multus-additional-cni-plugins-p8wsn\" (UID: \"5232e65a-4484-4196-bb63-6fd0840e83e0\") " pod="openshift-multus/multus-additional-cni-plugins-p8wsn"
Sep 11 04:39:45.779002 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.778988    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9fhf\" (UniqueName: \"kubernetes.io/projected/0dc2bea8-03da-45f3-ac2b-d428b89a7542-kube-api-access-g9fhf\") pod \"network-metrics-daemon-dfv69\" (UID: \"0dc2bea8-03da-45f3-ac2b-d428b89a7542\") " pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:39:45.780595 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:45.780580    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 04:39:45.780641 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:45.780599    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 04:39:45.780641 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:45.780608    2567 projected.go:196] Error preparing data for projected volume kube-api-access-wh7kw for pod openshift-network-diagnostics/network-check-target-nbqjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:39:45.780717 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:45.780657    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw podName:ae3ae47e-fe4b-4298-a2fd-5935514af075 nodeName:}" failed. No retries permitted until 2026-09-11 04:39:46.280645566 +0000 UTC m=+2.136838124 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wh7kw" (UniqueName: "kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw") pod "network-check-target-nbqjz" (UID: "ae3ae47e-fe4b-4298-a2fd-5935514af075") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:39:45.782450 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.782432    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7b76\" (UniqueName: \"kubernetes.io/projected/53a003c6-2a55-4b7c-9242-766116f41b2a-kube-api-access-p7b76\") pod \"node-resolver-clstx\" (UID: \"53a003c6-2a55-4b7c-9242-766116f41b2a\") " pod="openshift-dns/node-resolver-clstx"
Sep 11 04:39:45.782584 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.782569    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flls2\" (UniqueName: \"kubernetes.io/projected/078c1f6d-dc41-44ae-9828-96baf9124176-kube-api-access-flls2\") pod \"multus-zs45f\" (UID: \"078c1f6d-dc41-44ae-9828-96baf9124176\") " pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.807287 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.807264    2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 04:39:45.877519 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.877456    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7thkg"
Sep 11 04:39:45.884705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.884673    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p8wsn"
Sep 11 04:39:45.887436 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:45.887418    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57e4b5b7_b3cc_4052_be3c_07a138c81e91.slice/crio-f3633f7fb443bb12e54bd51b5713bb0e5352c0c428316883bf3c2b36cfb3ca44 WatchSource:0}: Error finding container f3633f7fb443bb12e54bd51b5713bb0e5352c0c428316883bf3c2b36cfb3ca44: Status 404 returned error can't find the container with id f3633f7fb443bb12e54bd51b5713bb0e5352c0c428316883bf3c2b36cfb3ca44
Sep 11 04:39:45.892728 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:45.892707    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5232e65a_4484_4196_bb63_6fd0840e83e0.slice/crio-cc1c98db0cfdf66342888e3eaf856350c1fb4cab365431fa96347f77987e8d89 WatchSource:0}: Error finding container cc1c98db0cfdf66342888e3eaf856350c1fb4cab365431fa96347f77987e8d89: Status 404 returned error can't find the container with id cc1c98db0cfdf66342888e3eaf856350c1fb4cab365431fa96347f77987e8d89
Sep 11 04:39:45.899043 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.899010    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zs45f"
Sep 11 04:39:45.907699 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:45.907681    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod078c1f6d_dc41_44ae_9828_96baf9124176.slice/crio-decf62fedf97030897a362917d39b8a9b828719537052b97e58e74c671954237 WatchSource:0}: Error finding container decf62fedf97030897a362917d39b8a9b828719537052b97e58e74c671954237: Status 404 returned error can't find the container with id decf62fedf97030897a362917d39b8a9b828719537052b97e58e74c671954237
Sep 11 04:39:45.909362 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.909329    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2lz5t"
Sep 11 04:39:45.918717 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:45.918697    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5d3232f_f007_4c0d_976f_80397eb71a4b.slice/crio-9247c40981327f6bdeb3ccd64301df97289f02e5e571fd1c508eaa591abc37b3 WatchSource:0}: Error finding container 9247c40981327f6bdeb3ccd64301df97289f02e5e571fd1c508eaa591abc37b3: Status 404 returned error can't find the container with id 9247c40981327f6bdeb3ccd64301df97289f02e5e571fd1c508eaa591abc37b3
Sep 11 04:39:45.922923 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.922910    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4d47f"
Sep 11 04:39:45.930155 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:45.930138    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2133f096_0741_459c_a579_1742c273f6f0.slice/crio-a88ded839a0c6e8c4ef0708c974a5f96ea2357a982d090cac04f6d8eb23e3606 WatchSource:0}: Error finding container a88ded839a0c6e8c4ef0708c974a5f96ea2357a982d090cac04f6d8eb23e3606: Status 404 returned error can't find the container with id a88ded839a0c6e8c4ef0708c974a5f96ea2357a982d090cac04f6d8eb23e3606
Sep 11 04:39:45.934190 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.934175    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:39:45.942511 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:45.942495    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ffd3e8f_56a8_4b04_96f3_ca0921820cbe.slice/crio-f8548e9d6fe50dcc1d6a9490c4668a140bea04524d999b58c0abec53e4ae9fca WatchSource:0}: Error finding container f8548e9d6fe50dcc1d6a9490c4668a140bea04524d999b58c0abec53e4ae9fca: Status 404 returned error can't find the container with id f8548e9d6fe50dcc1d6a9490c4668a140bea04524d999b58c0abec53e4ae9fca
Sep 11 04:39:45.957508 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.957479    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz"
Sep 11 04:39:45.964286 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.964268    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hjkmf"
Sep 11 04:39:45.964807 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:45.964789    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-clstx"
Sep 11 04:39:45.964922 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:45.964883    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b5102b3_adf5_4fce_a49a_474eec1ad621.slice/crio-90b77d70e238ab4b3214736b30903270729ebc85f6e64b3b7fd0eba28dcc53d8 WatchSource:0}: Error finding container 90b77d70e238ab4b3214736b30903270729ebc85f6e64b3b7fd0eba28dcc53d8: Status 404 returned error can't find the container with id 90b77d70e238ab4b3214736b30903270729ebc85f6e64b3b7fd0eba28dcc53d8
Sep 11 04:39:45.972492 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:45.972469    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd07dca5c_5fab_4523_bc76_884d85f79afe.slice/crio-63b27ac6b635f28938f0880c768586f1f6c2b3e99db91037152140d12878c193 WatchSource:0}: Error finding container 63b27ac6b635f28938f0880c768586f1f6c2b3e99db91037152140d12878c193: Status 404 returned error can't find the container with id 63b27ac6b635f28938f0880c768586f1f6c2b3e99db91037152140d12878c193
Sep 11 04:39:45.975521 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:39:45.975505    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53a003c6_2a55_4b7c_9242_766116f41b2a.slice/crio-811649e9d83da19a2dc7b4d3398528184e8de69c5367c43b4dcad4990878b014 WatchSource:0}: Error finding container 811649e9d83da19a2dc7b4d3398528184e8de69c5367c43b4dcad4990878b014: Status 404 returned error can't find the container with id 811649e9d83da19a2dc7b4d3398528184e8de69c5367c43b4dcad4990878b014
Sep 11 04:39:47.654521 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:47.654140    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs\") pod \"network-metrics-daemon-dfv69\" (UID: \"0dc2bea8-03da-45f3-ac2b-d428b89a7542\") " pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:39:47.654521 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:47.654204    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret\") pod \"global-pull-secret-syncer-9zqb2\" (UID: \"dbc886e3-0aab-4149-87f3-1d4ec91672f9\") " pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:39:47.654521 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:47.654328    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 04:39:47.654521 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:47.654402    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret podName:dbc886e3-0aab-4149-87f3-1d4ec91672f9 nodeName:}" failed. No retries permitted until 2026-09-11 04:39:48.654385274 +0000 UTC m=+3.125849131 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret") pod "global-pull-secret-syncer-9zqb2" (UID: "dbc886e3-0aab-4149-87f3-1d4ec91672f9") : object "kube-system"/"original-pull-secret" not registered
Sep 11 04:39:47.654846 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:47.654774    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:39:47.654846 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:47.654830    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs podName:0dc2bea8-03da-45f3-ac2b-d428b89a7542 nodeName:}" failed. No retries permitted until 2026-09-11 04:39:48.654812295 +0000 UTC m=+3.126276168 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs") pod "network-metrics-daemon-dfv69" (UID: "0dc2bea8-03da-45f3-ac2b-d428b89a7542") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:39:47.756010 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:47.755298    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh7kw\" (UniqueName: \"kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw\") pod \"network-check-target-nbqjz\" (UID: \"ae3ae47e-fe4b-4298-a2fd-5935514af075\") " pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:39:47.756010 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:47.755525    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 04:39:47.756010 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:47.755544    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 04:39:47.756010 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:47.755557    2567 projected.go:196] Error preparing data for projected volume kube-api-access-wh7kw for pod openshift-network-diagnostics/network-check-target-nbqjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:39:47.756010 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:47.755614    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw podName:ae3ae47e-fe4b-4298-a2fd-5935514af075 nodeName:}" failed. No retries permitted until 2026-09-11 04:39:48.755597084 +0000 UTC m=+3.227060942 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wh7kw" (UniqueName: "kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw") pod "network-check-target-nbqjz" (UID: "ae3ae47e-fe4b-4298-a2fd-5935514af075") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:39:47.994835 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:47.994745    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 04:34:46 +0000 UTC" deadline="2028-02-23 00:37:23.993506039 +0000 UTC"
Sep 11 04:39:47.994835 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:47.994786    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12715h57m35.998724333s"
Sep 11 04:39:48.012212 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:48.012158    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:39:48.012374 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:48.012313    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfv69" podUID="0dc2bea8-03da-45f3-ac2b-d428b89a7542"
Sep 11 04:39:48.034474 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:48.034409    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zs45f" event={"ID":"078c1f6d-dc41-44ae-9828-96baf9124176","Type":"ContainerStarted","Data":"decf62fedf97030897a362917d39b8a9b828719537052b97e58e74c671954237"}
Sep 11 04:39:48.050395 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:48.050361    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p8wsn" event={"ID":"5232e65a-4484-4196-bb63-6fd0840e83e0","Type":"ContainerStarted","Data":"cc1c98db0cfdf66342888e3eaf856350c1fb4cab365431fa96347f77987e8d89"}
Sep 11 04:39:48.058627 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:48.058595    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-18.ec2.internal" event={"ID":"81eb68d3a2c532023a6d1abc2053e6b2","Type":"ContainerStarted","Data":"894d7cbb1d63f621dbe51215144c1ae5be42a077d9df8590f4382b0ae80a8e8c"}
Sep 11 04:39:48.081289 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:48.081258    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-clstx" event={"ID":"53a003c6-2a55-4b7c-9242-766116f41b2a","Type":"ContainerStarted","Data":"811649e9d83da19a2dc7b4d3398528184e8de69c5367c43b4dcad4990878b014"}
Sep 11 04:39:48.098858 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:48.098827    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hjkmf" event={"ID":"d07dca5c-5fab-4523-bc76-884d85f79afe","Type":"ContainerStarted","Data":"63b27ac6b635f28938f0880c768586f1f6c2b3e99db91037152140d12878c193"}
Sep 11 04:39:48.113504 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:48.113458    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9" event={"ID":"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe","Type":"ContainerStarted","Data":"f8548e9d6fe50dcc1d6a9490c4668a140bea04524d999b58c0abec53e4ae9fca"}
Sep 11 04:39:48.115278 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:48.115254    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4d47f" event={"ID":"2133f096-0741-459c-a579-1742c273f6f0","Type":"ContainerStarted","Data":"a88ded839a0c6e8c4ef0708c974a5f96ea2357a982d090cac04f6d8eb23e3606"}
Sep 11 04:39:48.121379 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:48.121354    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7thkg" event={"ID":"57e4b5b7-b3cc-4052-be3c-07a138c81e91","Type":"ContainerStarted","Data":"f3633f7fb443bb12e54bd51b5713bb0e5352c0c428316883bf3c2b36cfb3ca44"}
Sep 11 04:39:48.123526 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:48.123501    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-18.ec2.internal" event={"ID":"26845c0b3b67ab14283ca3124672441d","Type":"ContainerStarted","Data":"60b551fcb1da64aa0a364f28a2bfd29b36cc4c7cd55298bbc65786c917a8dba6"}
Sep 11 04:39:48.154513 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:48.154485    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz" event={"ID":"7b5102b3-adf5-4fce-a49a-474eec1ad621","Type":"ContainerStarted","Data":"90b77d70e238ab4b3214736b30903270729ebc85f6e64b3b7fd0eba28dcc53d8"}
Sep 11 04:39:48.165134 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:48.165107    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2lz5t" event={"ID":"d5d3232f-f007-4c0d-976f-80397eb71a4b","Type":"ContainerStarted","Data":"9247c40981327f6bdeb3ccd64301df97289f02e5e571fd1c508eaa591abc37b3"}
Sep 11 04:39:48.273785 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:48.273700    2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 04:39:48.662623 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:48.662539    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs\") pod \"network-metrics-daemon-dfv69\" (UID: \"0dc2bea8-03da-45f3-ac2b-d428b89a7542\") " pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:39:48.662623 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:48.662604    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret\") pod \"global-pull-secret-syncer-9zqb2\" (UID: \"dbc886e3-0aab-4149-87f3-1d4ec91672f9\") " pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:39:48.662842 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:48.662743    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 04:39:48.662842 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:48.662798    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret podName:dbc886e3-0aab-4149-87f3-1d4ec91672f9 nodeName:}" failed. No retries permitted until 2026-09-11 04:39:50.662781281 +0000 UTC m=+5.134245141 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret") pod "global-pull-secret-syncer-9zqb2" (UID: "dbc886e3-0aab-4149-87f3-1d4ec91672f9") : object "kube-system"/"original-pull-secret" not registered
Sep 11 04:39:48.663365 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:48.663176    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:39:48.663365 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:48.663226    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs podName:0dc2bea8-03da-45f3-ac2b-d428b89a7542 nodeName:}" failed. No retries permitted until 2026-09-11 04:39:50.66321026 +0000 UTC m=+5.134674128 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs") pod "network-metrics-daemon-dfv69" (UID: "0dc2bea8-03da-45f3-ac2b-d428b89a7542") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:39:48.763222 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:48.763188    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh7kw\" (UniqueName: \"kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw\") pod \"network-check-target-nbqjz\" (UID: \"ae3ae47e-fe4b-4298-a2fd-5935514af075\") " pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:39:48.763403 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:48.763386    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 04:39:48.763475 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:48.763415    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 04:39:48.763475 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:48.763428    2567 projected.go:196] Error preparing data for projected volume kube-api-access-wh7kw for pod openshift-network-diagnostics/network-check-target-nbqjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:39:48.763582 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:48.763482    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw podName:ae3ae47e-fe4b-4298-a2fd-5935514af075 nodeName:}" failed. No retries permitted until 2026-09-11 04:39:50.763463605 +0000 UTC m=+5.234927463 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wh7kw" (UniqueName: "kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw") pod "network-check-target-nbqjz" (UID: "ae3ae47e-fe4b-4298-a2fd-5935514af075") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:39:48.995279 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:48.995160    2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 04:34:46 +0000 UTC" deadline="2028-06-28 16:52:47.149731948 +0000 UTC"
Sep 11 04:39:48.995279 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:48.995198    2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15756h12m58.154538626s"
Sep 11 04:39:49.012197 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:49.011449    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:39:49.012197 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:49.011575    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zqb2" podUID="dbc886e3-0aab-4149-87f3-1d4ec91672f9"
Sep 11 04:39:49.012197 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:49.011675    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:39:49.012197 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:49.011775    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbqjz" podUID="ae3ae47e-fe4b-4298-a2fd-5935514af075"
Sep 11 04:39:49.927916 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:49.927658    2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 04:39:50.014970 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:50.014348    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:39:50.014970 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:50.014486    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfv69" podUID="0dc2bea8-03da-45f3-ac2b-d428b89a7542"
Sep 11 04:39:50.683058 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:50.683023    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret\") pod \"global-pull-secret-syncer-9zqb2\" (UID: \"dbc886e3-0aab-4149-87f3-1d4ec91672f9\") " pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:39:50.683242 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:50.683129    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs\") pod \"network-metrics-daemon-dfv69\" (UID: \"0dc2bea8-03da-45f3-ac2b-d428b89a7542\") " pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:39:50.683314 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:50.683243    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:39:50.683314 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:50.683293    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs podName:0dc2bea8-03da-45f3-ac2b-d428b89a7542 nodeName:}" failed. No retries permitted until 2026-09-11 04:39:54.683279119 +0000 UTC m=+9.154742972 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs") pod "network-metrics-daemon-dfv69" (UID: "0dc2bea8-03da-45f3-ac2b-d428b89a7542") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:39:50.683479 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:50.683448    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 04:39:50.683549 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:50.683521    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret podName:dbc886e3-0aab-4149-87f3-1d4ec91672f9 nodeName:}" failed. No retries permitted until 2026-09-11 04:39:54.683504363 +0000 UTC m=+9.154968221 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret") pod "global-pull-secret-syncer-9zqb2" (UID: "dbc886e3-0aab-4149-87f3-1d4ec91672f9") : object "kube-system"/"original-pull-secret" not registered
Sep 11 04:39:50.784131 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:50.784094    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh7kw\" (UniqueName: \"kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw\") pod \"network-check-target-nbqjz\" (UID: \"ae3ae47e-fe4b-4298-a2fd-5935514af075\") " pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:39:50.784813 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:50.784312    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 04:39:50.784813 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:50.784342    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 04:39:50.784813 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:50.784355    2567 projected.go:196] Error preparing data for projected volume kube-api-access-wh7kw for pod openshift-network-diagnostics/network-check-target-nbqjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:39:50.784813 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:50.784419    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw podName:ae3ae47e-fe4b-4298-a2fd-5935514af075 nodeName:}" failed. No retries permitted until 2026-09-11 04:39:54.784401196 +0000 UTC m=+9.255865074 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wh7kw" (UniqueName: "kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw") pod "network-check-target-nbqjz" (UID: "ae3ae47e-fe4b-4298-a2fd-5935514af075") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:39:51.011848 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:51.011765    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:39:51.011999 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:51.011893    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbqjz" podUID="ae3ae47e-fe4b-4298-a2fd-5935514af075"
Sep 11 04:39:51.012333 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:51.012314    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:39:51.012434 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:51.012408    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zqb2" podUID="dbc886e3-0aab-4149-87f3-1d4ec91672f9"
Sep 11 04:39:52.012737 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:52.012243    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:39:52.012737 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:52.012389    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfv69" podUID="0dc2bea8-03da-45f3-ac2b-d428b89a7542"
Sep 11 04:39:53.011278 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:53.011240    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:39:53.011466 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:53.011245    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:39:53.011466 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:53.011373    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zqb2" podUID="dbc886e3-0aab-4149-87f3-1d4ec91672f9"
Sep 11 04:39:53.011466 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:53.011450    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbqjz" podUID="ae3ae47e-fe4b-4298-a2fd-5935514af075"
Sep 11 04:39:54.011801 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:54.011767    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:39:54.012218 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:54.011906    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfv69" podUID="0dc2bea8-03da-45f3-ac2b-d428b89a7542"
Sep 11 04:39:54.716820 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:54.716732    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret\") pod \"global-pull-secret-syncer-9zqb2\" (UID: \"dbc886e3-0aab-4149-87f3-1d4ec91672f9\") " pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:39:54.716993 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:54.716822    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs\") pod \"network-metrics-daemon-dfv69\" (UID: \"0dc2bea8-03da-45f3-ac2b-d428b89a7542\") " pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:39:54.716993 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:54.716907    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 04:39:54.716993 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:54.716942    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:39:54.716993 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:54.716977    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret podName:dbc886e3-0aab-4149-87f3-1d4ec91672f9 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:02.71695662 +0000 UTC m=+17.188420478 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret") pod "global-pull-secret-syncer-9zqb2" (UID: "dbc886e3-0aab-4149-87f3-1d4ec91672f9") : object "kube-system"/"original-pull-secret" not registered
Sep 11 04:39:54.717225 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:54.716999    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs podName:0dc2bea8-03da-45f3-ac2b-d428b89a7542 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:02.716987995 +0000 UTC m=+17.188451853 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs") pod "network-metrics-daemon-dfv69" (UID: "0dc2bea8-03da-45f3-ac2b-d428b89a7542") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:39:54.817389 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:54.817350    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh7kw\" (UniqueName: \"kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw\") pod \"network-check-target-nbqjz\" (UID: \"ae3ae47e-fe4b-4298-a2fd-5935514af075\") " pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:39:54.817584 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:54.817561    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 04:39:54.817668 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:54.817588    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 04:39:54.817668 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:54.817601    2567 projected.go:196] Error preparing data for projected volume kube-api-access-wh7kw for pod openshift-network-diagnostics/network-check-target-nbqjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:39:54.817668 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:54.817656    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw podName:ae3ae47e-fe4b-4298-a2fd-5935514af075 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:02.817637929 +0000 UTC m=+17.289101788 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wh7kw" (UniqueName: "kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw") pod "network-check-target-nbqjz" (UID: "ae3ae47e-fe4b-4298-a2fd-5935514af075") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:39:55.011484 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:55.011408    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:39:55.011645 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:55.011539    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbqjz" podUID="ae3ae47e-fe4b-4298-a2fd-5935514af075"
Sep 11 04:39:55.011983 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:55.011963    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:39:55.012378 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:55.012081    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zqb2" podUID="dbc886e3-0aab-4149-87f3-1d4ec91672f9"
Sep 11 04:39:56.014566 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:56.014534    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:39:56.014937 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:56.014657    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfv69" podUID="0dc2bea8-03da-45f3-ac2b-d428b89a7542"
Sep 11 04:39:57.011133 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:57.011098    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:39:57.011276 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:57.011223    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zqb2" podUID="dbc886e3-0aab-4149-87f3-1d4ec91672f9"
Sep 11 04:39:57.011395 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:57.011098    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:39:57.011523 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:57.011498    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbqjz" podUID="ae3ae47e-fe4b-4298-a2fd-5935514af075"
Sep 11 04:39:58.011416 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:58.011385    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:39:58.011965 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:58.011519    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfv69" podUID="0dc2bea8-03da-45f3-ac2b-d428b89a7542"
Sep 11 04:39:59.011420 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:59.011385    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:39:59.011420 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:39:59.011399    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:39:59.011849 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:59.011497    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zqb2" podUID="dbc886e3-0aab-4149-87f3-1d4ec91672f9"
Sep 11 04:39:59.011849 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:39:59.011634    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbqjz" podUID="ae3ae47e-fe4b-4298-a2fd-5935514af075"
Sep 11 04:40:00.011813 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:00.011312    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:40:00.011813 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:00.011451    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfv69" podUID="0dc2bea8-03da-45f3-ac2b-d428b89a7542"
Sep 11 04:40:01.011423 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:01.011392    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:40:01.011610 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:01.011523    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbqjz" podUID="ae3ae47e-fe4b-4298-a2fd-5935514af075"
Sep 11 04:40:01.011610 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:01.011585    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:40:01.011726 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:01.011705    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zqb2" podUID="dbc886e3-0aab-4149-87f3-1d4ec91672f9"
Sep 11 04:40:02.011848 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:02.011803    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:40:02.012301 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:02.011975    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfv69" podUID="0dc2bea8-03da-45f3-ac2b-d428b89a7542"
Sep 11 04:40:02.778392 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:02.778361    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs\") pod \"network-metrics-daemon-dfv69\" (UID: \"0dc2bea8-03da-45f3-ac2b-d428b89a7542\") " pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:40:02.778540 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:02.778404    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret\") pod \"global-pull-secret-syncer-9zqb2\" (UID: \"dbc886e3-0aab-4149-87f3-1d4ec91672f9\") " pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:40:02.778540 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:02.778524    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 04:40:02.778540 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:02.778528    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:40:02.778637 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:02.778573    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret podName:dbc886e3-0aab-4149-87f3-1d4ec91672f9 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:18.778560852 +0000 UTC m=+33.250024706 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret") pod "global-pull-secret-syncer-9zqb2" (UID: "dbc886e3-0aab-4149-87f3-1d4ec91672f9") : object "kube-system"/"original-pull-secret" not registered
Sep 11 04:40:02.778637 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:02.778588    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs podName:0dc2bea8-03da-45f3-ac2b-d428b89a7542 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:18.778580372 +0000 UTC m=+33.250044225 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs") pod "network-metrics-daemon-dfv69" (UID: "0dc2bea8-03da-45f3-ac2b-d428b89a7542") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:40:02.879405 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:02.879365    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh7kw\" (UniqueName: \"kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw\") pod \"network-check-target-nbqjz\" (UID: \"ae3ae47e-fe4b-4298-a2fd-5935514af075\") " pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:40:02.879556 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:02.879533    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 04:40:02.879603 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:02.879562    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 04:40:02.879603 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:02.879577    2567 projected.go:196] Error preparing data for projected volume kube-api-access-wh7kw for pod openshift-network-diagnostics/network-check-target-nbqjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:40:02.879699 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:02.879638    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw podName:ae3ae47e-fe4b-4298-a2fd-5935514af075 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:18.879620056 +0000 UTC m=+33.351083926 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wh7kw" (UniqueName: "kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw") pod "network-check-target-nbqjz" (UID: "ae3ae47e-fe4b-4298-a2fd-5935514af075") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:40:03.011391 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:03.011313    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:40:03.011551 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:03.011322    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:40:03.011551 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:03.011421    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbqjz" podUID="ae3ae47e-fe4b-4298-a2fd-5935514af075"
Sep 11 04:40:03.011551 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:03.011495    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zqb2" podUID="dbc886e3-0aab-4149-87f3-1d4ec91672f9"
Sep 11 04:40:04.012084 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:04.012042    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:40:04.012612 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:04.012198    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfv69" podUID="0dc2bea8-03da-45f3-ac2b-d428b89a7542"
Sep 11 04:40:05.011731 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:05.011696    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:40:05.011914 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:05.011735    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:40:05.011914 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:05.011822    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbqjz" podUID="ae3ae47e-fe4b-4298-a2fd-5935514af075"
Sep 11 04:40:05.011914 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:05.011906    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zqb2" podUID="dbc886e3-0aab-4149-87f3-1d4ec91672f9"
Sep 11 04:40:06.011826 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:06.011799    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:40:06.012320 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:06.011919    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfv69" podUID="0dc2bea8-03da-45f3-ac2b-d428b89a7542"
Sep 11 04:40:07.011449 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.011248    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:40:07.011590 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:07.011562    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbqjz" podUID="ae3ae47e-fe4b-4298-a2fd-5935514af075"
Sep 11 04:40:07.011961 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.011936    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:40:07.012404 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:07.012025    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zqb2" podUID="dbc886e3-0aab-4149-87f3-1d4ec91672f9"
Sep 11 04:40:07.211216 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.210975    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-18.ec2.internal" event={"ID":"81eb68d3a2c532023a6d1abc2053e6b2","Type":"ContainerStarted","Data":"8e4ed6aba416e8a68112f047a6aa23e79457c51b60a928c1b6bc0566bc7ead7e"}
Sep 11 04:40:07.212466 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.212436    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-clstx" event={"ID":"53a003c6-2a55-4b7c-9242-766116f41b2a","Type":"ContainerStarted","Data":"de68a3c735c71ac2ff43c67ebd3c5746d6d09ea11b47712da35fa6feb4f7f870"}
Sep 11 04:40:07.213902 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.213869    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hjkmf" event={"ID":"d07dca5c-5fab-4523-bc76-884d85f79afe","Type":"ContainerStarted","Data":"95ba8c871faf8203e2d866594e20772d76dd1f90db8be81be5c38f523b6123d6"}
Sep 11 04:40:07.216340 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.216316    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9" event={"ID":"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe","Type":"ContainerStarted","Data":"d28df32dd0a562b4a2199243fb213604107290fab3aa3ee4b78f1ea3167ec495"}
Sep 11 04:40:07.216448 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.216382    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9" event={"ID":"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe","Type":"ContainerStarted","Data":"9840b8922a77d6fc1107a75d9bf34be97cc402110d9fbc58fa9839ac4279fba8"}
Sep 11 04:40:07.216448 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.216431    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9" event={"ID":"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe","Type":"ContainerStarted","Data":"16691f219b98547328998c742f7e28011c385b1c365c45315688532c8ad17008"}
Sep 11 04:40:07.216544 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.216463    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9" event={"ID":"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe","Type":"ContainerStarted","Data":"5bfdcec2082dcce01e16c4f116885d79dbc3833c26fd36f35f2944fed17c0f3d"}
Sep 11 04:40:07.217617 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.217598    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7thkg" event={"ID":"57e4b5b7-b3cc-4052-be3c-07a138c81e91","Type":"ContainerStarted","Data":"c619e299e5cd7ded052d238028b0d7496292846994be7e58a1c2717e20836cbd"}
Sep 11 04:40:07.219043 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.219021    2567 generic.go:356] "Generic (PLEG): container finished" podID="26845c0b3b67ab14283ca3124672441d" containerID="30b3ce7a3854ff0957abb9d394054f88fd9c6f55a8a688aea65e4834b2de3edd" exitCode=0
Sep 11 04:40:07.219138 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.219101    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-18.ec2.internal" event={"ID":"26845c0b3b67ab14283ca3124672441d","Type":"ContainerDied","Data":"30b3ce7a3854ff0957abb9d394054f88fd9c6f55a8a688aea65e4834b2de3edd"}
Sep 11 04:40:07.220529 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.220509    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz" event={"ID":"7b5102b3-adf5-4fce-a49a-474eec1ad621","Type":"ContainerStarted","Data":"021de16851182394d54e377749bf8b7a7bbca7d6bd64aa3d8b25c5d7b6b51b91"}
Sep 11 04:40:07.221942 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.221924    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2lz5t" event={"ID":"d5d3232f-f007-4c0d-976f-80397eb71a4b","Type":"ContainerStarted","Data":"4bd6043e39f7f1fa2c625288978c92647db2b29099a6e1d45a1d2854b14bad5a"}
Sep 11 04:40:07.223383 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.223363    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zs45f" event={"ID":"078c1f6d-dc41-44ae-9828-96baf9124176","Type":"ContainerStarted","Data":"4eb5a1fe4cfa22dabab942105b984e517e5466218d2c7dd4e5e59a531e08c37f"}
Sep 11 04:40:07.224806 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.224787    2567 generic.go:356] "Generic (PLEG): container finished" podID="5232e65a-4484-4196-bb63-6fd0840e83e0" containerID="76e54d0116e8ce5a9e828a2883202c786a79d9fda111b4c5dd94c0bd5f5e0c4b" exitCode=0
Sep 11 04:40:07.224899 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.224812    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p8wsn" event={"ID":"5232e65a-4484-4196-bb63-6fd0840e83e0","Type":"ContainerDied","Data":"76e54d0116e8ce5a9e828a2883202c786a79d9fda111b4c5dd94c0bd5f5e0c4b"}
Sep 11 04:40:07.225682 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.225632    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-18.ec2.internal" podStartSLOduration=21.225621314 podStartE2EDuration="21.225621314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:39:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:07.225551297 +0000 UTC m=+21.697015172" watchObservedRunningTime="2026-09-11 04:40:07.225621314 +0000 UTC m=+21.697085189"
Sep 11 04:40:07.264413 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.264373    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zs45f" podStartSLOduration=2.206539376 podStartE2EDuration="21.264354418s" totalImagesPullingTime="19.057815042s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:07.241794836 +0000 UTC m=+21.713258711" watchObservedRunningTime="2026-09-11 04:40:07.264354418 +0000 UTC m=+21.735818296"
Sep 11 04:40:07.265018 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.265004    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2lz5t"
Sep 11 04:40:07.265590 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.265547    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2lz5t"
Sep 11 04:40:07.280004 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.279970    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hjkmf" podStartSLOduration=2.332671347 podStartE2EDuration="21.279959552s" totalImagesPullingTime="18.947288205s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:07.279694026 +0000 UTC m=+21.751157914" watchObservedRunningTime="2026-09-11 04:40:07.279959552 +0000 UTC m=+21.751423457"
Sep 11 04:40:07.294263 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.294179    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-clstx" podStartSLOduration=2.370020879 podStartE2EDuration="21.29416337s" totalImagesPullingTime="18.924142491s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:07.293369058 +0000 UTC m=+21.764832937" watchObservedRunningTime="2026-09-11 04:40:07.29416337 +0000 UTC m=+21.765627247"
Sep 11 04:40:07.312827 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.312793    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7thkg" podStartSLOduration=2.369525302 podStartE2EDuration="21.312781208s" totalImagesPullingTime="18.943255906s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:07.312685914 +0000 UTC m=+21.784149789" watchObservedRunningTime="2026-09-11 04:40:07.312781208 +0000 UTC m=+21.784245084"
Sep 11 04:40:07.336395 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:07.336341    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2lz5t" podStartSLOduration=2.423944626 podStartE2EDuration="21.336328319s" totalImagesPullingTime="18.912383693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:07.336036249 +0000 UTC m=+21.807500118" watchObservedRunningTime="2026-09-11 04:40:07.336328319 +0000 UTC m=+21.807792195"
Sep 11 04:40:08.012148 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:08.012113    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:40:08.012932 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:08.012274    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfv69" podUID="0dc2bea8-03da-45f3-ac2b-d428b89a7542"
Sep 11 04:40:08.231262 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:08.231178    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9" event={"ID":"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe","Type":"ContainerStarted","Data":"ecc0527afc1df8f1bec696e1e97c14d9cc00d907bfffb20427e72262518b9618"}
Sep 11 04:40:08.231262 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:08.231260    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9" event={"ID":"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe","Type":"ContainerStarted","Data":"5d0048e0b8705dfba6e3d82f73b78a8cf66a541fc84fc3bf9bc473f136f2a033"}
Sep 11 04:40:08.233031 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:08.233002    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4d47f" event={"ID":"2133f096-0741-459c-a579-1742c273f6f0","Type":"ContainerStarted","Data":"1293d2229ff82b7f75f852f790a600f6147db0f6bb4fc0d82c6fb86c8910c869"}
Sep 11 04:40:08.235416 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:08.235392    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-18.ec2.internal" event={"ID":"26845c0b3b67ab14283ca3124672441d","Type":"ContainerStarted","Data":"5c4132a708625acb6a42d8b551dddd28b1c3b3d0f11b43af2ae6935a520dca07"}
Sep 11 04:40:08.236204 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:08.236181    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2lz5t"
Sep 11 04:40:08.236719 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:08.236697    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2lz5t"
Sep 11 04:40:08.248020 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:08.247982    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4d47f" podStartSLOduration=3.255869542 podStartE2EDuration="22.247970373s" totalImagesPullingTime="18.992100831s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:08.247781916 +0000 UTC m=+22.719245794" watchObservedRunningTime="2026-09-11 04:40:08.247970373 +0000 UTC m=+22.719434249"
Sep 11 04:40:08.281805 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:08.281759    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-18.ec2.internal" podStartSLOduration=22.281749328 podStartE2EDuration="22.281749328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:39:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:08.28151798 +0000 UTC m=+22.752981858" watchObservedRunningTime="2026-09-11 04:40:08.281749328 +0000 UTC m=+22.753213203"
Sep 11 04:40:08.613434 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:08.613403    2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 04:40:09.009871 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:09.009782    2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T04:40:08.613420161Z","UUID":"73549064-6501-4edf-af65-0036891f9f04","Handler":null,"Name":"","Endpoint":""}
Sep 11 04:40:09.011165 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:09.011135    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:40:09.011277 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:09.011138    2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 04:40:09.011277 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:09.011235    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zqb2" podUID="dbc886e3-0aab-4149-87f3-1d4ec91672f9"
Sep 11 04:40:09.011277 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:09.011254    2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 04:40:09.011398 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:09.011144    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:40:09.011398 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:09.011362    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbqjz" podUID="ae3ae47e-fe4b-4298-a2fd-5935514af075"
Sep 11 04:40:09.239532 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:09.239497    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz" event={"ID":"7b5102b3-adf5-4fce-a49a-474eec1ad621","Type":"ContainerStarted","Data":"aa4198265bc58300b13876b517fc34772ceae1ca3529bed822d996b82e49b931"}
Sep 11 04:40:10.011918 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:10.011734    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:40:10.012052 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:10.012000    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfv69" podUID="0dc2bea8-03da-45f3-ac2b-d428b89a7542"
Sep 11 04:40:10.244810 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:10.244779    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9" event={"ID":"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe","Type":"ContainerStarted","Data":"b50547e2cf677f3dfb18b5ec5fc79f323622bb19f5ffe9595b94b65ce863529b"}
Sep 11 04:40:10.246802 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:10.246777    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz" event={"ID":"7b5102b3-adf5-4fce-a49a-474eec1ad621","Type":"ContainerStarted","Data":"62540fde19e1258727d39d0a457ab64ee78760865a71bd2d0fb2444927941e8d"}
Sep 11 04:40:10.265270 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:10.265204    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-l8kvz" podStartSLOduration=2.540024009 podStartE2EDuration="24.26519215s" totalImagesPullingTime="21.725168141s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:39:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:10.264763581 +0000 UTC m=+24.736227456" watchObservedRunningTime="2026-09-11 04:40:10.26519215 +0000 UTC m=+24.736656059"
Sep 11 04:40:11.011904 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:11.011878    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:40:11.012062 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:11.011883    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:40:11.012062 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:11.011970    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbqjz" podUID="ae3ae47e-fe4b-4298-a2fd-5935514af075"
Sep 11 04:40:11.012062 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:11.012043    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zqb2" podUID="dbc886e3-0aab-4149-87f3-1d4ec91672f9"
Sep 11 04:40:12.011349 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:12.011270    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:40:12.011790 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:12.011384    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfv69" podUID="0dc2bea8-03da-45f3-ac2b-d428b89a7542"
Sep 11 04:40:13.011347 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:13.011170    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:40:13.011458 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:13.011220    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:40:13.011458 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:13.011430    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbqjz" podUID="ae3ae47e-fe4b-4298-a2fd-5935514af075"
Sep 11 04:40:13.011727 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:13.011485    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zqb2" podUID="dbc886e3-0aab-4149-87f3-1d4ec91672f9"
Sep 11 04:40:13.254043 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:13.253962    2567 generic.go:356] "Generic (PLEG): container finished" podID="5232e65a-4484-4196-bb63-6fd0840e83e0" containerID="934127bdb0ae947a61c94f6ad01f87fb60be570c11de0203eecacb279d5ba4ef" exitCode=0
Sep 11 04:40:13.254233 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:13.254049    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p8wsn" event={"ID":"5232e65a-4484-4196-bb63-6fd0840e83e0","Type":"ContainerDied","Data":"934127bdb0ae947a61c94f6ad01f87fb60be570c11de0203eecacb279d5ba4ef"}
Sep 11 04:40:13.258048 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:13.258027    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9" event={"ID":"4ffd3e8f-56a8-4b04-96f3-ca0921820cbe","Type":"ContainerStarted","Data":"bb818fb0fb14f6899c1f69e348eafcf5a6f619b8ede07e76097979852bea7116"}
Sep 11 04:40:13.258302 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:13.258286    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:40:13.258369 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:13.258325    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:40:13.258369 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:13.258355    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:40:13.275101 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:13.275054    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:40:13.275595 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:13.275580    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:40:13.303736 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:13.303699    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9" podStartSLOduration=7.837191892 podStartE2EDuration="27.303687104s" totalImagesPullingTime="19.466495212s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:13.303217909 +0000 UTC m=+27.774681785" watchObservedRunningTime="2026-09-11 04:40:13.303687104 +0000 UTC m=+27.775150979"
Sep 11 04:40:14.012021 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:14.011990    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:40:14.012651 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:14.012160    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfv69" podUID="0dc2bea8-03da-45f3-ac2b-d428b89a7542"
Sep 11 04:40:14.579838 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:14.579760    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dfv69"]
Sep 11 04:40:14.580020 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:14.579863    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:40:14.580020 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:14.579952    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfv69" podUID="0dc2bea8-03da-45f3-ac2b-d428b89a7542"
Sep 11 04:40:14.582979 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:14.582948    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9zqb2"]
Sep 11 04:40:14.583168 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:14.583046    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:40:14.583244 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:14.583158    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zqb2" podUID="dbc886e3-0aab-4149-87f3-1d4ec91672f9"
Sep 11 04:40:14.583779 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:14.583757    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nbqjz"]
Sep 11 04:40:14.583882 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:14.583839    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:40:14.583943 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:14.583906    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbqjz" podUID="ae3ae47e-fe4b-4298-a2fd-5935514af075"
Sep 11 04:40:15.265238 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:15.265203    2567 generic.go:356] "Generic (PLEG): container finished" podID="5232e65a-4484-4196-bb63-6fd0840e83e0" containerID="f0dcad38487eae2d324b0af515a5944a8c2047650c8500e742164221fe045331" exitCode=0
Sep 11 04:40:15.265838 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:15.265276    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p8wsn" event={"ID":"5232e65a-4484-4196-bb63-6fd0840e83e0","Type":"ContainerDied","Data":"f0dcad38487eae2d324b0af515a5944a8c2047650c8500e742164221fe045331"}
Sep 11 04:40:16.012168 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:16.012092    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:40:16.012168 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:16.012120    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:40:16.012361 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:16.012173    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbqjz" podUID="ae3ae47e-fe4b-4298-a2fd-5935514af075"
Sep 11 04:40:16.012361 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:16.012298    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zqb2" podUID="dbc886e3-0aab-4149-87f3-1d4ec91672f9"
Sep 11 04:40:17.011804 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:17.011582    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:40:17.012157 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:17.011866    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfv69" podUID="0dc2bea8-03da-45f3-ac2b-d428b89a7542"
Sep 11 04:40:17.272939 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:17.272862    2567 generic.go:356] "Generic (PLEG): container finished" podID="5232e65a-4484-4196-bb63-6fd0840e83e0" containerID="b7cc60f4ccedba6e937106d11ce4106dc0b5fc8cf9ac55d6c24e1af4c6360a5a" exitCode=0
Sep 11 04:40:17.272939 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:17.272922    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p8wsn" event={"ID":"5232e65a-4484-4196-bb63-6fd0840e83e0","Type":"ContainerDied","Data":"b7cc60f4ccedba6e937106d11ce4106dc0b5fc8cf9ac55d6c24e1af4c6360a5a"}
Sep 11 04:40:18.011342 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:18.011279    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:40:18.011489 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:18.011384    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zqb2" podUID="dbc886e3-0aab-4149-87f3-1d4ec91672f9"
Sep 11 04:40:18.014465 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:18.011784    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:40:18.014465 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:18.011936    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nbqjz" podUID="ae3ae47e-fe4b-4298-a2fd-5935514af075"
Sep 11 04:40:18.794499 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:18.794436    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs\") pod \"network-metrics-daemon-dfv69\" (UID: \"0dc2bea8-03da-45f3-ac2b-d428b89a7542\") " pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:40:18.794499 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:18.794497    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret\") pod \"global-pull-secret-syncer-9zqb2\" (UID: \"dbc886e3-0aab-4149-87f3-1d4ec91672f9\") " pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:40:18.794802 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:18.794606    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:40:18.794802 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:18.794634    2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 04:40:18.794802 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:18.794686    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs podName:0dc2bea8-03da-45f3-ac2b-d428b89a7542 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:50.794665962 +0000 UTC m=+65.266129815 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs") pod "network-metrics-daemon-dfv69" (UID: "0dc2bea8-03da-45f3-ac2b-d428b89a7542") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 04:40:18.794802 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:18.794704    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret podName:dbc886e3-0aab-4149-87f3-1d4ec91672f9 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:50.794696232 +0000 UTC m=+65.266160086 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret") pod "global-pull-secret-syncer-9zqb2" (UID: "dbc886e3-0aab-4149-87f3-1d4ec91672f9") : object "kube-system"/"original-pull-secret" not registered
Sep 11 04:40:18.895523 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:18.895485    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh7kw\" (UniqueName: \"kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw\") pod \"network-check-target-nbqjz\" (UID: \"ae3ae47e-fe4b-4298-a2fd-5935514af075\") " pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:40:18.895778 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:18.895758    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 04:40:18.895859 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:18.895788    2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 04:40:18.895859 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:18.895820    2567 projected.go:196] Error preparing data for projected volume kube-api-access-wh7kw for pod openshift-network-diagnostics/network-check-target-nbqjz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:40:18.895946 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:18.895910    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw podName:ae3ae47e-fe4b-4298-a2fd-5935514af075 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:50.8958913 +0000 UTC m=+65.367355160 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wh7kw" (UniqueName: "kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw") pod "network-check-target-nbqjz" (UID: "ae3ae47e-fe4b-4298-a2fd-5935514af075") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 04:40:19.011966 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.011932    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:40:19.012147 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:19.012087    2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dfv69" podUID="0dc2bea8-03da-45f3-ac2b-d428b89a7542"
Sep 11 04:40:19.369813 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.369782    2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-18.ec2.internal" event="NodeReady"
Sep 11 04:40:19.370285 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.369921    2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 04:40:19.400918 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.400894    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-79f69c7884-pz9cm"]
Sep 11 04:40:19.424271 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.424246    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-dt7d2"]
Sep 11 04:40:19.424537 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.424346    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.426686 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.426662    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 04:40:19.426884 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.426869    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fnkc4\""
Sep 11 04:40:19.426971 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.426945    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 04:40:19.427058 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.427042    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 04:40:19.432997 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.432976    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 04:40:19.433255 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.433236    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79f69c7884-pz9cm"]
Sep 11 04:40:19.433311 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.433273    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2t5fm"]
Sep 11 04:40:19.433392 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.433373    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dt7d2"
Sep 11 04:40:19.435509 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.435491    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v459k\""
Sep 11 04:40:19.435509 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.435496    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 04:40:19.435684 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.435670    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 04:40:19.447575 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.447554    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dt7d2"]
Sep 11 04:40:19.447680 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.447584    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2t5fm"]
Sep 11 04:40:19.447680 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.447653    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2t5fm"
Sep 11 04:40:19.449909 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.449892    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 04:40:19.450007 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.449937    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 04:40:19.450087 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.450011    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6kvx7\""
Sep 11 04:40:19.450398 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.450380    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 04:40:19.500518 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.500453    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b855f12e-f907-45d0-b777-5930f8a8af4c-tmp-dir\") pod \"dns-default-dt7d2\" (UID: \"b855f12e-f907-45d0-b777-5930f8a8af4c\") " pod="openshift-dns/dns-default-dt7d2"
Sep 11 04:40:19.500646 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.500549    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/01a959b4-ae03-4d51-bb69-26b59b7e19a5-trusted-ca\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.500646 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.500575    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr5mz\" (UniqueName: \"kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-kube-api-access-wr5mz\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.500646 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.500622    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/01a959b4-ae03-4d51-bb69-26b59b7e19a5-image-registry-private-configuration\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.500771 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.500654    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls\") pod \"dns-default-dt7d2\" (UID: \"b855f12e-f907-45d0-b777-5930f8a8af4c\") " pod="openshift-dns/dns-default-dt7d2"
Sep 11 04:40:19.500771 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.500675    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.500771 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.500710    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b855f12e-f907-45d0-b777-5930f8a8af4c-config-volume\") pod \"dns-default-dt7d2\" (UID: \"b855f12e-f907-45d0-b777-5930f8a8af4c\") " pod="openshift-dns/dns-default-dt7d2"
Sep 11 04:40:19.500771 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.500728    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdf9g\" (UniqueName: \"kubernetes.io/projected/b855f12e-f907-45d0-b777-5930f8a8af4c-kube-api-access-sdf9g\") pod \"dns-default-dt7d2\" (UID: \"b855f12e-f907-45d0-b777-5930f8a8af4c\") " pod="openshift-dns/dns-default-dt7d2"
Sep 11 04:40:19.500918 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.500749    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/01a959b4-ae03-4d51-bb69-26b59b7e19a5-installation-pull-secrets\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.500918 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.500790    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-bound-sa-token\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.500918 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.500845    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/01a959b4-ae03-4d51-bb69-26b59b7e19a5-ca-trust-extracted\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.500918 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.500862    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-certificates\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.548673 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.548638    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:19.601605 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.601576    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/01a959b4-ae03-4d51-bb69-26b59b7e19a5-image-registry-private-configuration\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.601605 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.601610    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9285e03b-0880-4ae3-b25d-3d4cd81ba474-cert\") pod \"ingress-canary-2t5fm\" (UID: \"9285e03b-0880-4ae3-b25d-3d4cd81ba474\") " pod="openshift-ingress-canary/ingress-canary-2t5fm"
Sep 11 04:40:19.601819 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.601636    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls\") pod \"dns-default-dt7d2\" (UID: \"b855f12e-f907-45d0-b777-5930f8a8af4c\") " pod="openshift-dns/dns-default-dt7d2"
Sep 11 04:40:19.601819 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.601660    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.601819 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:19.601742    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 04:40:19.601819 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.601749    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b855f12e-f907-45d0-b777-5930f8a8af4c-config-volume\") pod \"dns-default-dt7d2\" (UID: \"b855f12e-f907-45d0-b777-5930f8a8af4c\") " pod="openshift-dns/dns-default-dt7d2"
Sep 11 04:40:19.601819 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.601781    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdf9g\" (UniqueName: \"kubernetes.io/projected/b855f12e-f907-45d0-b777-5930f8a8af4c-kube-api-access-sdf9g\") pod \"dns-default-dt7d2\" (UID: \"b855f12e-f907-45d0-b777-5930f8a8af4c\") " pod="openshift-dns/dns-default-dt7d2"
Sep 11 04:40:19.601819 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.601800    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/01a959b4-ae03-4d51-bb69-26b59b7e19a5-installation-pull-secrets\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.601819 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:19.601756    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79f69c7884-pz9cm: secret "image-registry-tls" not found
Sep 11 04:40:19.602182 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.601824    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-bound-sa-token\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.602182 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:19.601828    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 04:40:19.602182 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.601863    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/01a959b4-ae03-4d51-bb69-26b59b7e19a5-ca-trust-extracted\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.602182 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:19.601888    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls podName:01a959b4-ae03-4d51-bb69-26b59b7e19a5 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:20.101863452 +0000 UTC m=+34.573327306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls") pod "image-registry-79f69c7884-pz9cm" (UID: "01a959b4-ae03-4d51-bb69-26b59b7e19a5") : secret "image-registry-tls" not found
Sep 11 04:40:19.602367 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:19.602208    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls podName:b855f12e-f907-45d0-b777-5930f8a8af4c nodeName:}" failed. No retries permitted until 2026-09-11 04:40:20.102187603 +0000 UTC m=+34.573651461 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls") pod "dns-default-dt7d2" (UID: "b855f12e-f907-45d0-b777-5930f8a8af4c") : secret "dns-default-metrics-tls" not found
Sep 11 04:40:19.602367 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.602241    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/01a959b4-ae03-4d51-bb69-26b59b7e19a5-ca-trust-extracted\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.602367 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.602247    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-certificates\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.602367 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.602258    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b855f12e-f907-45d0-b777-5930f8a8af4c-config-volume\") pod \"dns-default-dt7d2\" (UID: \"b855f12e-f907-45d0-b777-5930f8a8af4c\") " pod="openshift-dns/dns-default-dt7d2"
Sep 11 04:40:19.602367 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.602315    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b855f12e-f907-45d0-b777-5930f8a8af4c-tmp-dir\") pod \"dns-default-dt7d2\" (UID: \"b855f12e-f907-45d0-b777-5930f8a8af4c\") " pod="openshift-dns/dns-default-dt7d2"
Sep 11 04:40:19.602367 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.602341    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/01a959b4-ae03-4d51-bb69-26b59b7e19a5-trusted-ca\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.602367 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.602367    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr5mz\" (UniqueName: \"kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-kube-api-access-wr5mz\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.602735 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.602394    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqn75\" (UniqueName: \"kubernetes.io/projected/9285e03b-0880-4ae3-b25d-3d4cd81ba474-kube-api-access-lqn75\") pod \"ingress-canary-2t5fm\" (UID: \"9285e03b-0880-4ae3-b25d-3d4cd81ba474\") " pod="openshift-ingress-canary/ingress-canary-2t5fm"
Sep 11 04:40:19.602735 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.602724    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b855f12e-f907-45d0-b777-5930f8a8af4c-tmp-dir\") pod \"dns-default-dt7d2\" (UID: \"b855f12e-f907-45d0-b777-5930f8a8af4c\") " pod="openshift-dns/dns-default-dt7d2"
Sep 11 04:40:19.602952 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.602932    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-certificates\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.603363 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.603343    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/01a959b4-ae03-4d51-bb69-26b59b7e19a5-trusted-ca\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.606413 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.606390    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/01a959b4-ae03-4d51-bb69-26b59b7e19a5-image-registry-private-configuration\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.606535 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.606393    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/01a959b4-ae03-4d51-bb69-26b59b7e19a5-installation-pull-secrets\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.611001 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.610981    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdf9g\" (UniqueName: \"kubernetes.io/projected/b855f12e-f907-45d0-b777-5930f8a8af4c-kube-api-access-sdf9g\") pod \"dns-default-dt7d2\" (UID: \"b855f12e-f907-45d0-b777-5930f8a8af4c\") " pod="openshift-dns/dns-default-dt7d2"
Sep 11 04:40:19.612025 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.611999    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr5mz\" (UniqueName: \"kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-kube-api-access-wr5mz\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.612243 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.612199    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-bound-sa-token\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:19.628357 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.628330    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:19.703491 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.703461    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqn75\" (UniqueName: \"kubernetes.io/projected/9285e03b-0880-4ae3-b25d-3d4cd81ba474-kube-api-access-lqn75\") pod \"ingress-canary-2t5fm\" (UID: \"9285e03b-0880-4ae3-b25d-3d4cd81ba474\") " pod="openshift-ingress-canary/ingress-canary-2t5fm"
Sep 11 04:40:19.703610 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.703498    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9285e03b-0880-4ae3-b25d-3d4cd81ba474-cert\") pod \"ingress-canary-2t5fm\" (UID: \"9285e03b-0880-4ae3-b25d-3d4cd81ba474\") " pod="openshift-ingress-canary/ingress-canary-2t5fm"
Sep 11 04:40:19.706297 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.706276    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9285e03b-0880-4ae3-b25d-3d4cd81ba474-cert\") pod \"ingress-canary-2t5fm\" (UID: \"9285e03b-0880-4ae3-b25d-3d4cd81ba474\") " pod="openshift-ingress-canary/ingress-canary-2t5fm"
Sep 11 04:40:19.713497 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.713475    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqn75\" (UniqueName: \"kubernetes.io/projected/9285e03b-0880-4ae3-b25d-3d4cd81ba474-kube-api-access-lqn75\") pod \"ingress-canary-2t5fm\" (UID: \"9285e03b-0880-4ae3-b25d-3d4cd81ba474\") " pod="openshift-ingress-canary/ingress-canary-2t5fm"
Sep 11 04:40:19.756637 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:19.756545    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2t5fm"
Sep 11 04:40:20.011290 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:20.011209    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:40:20.011537 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:20.011526    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:40:20.014267 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:20.014089    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 04:40:20.014773 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:20.014749    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6kl86\""
Sep 11 04:40:20.014773 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:20.014762    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 04:40:20.014927 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:20.014760    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 04:40:20.063441 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:20.063417    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2t5fm"]
Sep 11 04:40:20.067182 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:40:20.067155    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9285e03b_0880_4ae3_b25d_3d4cd81ba474.slice/crio-f3341323cdeb0f2ef185f194322a82671a45c1f93fc3cb6f7883b108cc47dcee WatchSource:0}: Error finding container f3341323cdeb0f2ef185f194322a82671a45c1f93fc3cb6f7883b108cc47dcee: Status 404 returned error can't find the container with id f3341323cdeb0f2ef185f194322a82671a45c1f93fc3cb6f7883b108cc47dcee
Sep 11 04:40:20.107321 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:20.107300    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls\") pod \"dns-default-dt7d2\" (UID: \"b855f12e-f907-45d0-b777-5930f8a8af4c\") " pod="openshift-dns/dns-default-dt7d2"
Sep 11 04:40:20.107434 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:20.107331    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:20.107501 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:20.107464    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 04:40:20.107569 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:20.107466    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 04:40:20.107635 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:20.107569    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls podName:b855f12e-f907-45d0-b777-5930f8a8af4c nodeName:}" failed. No retries permitted until 2026-09-11 04:40:21.107546503 +0000 UTC m=+35.579010385 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls") pod "dns-default-dt7d2" (UID: "b855f12e-f907-45d0-b777-5930f8a8af4c") : secret "dns-default-metrics-tls" not found
Sep 11 04:40:20.107635 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:20.107571    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79f69c7884-pz9cm: secret "image-registry-tls" not found
Sep 11 04:40:20.107635 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:20.107631    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls podName:01a959b4-ae03-4d51-bb69-26b59b7e19a5 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:21.107613546 +0000 UTC m=+35.579077409 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls") pod "image-registry-79f69c7884-pz9cm" (UID: "01a959b4-ae03-4d51-bb69-26b59b7e19a5") : secret "image-registry-tls" not found
Sep 11 04:40:20.282130 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:20.282044    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2t5fm" event={"ID":"9285e03b-0880-4ae3-b25d-3d4cd81ba474","Type":"ContainerStarted","Data":"f3341323cdeb0f2ef185f194322a82671a45c1f93fc3cb6f7883b108cc47dcee"}
Sep 11 04:40:21.011912 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:21.011871    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:40:21.014458 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:21.014435    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 04:40:21.014593 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:21.014491    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rs4qk\""
Sep 11 04:40:21.115633 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:21.115603    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls\") pod \"dns-default-dt7d2\" (UID: \"b855f12e-f907-45d0-b777-5930f8a8af4c\") " pod="openshift-dns/dns-default-dt7d2"
Sep 11 04:40:21.115633 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:21.115635    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:21.115858 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:21.115772    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 04:40:21.115858 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:21.115778    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 04:40:21.115858 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:21.115798    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79f69c7884-pz9cm: secret "image-registry-tls" not found
Sep 11 04:40:21.115975 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:21.115862    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls podName:01a959b4-ae03-4d51-bb69-26b59b7e19a5 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:23.115842827 +0000 UTC m=+37.587306685 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls") pod "image-registry-79f69c7884-pz9cm" (UID: "01a959b4-ae03-4d51-bb69-26b59b7e19a5") : secret "image-registry-tls" not found
Sep 11 04:40:21.115975 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:21.115881    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls podName:b855f12e-f907-45d0-b777-5930f8a8af4c nodeName:}" failed. No retries permitted until 2026-09-11 04:40:23.115872414 +0000 UTC m=+37.587336267 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls") pod "dns-default-dt7d2" (UID: "b855f12e-f907-45d0-b777-5930f8a8af4c") : secret "dns-default-metrics-tls" not found
Sep 11 04:40:23.132118 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:23.131723    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls\") pod \"dns-default-dt7d2\" (UID: \"b855f12e-f907-45d0-b777-5930f8a8af4c\") " pod="openshift-dns/dns-default-dt7d2"
Sep 11 04:40:23.132471 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:23.132104    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:23.132471 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:23.131848    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 04:40:23.132471 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:23.132227    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls podName:b855f12e-f907-45d0-b777-5930f8a8af4c nodeName:}" failed. No retries permitted until 2026-09-11 04:40:27.132201531 +0000 UTC m=+41.603665388 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls") pod "dns-default-dt7d2" (UID: "b855f12e-f907-45d0-b777-5930f8a8af4c") : secret "dns-default-metrics-tls" not found
Sep 11 04:40:23.132471 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:23.132297    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 04:40:23.132471 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:23.132311    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79f69c7884-pz9cm: secret "image-registry-tls" not found
Sep 11 04:40:23.132471 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:23.132371    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls podName:01a959b4-ae03-4d51-bb69-26b59b7e19a5 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:27.132353726 +0000 UTC m=+41.603817585 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls") pod "image-registry-79f69c7884-pz9cm" (UID: "01a959b4-ae03-4d51-bb69-26b59b7e19a5") : secret "image-registry-tls" not found
Sep 11 04:40:23.291555 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:23.291525    2567 generic.go:356] "Generic (PLEG): container finished" podID="5232e65a-4484-4196-bb63-6fd0840e83e0" containerID="079aed1e55f190e9a8b6ca3e9987407221f04c9a53c26c4217e10ae336ef10bc" exitCode=0
Sep 11 04:40:23.291700 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:23.291565    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p8wsn" event={"ID":"5232e65a-4484-4196-bb63-6fd0840e83e0","Type":"ContainerDied","Data":"079aed1e55f190e9a8b6ca3e9987407221f04c9a53c26c4217e10ae336ef10bc"}
Sep 11 04:40:24.297928 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:24.297897    2567 generic.go:356] "Generic (PLEG): container finished" podID="5232e65a-4484-4196-bb63-6fd0840e83e0" containerID="f810943154ff3be28a2dc63e6ae47632aba47ee3e668e193bdcc532c68a03baa" exitCode=0
Sep 11 04:40:24.298427 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:24.297954    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p8wsn" event={"ID":"5232e65a-4484-4196-bb63-6fd0840e83e0","Type":"ContainerDied","Data":"f810943154ff3be28a2dc63e6ae47632aba47ee3e668e193bdcc532c68a03baa"}
Sep 11 04:40:24.932290 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:24.932260    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:25.304114 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:25.303999    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p8wsn" event={"ID":"5232e65a-4484-4196-bb63-6fd0840e83e0","Type":"ContainerStarted","Data":"3cb1355ce684b14065d5092d723870c0f65eab8e9161815bebcee5203342a539"}
Sep 11 04:40:25.305431 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:25.305408    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2t5fm" event={"ID":"9285e03b-0880-4ae3-b25d-3d4cd81ba474","Type":"ContainerStarted","Data":"6142a9639f8d152cc07d9363d985ca5cbf40b0de46c3e5f7d1fe6b38e6e70063"}
Sep 11 04:40:25.328014 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:25.327967    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-p8wsn" podStartSLOduration=7.49553896 podStartE2EDuration="39.327949948s" totalImagesPullingTime="31.832410988s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:39:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:25.326467642 +0000 UTC m=+39.797931517" watchObservedRunningTime="2026-09-11 04:40:25.327949948 +0000 UTC m=+39.799413826"
Sep 11 04:40:25.341241 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:25.341199    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2t5fm" podStartSLOduration=1.547033684 podStartE2EDuration="6.341185465s" totalImagesPullingTime="4.794151781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:40:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:25.340643971 +0000 UTC m=+39.812107848" watchObservedRunningTime="2026-09-11 04:40:25.341185465 +0000 UTC m=+39.812649342"
Sep 11 04:40:26.338227 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:26.338186    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:27.166697 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:27.166661    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls\") pod \"dns-default-dt7d2\" (UID: \"b855f12e-f907-45d0-b777-5930f8a8af4c\") " pod="openshift-dns/dns-default-dt7d2"
Sep 11 04:40:27.166697 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:27.166701    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:27.166981 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:27.166807    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 04:40:27.166981 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:27.166818    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79f69c7884-pz9cm: secret "image-registry-tls" not found
Sep 11 04:40:27.166981 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:27.166835    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 04:40:27.166981 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:27.166863    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls podName:01a959b4-ae03-4d51-bb69-26b59b7e19a5 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:35.166850188 +0000 UTC m=+49.638314047 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls") pod "image-registry-79f69c7884-pz9cm" (UID: "01a959b4-ae03-4d51-bb69-26b59b7e19a5") : secret "image-registry-tls" not found
Sep 11 04:40:27.166981 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:27.166908    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls podName:b855f12e-f907-45d0-b777-5930f8a8af4c nodeName:}" failed. No retries permitted until 2026-09-11 04:40:35.166890197 +0000 UTC m=+49.638354050 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls") pod "dns-default-dt7d2" (UID: "b855f12e-f907-45d0-b777-5930f8a8af4c") : secret "dns-default-metrics-tls" not found
Sep 11 04:40:27.677527 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:27.677495    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:27.778299 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:27.778260    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:28.571981 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:28.571949    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:28.671990 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:28.671954    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:28.729527 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:28.729492    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:29.556386 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:29.556359    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:29.656590 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:29.656561    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:33.086001 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:33.085968    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:33.152236 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:33.152208    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:33.209572 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:33.209545    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:34.080549 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:34.080511    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:34.131506 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:34.131467    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:35.219294 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:35.219204    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls\") pod \"dns-default-dt7d2\" (UID: \"b855f12e-f907-45d0-b777-5930f8a8af4c\") " pod="openshift-dns/dns-default-dt7d2"
Sep 11 04:40:35.219294 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:35.219239    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:35.219670 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:35.219345    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 04:40:35.219670 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:35.219348    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 04:40:35.219670 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:35.219418    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls podName:b855f12e-f907-45d0-b777-5930f8a8af4c nodeName:}" failed. No retries permitted until 2026-09-11 04:40:51.219402898 +0000 UTC m=+65.690866757 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls") pod "dns-default-dt7d2" (UID: "b855f12e-f907-45d0-b777-5930f8a8af4c") : secret "dns-default-metrics-tls" not found
Sep 11 04:40:35.219670 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:35.219356    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79f69c7884-pz9cm: secret "image-registry-tls" not found
Sep 11 04:40:35.219670 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:35.219471    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls podName:01a959b4-ae03-4d51-bb69-26b59b7e19a5 nodeName:}" failed. No retries permitted until 2026-09-11 04:40:51.21946089 +0000 UTC m=+65.690924777 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls") pod "image-registry-79f69c7884-pz9cm" (UID: "01a959b4-ae03-4d51-bb69-26b59b7e19a5") : secret "image-registry-tls" not found
Sep 11 04:40:39.278389 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:39.278362    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:40.202576 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:40.202536    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:40.257624 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:40.257592    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:44.555856 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:44.555817    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:44.652499 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:44.652472    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:44.752748 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:44.752723    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:45.276147 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:45.276114    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-x8zh9"
Sep 11 04:40:45.424155 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:45.424120    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:45.483849 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:45.483823    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:45.579978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:45.579894    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:45.674630 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:45.674598    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:45.805363 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:45.805332    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:45.867343 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:45.867244    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:49.159676 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:49.159630    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:49.259815 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:49.259772    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:50.327138 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:50.327103    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:50.427982 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:50.427948    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:40:50.826411 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:50.826384    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret\") pod \"global-pull-secret-syncer-9zqb2\" (UID: \"dbc886e3-0aab-4149-87f3-1d4ec91672f9\") " pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:40:50.826574 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:50.826438    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs\") pod \"network-metrics-daemon-dfv69\" (UID: \"0dc2bea8-03da-45f3-ac2b-d428b89a7542\") " pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:40:50.829219 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:50.829205    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 04:40:50.829281 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:50.829222    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 04:40:50.837134 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:50.837117    2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 11 04:40:50.837205 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:50.837171    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs podName:0dc2bea8-03da-45f3-ac2b-d428b89a7542 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:54.83715508 +0000 UTC m=+129.308618935 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs") pod "network-metrics-daemon-dfv69" (UID: "0dc2bea8-03da-45f3-ac2b-d428b89a7542") : secret "metrics-daemon-secret" not found
Sep 11 04:40:50.840421 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:50.840392    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dbc886e3-0aab-4149-87f3-1d4ec91672f9-original-pull-secret\") pod \"global-pull-secret-syncer-9zqb2\" (UID: \"dbc886e3-0aab-4149-87f3-1d4ec91672f9\") " pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:40:50.923231 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:50.923203    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zqb2"
Sep 11 04:40:50.927369 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:50.927347    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh7kw\" (UniqueName: \"kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw\") pod \"network-check-target-nbqjz\" (UID: \"ae3ae47e-fe4b-4298-a2fd-5935514af075\") " pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:40:50.930140 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:50.930119    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 04:40:50.940628 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:50.940612    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 04:40:50.951286 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:50.951266    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh7kw\" (UniqueName: \"kubernetes.io/projected/ae3ae47e-fe4b-4298-a2fd-5935514af075-kube-api-access-wh7kw\") pod \"network-check-target-nbqjz\" (UID: \"ae3ae47e-fe4b-4298-a2fd-5935514af075\") " pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:40:51.194238 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:51.194155    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9zqb2"]
Sep 11 04:40:51.197809 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:40:51.197776    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbc886e3_0aab_4149_87f3_1d4ec91672f9.slice/crio-956010d429839a948d911b345a2beb0bcc8eef6852568140011e05ee23d8da7d WatchSource:0}: Error finding container 956010d429839a948d911b345a2beb0bcc8eef6852568140011e05ee23d8da7d: Status 404 returned error can't find the container with id 956010d429839a948d911b345a2beb0bcc8eef6852568140011e05ee23d8da7d
Sep 11 04:40:51.230106 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:51.230060    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls\") pod \"dns-default-dt7d2\" (UID: \"b855f12e-f907-45d0-b777-5930f8a8af4c\") " pod="openshift-dns/dns-default-dt7d2"
Sep 11 04:40:51.230227 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:51.230117    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:40:51.230227 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:51.230207    2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 04:40:51.230316 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:51.230271    2567 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 04:40:51.230316 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:51.230287    2567 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-79f69c7884-pz9cm: secret "image-registry-tls" not found
Sep 11 04:40:51.230316 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:51.230277    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls podName:b855f12e-f907-45d0-b777-5930f8a8af4c nodeName:}" failed. No retries permitted until 2026-09-11 04:41:23.230262923 +0000 UTC m=+97.701726777 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls") pod "dns-default-dt7d2" (UID: "b855f12e-f907-45d0-b777-5930f8a8af4c") : secret "dns-default-metrics-tls" not found
Sep 11 04:40:51.230417 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:40:51.230330    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls podName:01a959b4-ae03-4d51-bb69-26b59b7e19a5 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:23.230319235 +0000 UTC m=+97.701783089 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls") pod "image-registry-79f69c7884-pz9cm" (UID: "01a959b4-ae03-4d51-bb69-26b59b7e19a5") : secret "image-registry-tls" not found
Sep 11 04:40:51.231204 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:51.231179    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6kl86\""
Sep 11 04:40:51.239839 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:51.239821    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:40:51.365877 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:51.365844    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9zqb2" event={"ID":"dbc886e3-0aab-4149-87f3-1d4ec91672f9","Type":"ContainerStarted","Data":"956010d429839a948d911b345a2beb0bcc8eef6852568140011e05ee23d8da7d"}
Sep 11 04:40:51.508343 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:51.508315    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nbqjz"]
Sep 11 04:40:51.511276 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:40:51.511251    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae3ae47e_fe4b_4298_a2fd_5935514af075.slice/crio-e82a4e1bcbe77af15d8d5d1a96561a20cf7093070687b77ddd7abe2ff714f7a9 WatchSource:0}: Error finding container e82a4e1bcbe77af15d8d5d1a96561a20cf7093070687b77ddd7abe2ff714f7a9: Status 404 returned error can't find the container with id e82a4e1bcbe77af15d8d5d1a96561a20cf7093070687b77ddd7abe2ff714f7a9
Sep 11 04:40:52.371820 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:52.371781    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nbqjz" event={"ID":"ae3ae47e-fe4b-4298-a2fd-5935514af075","Type":"ContainerStarted","Data":"e82a4e1bcbe77af15d8d5d1a96561a20cf7093070687b77ddd7abe2ff714f7a9"}
Sep 11 04:40:57.386235 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:57.386182    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9zqb2" event={"ID":"dbc886e3-0aab-4149-87f3-1d4ec91672f9","Type":"ContainerStarted","Data":"98ead13ab30e4d55073971fe3885250dd2ebd1355a94e5ad9afb95282d7b447b"}
Sep 11 04:40:57.387659 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:57.387624    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nbqjz" event={"ID":"ae3ae47e-fe4b-4298-a2fd-5935514af075","Type":"ContainerStarted","Data":"9dfd75ce04d7d2af06a580236bd02974b3de8a6bf76496867092056dfa69a6a5"}
Sep 11 04:40:57.387790 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:57.387673    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:40:57.401080 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:57.401033    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9zqb2" podStartSLOduration=65.848636638 podStartE2EDuration="1m11.4010189s" totalImagesPullingTime="5.552382262s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:57.400212427 +0000 UTC m=+71.871676300" watchObservedRunningTime="2026-09-11 04:40:57.4010189 +0000 UTC m=+71.872482776"
Sep 11 04:40:57.416380 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:40:57.416324    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nbqjz" podStartSLOduration=66.169206175 podStartE2EDuration="1m11.416312821s" totalImagesPullingTime="5.247106646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:40:57.415902052 +0000 UTC m=+71.887365940" watchObservedRunningTime="2026-09-11 04:40:57.416312821 +0000 UTC m=+71.887776698"
Sep 11 04:41:04.700568 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:04.700532    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-r2m4h"]
Sep 11 04:41:04.702752 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:04.702735    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-r2m4h"
Sep 11 04:41:04.705756 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:04.705728    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-k8lg5\""
Sep 11 04:41:04.713350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:04.713325    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-r2m4h"]
Sep 11 04:41:04.725732 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:04.725706    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdlq7\" (UniqueName: \"kubernetes.io/projected/be1a6cfe-22d0-43c7-812f-1aea55566a72-kube-api-access-mdlq7\") pod \"network-check-source-6f8b768455-r2m4h\" (UID: \"be1a6cfe-22d0-43c7-812f-1aea55566a72\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-r2m4h"
Sep 11 04:41:04.812842 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:04.812789    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:04.826866 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:04.826839    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdlq7\" (UniqueName: \"kubernetes.io/projected/be1a6cfe-22d0-43c7-812f-1aea55566a72-kube-api-access-mdlq7\") pod \"network-check-source-6f8b768455-r2m4h\" (UID: \"be1a6cfe-22d0-43c7-812f-1aea55566a72\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-r2m4h"
Sep 11 04:41:04.834046 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:04.834024    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdlq7\" (UniqueName: \"kubernetes.io/projected/be1a6cfe-22d0-43c7-812f-1aea55566a72-kube-api-access-mdlq7\") pod \"network-check-source-6f8b768455-r2m4h\" (UID: \"be1a6cfe-22d0-43c7-812f-1aea55566a72\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-r2m4h"
Sep 11 04:41:04.914863 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:04.914829    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:05.011033 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:05.011006    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-r2m4h"
Sep 11 04:41:05.279249 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:05.276445    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-r2m4h"]
Sep 11 04:41:05.280629 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:41:05.280593    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe1a6cfe_22d0_43c7_812f_1aea55566a72.slice/crio-74ea1402d6d82bd2f2d653e09169b58dbd9ec992a3d87090625d7967618aa89b WatchSource:0}: Error finding container 74ea1402d6d82bd2f2d653e09169b58dbd9ec992a3d87090625d7967618aa89b: Status 404 returned error can't find the container with id 74ea1402d6d82bd2f2d653e09169b58dbd9ec992a3d87090625d7967618aa89b
Sep 11 04:41:05.407656 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:05.407620    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-r2m4h" event={"ID":"be1a6cfe-22d0-43c7-812f-1aea55566a72","Type":"ContainerStarted","Data":"3c3fb46f47b2f4b602c94d849af2ffea89351f3115fd87fddde99563b63a9946"}
Sep 11 04:41:05.407847 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:05.407669    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-r2m4h" event={"ID":"be1a6cfe-22d0-43c7-812f-1aea55566a72","Type":"ContainerStarted","Data":"74ea1402d6d82bd2f2d653e09169b58dbd9ec992a3d87090625d7967618aa89b"}
Sep 11 04:41:05.423269 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:05.423229    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-r2m4h" podStartSLOduration=1.423213696 podStartE2EDuration="1.423213696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:41:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:05.422929467 +0000 UTC m=+79.894393343" watchObservedRunningTime="2026-09-11 04:41:05.423213696 +0000 UTC m=+79.894677570"
Sep 11 04:41:05.625909 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:05.625813    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:05.684240 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:05.684206    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:07.711838 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:07.711714    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:07.811889 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:07.811859    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:08.119810 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:08.119772    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:08.555845 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:08.555807    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:08.656386 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:08.656354    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:08.835468 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:08.835391    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:08.935323 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:08.935289    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:10.020880 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.020852    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-q9lj5"]
Sep 11 04:41:10.028931 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.028907    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:10.031349 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.031324    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 04:41:10.031471 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.031418    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 04:41:10.031559 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.031543    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-h277b\""
Sep 11 04:41:10.032345 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.032331    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 04:41:10.032431 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.032416    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 04:41:10.036981 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.036951    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q9lj5"]
Sep 11 04:41:10.055964 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.055938    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:10.062855 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.062838    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d9c0d97b-b5e9-4996-9091-84bab1fb8529-crio-socket\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:10.062926 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.062891    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9c0d97b-b5e9-4996-9091-84bab1fb8529-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:10.062967 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.062953    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9c0d97b-b5e9-4996-9091-84bab1fb8529-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:10.063000 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.062974    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d9c0d97b-b5e9-4996-9091-84bab1fb8529-data-volume\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:10.063034 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.063014    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n285t\" (UniqueName: \"kubernetes.io/projected/d9c0d97b-b5e9-4996-9091-84bab1fb8529-kube-api-access-n285t\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:10.115680 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.115639    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:10.164128 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.164100    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9c0d97b-b5e9-4996-9091-84bab1fb8529-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:10.164272 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.164137    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d9c0d97b-b5e9-4996-9091-84bab1fb8529-data-volume\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:10.164272 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.164194    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n285t\" (UniqueName: \"kubernetes.io/projected/d9c0d97b-b5e9-4996-9091-84bab1fb8529-kube-api-access-n285t\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:10.164272 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.164222    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d9c0d97b-b5e9-4996-9091-84bab1fb8529-crio-socket\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:10.164272 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:10.164251    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 04:41:10.164464 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.164276    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9c0d97b-b5e9-4996-9091-84bab1fb8529-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:10.164464 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:10.164324    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9c0d97b-b5e9-4996-9091-84bab1fb8529-insights-runtime-extractor-tls podName:d9c0d97b-b5e9-4996-9091-84bab1fb8529 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:10.664304227 +0000 UTC m=+85.135768095 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d9c0d97b-b5e9-4996-9091-84bab1fb8529-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q9lj5" (UID: "d9c0d97b-b5e9-4996-9091-84bab1fb8529") : secret "insights-runtime-extractor-tls" not found
Sep 11 04:41:10.164464 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:10.164351    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 04:41:10.164464 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.164358    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d9c0d97b-b5e9-4996-9091-84bab1fb8529-crio-socket\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:10.164464 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:10.164421    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9c0d97b-b5e9-4996-9091-84bab1fb8529-kube-rbac-proxy-cm podName:d9c0d97b-b5e9-4996-9091-84bab1fb8529 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:10.664403783 +0000 UTC m=+85.135867664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d9c0d97b-b5e9-4996-9091-84bab1fb8529-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q9lj5" (UID: "d9c0d97b-b5e9-4996-9091-84bab1fb8529") : configmap "kube-rbac-proxy" not found
Sep 11 04:41:10.164464 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.164459    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d9c0d97b-b5e9-4996-9091-84bab1fb8529-data-volume\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:10.174600 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.174579    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n285t\" (UniqueName: \"kubernetes.io/projected/d9c0d97b-b5e9-4996-9091-84bab1fb8529-kube-api-access-n285t\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:10.214391 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.214355    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:10.315236 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.315155    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:10.414611 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.414580    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-xpr6g"]
Sep 11 04:41:10.417544 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.417528    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-xpr6g"
Sep 11 04:41:10.419371 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.419343    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:10.419922 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.419897    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 11 04:41:10.420037 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.419904    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-d4x2x\""
Sep 11 04:41:10.420118 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.420101    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 11 04:41:10.427825 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.427803    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-xpr6g"]
Sep 11 04:41:10.466826 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.466800    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvp4f\" (UniqueName: \"kubernetes.io/projected/936d53ea-0a0d-4f20-9884-83c4a93f5550-kube-api-access-jvp4f\") pod \"migrator-8b4d697c5-xpr6g\" (UID: \"936d53ea-0a0d-4f20-9884-83c4a93f5550\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-xpr6g"
Sep 11 04:41:10.489603 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.489581    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:10.514706 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.514686    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-g7nhx"]
Sep 11 04:41:10.517418 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.517405    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-g7nhx"
Sep 11 04:41:10.519895 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.519878    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 11 04:41:10.520116 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.520092    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 11 04:41:10.520218 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.520094    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 11 04:41:10.520218 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.520143    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-ksxwq\""
Sep 11 04:41:10.520218 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.520148    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 11 04:41:10.525013 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.524994    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-g7nhx"]
Sep 11 04:41:10.567841 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.567788    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvp4f\" (UniqueName: \"kubernetes.io/projected/936d53ea-0a0d-4f20-9884-83c4a93f5550-kube-api-access-jvp4f\") pod \"migrator-8b4d697c5-xpr6g\" (UID: \"936d53ea-0a0d-4f20-9884-83c4a93f5550\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-xpr6g"
Sep 11 04:41:10.567841 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.567818    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ee02e12c-810c-4ee2-9bca-9a714638c46b-signing-key\") pod \"service-ca-568869fdc5-g7nhx\" (UID: \"ee02e12c-810c-4ee2-9bca-9a714638c46b\") " pod="openshift-service-ca/service-ca-568869fdc5-g7nhx"
Sep 11 04:41:10.567997 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.567862    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxfsc\" (UniqueName: \"kubernetes.io/projected/ee02e12c-810c-4ee2-9bca-9a714638c46b-kube-api-access-sxfsc\") pod \"service-ca-568869fdc5-g7nhx\" (UID: \"ee02e12c-810c-4ee2-9bca-9a714638c46b\") " pod="openshift-service-ca/service-ca-568869fdc5-g7nhx"
Sep 11 04:41:10.567997 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.567925    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ee02e12c-810c-4ee2-9bca-9a714638c46b-signing-cabundle\") pod \"service-ca-568869fdc5-g7nhx\" (UID: \"ee02e12c-810c-4ee2-9bca-9a714638c46b\") " pod="openshift-service-ca/service-ca-568869fdc5-g7nhx"
Sep 11 04:41:10.580463 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.580439    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvp4f\" (UniqueName: \"kubernetes.io/projected/936d53ea-0a0d-4f20-9884-83c4a93f5550-kube-api-access-jvp4f\") pod \"migrator-8b4d697c5-xpr6g\" (UID: \"936d53ea-0a0d-4f20-9884-83c4a93f5550\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-xpr6g"
Sep 11 04:41:10.589606 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.589582    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:10.669043 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.669013    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9c0d97b-b5e9-4996-9091-84bab1fb8529-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:10.669043 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.669047    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxfsc\" (UniqueName: \"kubernetes.io/projected/ee02e12c-810c-4ee2-9bca-9a714638c46b-kube-api-access-sxfsc\") pod \"service-ca-568869fdc5-g7nhx\" (UID: \"ee02e12c-810c-4ee2-9bca-9a714638c46b\") " pod="openshift-service-ca/service-ca-568869fdc5-g7nhx"
Sep 11 04:41:10.669258 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.669081    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ee02e12c-810c-4ee2-9bca-9a714638c46b-signing-cabundle\") pod \"service-ca-568869fdc5-g7nhx\" (UID: \"ee02e12c-810c-4ee2-9bca-9a714638c46b\") " pod="openshift-service-ca/service-ca-568869fdc5-g7nhx"
Sep 11 04:41:10.669258 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.669145    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ee02e12c-810c-4ee2-9bca-9a714638c46b-signing-key\") pod \"service-ca-568869fdc5-g7nhx\" (UID: \"ee02e12c-810c-4ee2-9bca-9a714638c46b\") " pod="openshift-service-ca/service-ca-568869fdc5-g7nhx"
Sep 11 04:41:10.669258 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.669164    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9c0d97b-b5e9-4996-9091-84bab1fb8529-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:10.669258 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:10.669162    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 04:41:10.669258 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:10.669249    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 04:41:10.669479 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:10.669257    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9c0d97b-b5e9-4996-9091-84bab1fb8529-insights-runtime-extractor-tls podName:d9c0d97b-b5e9-4996-9091-84bab1fb8529 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:11.669240408 +0000 UTC m=+86.140704285 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d9c0d97b-b5e9-4996-9091-84bab1fb8529-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q9lj5" (UID: "d9c0d97b-b5e9-4996-9091-84bab1fb8529") : secret "insights-runtime-extractor-tls" not found
Sep 11 04:41:10.669479 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:10.669306    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9c0d97b-b5e9-4996-9091-84bab1fb8529-kube-rbac-proxy-cm podName:d9c0d97b-b5e9-4996-9091-84bab1fb8529 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:11.669293708 +0000 UTC m=+86.140757565 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d9c0d97b-b5e9-4996-9091-84bab1fb8529-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q9lj5" (UID: "d9c0d97b-b5e9-4996-9091-84bab1fb8529") : configmap "kube-rbac-proxy" not found
Sep 11 04:41:10.669725 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.669700    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/ee02e12c-810c-4ee2-9bca-9a714638c46b-signing-cabundle\") pod \"service-ca-568869fdc5-g7nhx\" (UID: \"ee02e12c-810c-4ee2-9bca-9a714638c46b\") " pod="openshift-service-ca/service-ca-568869fdc5-g7nhx"
Sep 11 04:41:10.671635 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.671618    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/ee02e12c-810c-4ee2-9bca-9a714638c46b-signing-key\") pod \"service-ca-568869fdc5-g7nhx\" (UID: \"ee02e12c-810c-4ee2-9bca-9a714638c46b\") " pod="openshift-service-ca/service-ca-568869fdc5-g7nhx"
Sep 11 04:41:10.678055 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.678036    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxfsc\" (UniqueName: \"kubernetes.io/projected/ee02e12c-810c-4ee2-9bca-9a714638c46b-kube-api-access-sxfsc\") pod \"service-ca-568869fdc5-g7nhx\" (UID: \"ee02e12c-810c-4ee2-9bca-9a714638c46b\") " pod="openshift-service-ca/service-ca-568869fdc5-g7nhx"
Sep 11 04:41:10.727775 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.727741    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-xpr6g"
Sep 11 04:41:10.826513 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.826436    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-g7nhx"
Sep 11 04:41:10.992947 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:10.992917    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-xpr6g"]
Sep 11 04:41:10.995842 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:41:10.995814    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod936d53ea_0a0d_4f20_9884_83c4a93f5550.slice/crio-73b590a70a7a95aa559ceac1ea2e73906b21d647fa3ed99d87f627fce888e636 WatchSource:0}: Error finding container 73b590a70a7a95aa559ceac1ea2e73906b21d647fa3ed99d87f627fce888e636: Status 404 returned error can't find the container with id 73b590a70a7a95aa559ceac1ea2e73906b21d647fa3ed99d87f627fce888e636
Sep 11 04:41:11.098985 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:11.098902    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-g7nhx"]
Sep 11 04:41:11.102359 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:41:11.102335    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee02e12c_810c_4ee2_9bca_9a714638c46b.slice/crio-839ce8492d6ba2e69fa42acaa9ffcef3aa4e33f16795499dd2a7834dd96fc7b0 WatchSource:0}: Error finding container 839ce8492d6ba2e69fa42acaa9ffcef3aa4e33f16795499dd2a7834dd96fc7b0: Status 404 returned error can't find the container with id 839ce8492d6ba2e69fa42acaa9ffcef3aa4e33f16795499dd2a7834dd96fc7b0
Sep 11 04:41:11.424447 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:11.424369    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-g7nhx" event={"ID":"ee02e12c-810c-4ee2-9bca-9a714638c46b","Type":"ContainerStarted","Data":"839ce8492d6ba2e69fa42acaa9ffcef3aa4e33f16795499dd2a7834dd96fc7b0"}
Sep 11 04:41:11.425618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:11.425592    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-xpr6g" event={"ID":"936d53ea-0a0d-4f20-9884-83c4a93f5550","Type":"ContainerStarted","Data":"73b590a70a7a95aa559ceac1ea2e73906b21d647fa3ed99d87f627fce888e636"}
Sep 11 04:41:11.463476 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:11.463435    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:11.678204 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:11.678120    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9c0d97b-b5e9-4996-9091-84bab1fb8529-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:11.678204 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:11.678187    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9c0d97b-b5e9-4996-9091-84bab1fb8529-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:11.678429 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:11.678263    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 04:41:11.678429 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:11.678310    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 04:41:11.678429 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:11.678336    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9c0d97b-b5e9-4996-9091-84bab1fb8529-kube-rbac-proxy-cm podName:d9c0d97b-b5e9-4996-9091-84bab1fb8529 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:13.678314138 +0000 UTC m=+88.149777992 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d9c0d97b-b5e9-4996-9091-84bab1fb8529-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q9lj5" (UID: "d9c0d97b-b5e9-4996-9091-84bab1fb8529") : configmap "kube-rbac-proxy" not found
Sep 11 04:41:11.678429 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:11.678362    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9c0d97b-b5e9-4996-9091-84bab1fb8529-insights-runtime-extractor-tls podName:d9c0d97b-b5e9-4996-9091-84bab1fb8529 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:13.678346907 +0000 UTC m=+88.149810768 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d9c0d97b-b5e9-4996-9091-84bab1fb8529-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q9lj5" (UID: "d9c0d97b-b5e9-4996-9091-84bab1fb8529") : secret "insights-runtime-extractor-tls" not found
Sep 11 04:41:11.799021 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:11.798990    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-clstx_53a003c6-2a55-4b7c-9242-766116f41b2a/dns-node-resolver/0.log"
Sep 11 04:41:12.434649 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:12.434576    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-xpr6g" event={"ID":"936d53ea-0a0d-4f20-9884-83c4a93f5550","Type":"ContainerStarted","Data":"22e2114e1e82567aab74625c557b8b3cd3c4f5c4aba01c313618fc4bc5112329"}
Sep 11 04:41:12.435105 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:12.434669    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-xpr6g" event={"ID":"936d53ea-0a0d-4f20-9884-83c4a93f5550","Type":"ContainerStarted","Data":"935d290d6c799908ceae286c71868cb4dce5301653be7f71eb1463ca7872a14f"}
Sep 11 04:41:12.451850 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:12.451807    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-xpr6g" podStartSLOduration=1.386666142 podStartE2EDuration="2.451792199s" totalImagesPullingTime="1.065126057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:41:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:12.450970902 +0000 UTC m=+86.922434779" watchObservedRunningTime="2026-09-11 04:41:12.451792199 +0000 UTC m=+86.923256074"
Sep 11 04:41:12.997890 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:12.997871    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7thkg_57e4b5b7-b3cc-4052-be3c-07a138c81e91/node-ca/0.log"
Sep 11 04:41:13.439363 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:13.439326    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-g7nhx" event={"ID":"ee02e12c-810c-4ee2-9bca-9a714638c46b","Type":"ContainerStarted","Data":"0118ad31760073f83aea842cfb9c3cb9559844a4130265b04c9ffba7cbb7df5c"}
Sep 11 04:41:13.454283 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:13.454226    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-g7nhx" podStartSLOduration=1.603374069 podStartE2EDuration="3.454211487s" totalImagesPullingTime="1.850837418s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:41:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:13.453379259 +0000 UTC m=+87.924843133" watchObservedRunningTime="2026-09-11 04:41:13.454211487 +0000 UTC m=+87.925675363"
Sep 11 04:41:13.602313 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:13.602268    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2t5fm_9285e03b-0880-4ae3-b25d-3d4cd81ba474/serve-healthcheck-canary/0.log"
Sep 11 04:41:13.696190 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:13.696118    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9c0d97b-b5e9-4996-9091-84bab1fb8529-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:13.696321 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:13.696202    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9c0d97b-b5e9-4996-9091-84bab1fb8529-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:13.696321 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:13.696272    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 04:41:13.696321 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:13.696278    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 04:41:13.696424 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:13.696328    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9c0d97b-b5e9-4996-9091-84bab1fb8529-kube-rbac-proxy-cm podName:d9c0d97b-b5e9-4996-9091-84bab1fb8529 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:17.69631682 +0000 UTC m=+92.167780677 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d9c0d97b-b5e9-4996-9091-84bab1fb8529-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q9lj5" (UID: "d9c0d97b-b5e9-4996-9091-84bab1fb8529") : configmap "kube-rbac-proxy" not found
Sep 11 04:41:13.696424 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:13.696341    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9c0d97b-b5e9-4996-9091-84bab1fb8529-insights-runtime-extractor-tls podName:d9c0d97b-b5e9-4996-9091-84bab1fb8529 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:17.696335689 +0000 UTC m=+92.167799543 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d9c0d97b-b5e9-4996-9091-84bab1fb8529-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q9lj5" (UID: "d9c0d97b-b5e9-4996-9091-84bab1fb8529") : secret "insights-runtime-extractor-tls" not found
Sep 11 04:41:14.311613 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:14.311568    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:14.408151 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:14.408118    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:14.510325 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:14.510291    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:14.616080 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:14.615980    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:14.714274 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:14.714240    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:14.809573 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:14.809543    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:15.292503 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:15.292472    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:15.895000 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:15.894963    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:16.284789 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:16.284749    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:16.686523 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:16.686426    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:16.892901 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:16.892861    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:17.289604 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:17.289570    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:17.642686 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:17.642597    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:17.727547 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:17.727515    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9c0d97b-b5e9-4996-9091-84bab1fb8529-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:17.727709 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:17.727565    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9c0d97b-b5e9-4996-9091-84bab1fb8529-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:17.727709 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:17.727639    2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 04:41:17.727709 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:17.727706    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9c0d97b-b5e9-4996-9091-84bab1fb8529-kube-rbac-proxy-cm podName:d9c0d97b-b5e9-4996-9091-84bab1fb8529 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:25.727688609 +0000 UTC m=+100.199152467 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d9c0d97b-b5e9-4996-9091-84bab1fb8529-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q9lj5" (UID: "d9c0d97b-b5e9-4996-9091-84bab1fb8529") : configmap "kube-rbac-proxy" not found
Sep 11 04:41:17.727824 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:17.727739    2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 04:41:17.727824 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:17.727813    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9c0d97b-b5e9-4996-9091-84bab1fb8529-insights-runtime-extractor-tls podName:d9c0d97b-b5e9-4996-9091-84bab1fb8529 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:25.727794989 +0000 UTC m=+100.199258846 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d9c0d97b-b5e9-4996-9091-84bab1fb8529-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q9lj5" (UID: "d9c0d97b-b5e9-4996-9091-84bab1fb8529") : secret "insights-runtime-extractor-tls" not found
Sep 11 04:41:17.887978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:17.887939    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:18.086337 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:18.086303    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:18.392948 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:18.392854    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:18.493629 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:18.493586    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:19.088175 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:19.088134    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:19.149258 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:19.149230    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:19.687193 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:19.687164    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:19.890599 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:19.890566    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:20.097729 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:20.097695    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:20.291324 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:20.291290    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:20.583639 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:20.583610    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:21.491720 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:21.491685    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:21.690567 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:21.690533    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:21.791888 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:21.791850    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:21.892688 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:21.892654    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:22.082350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:22.082271    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:22.290364 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:22.290334    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:23.270106 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:23.270048    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls\") pod \"dns-default-dt7d2\" (UID: \"b855f12e-f907-45d0-b777-5930f8a8af4c\") " pod="openshift-dns/dns-default-dt7d2"
Sep 11 04:41:23.270106 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:23.270098    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:41:23.272580 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:23.272548    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b855f12e-f907-45d0-b777-5930f8a8af4c-metrics-tls\") pod \"dns-default-dt7d2\" (UID: \"b855f12e-f907-45d0-b777-5930f8a8af4c\") " pod="openshift-dns/dns-default-dt7d2"
Sep 11 04:41:23.273300 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:23.273274    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls\") pod \"image-registry-79f69c7884-pz9cm\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") " pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:41:23.287958 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:23.287931    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:23.340242 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:23.340224    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-fnkc4\""
Sep 11 04:41:23.347375 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:23.347354    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v459k\""
Sep 11 04:41:23.348373 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:23.348358    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:41:23.356216 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:23.356192    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-dt7d2"
Sep 11 04:41:23.624016 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:23.623936    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79f69c7884-pz9cm"]
Sep 11 04:41:23.626763 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:41:23.626719    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01a959b4_ae03_4d51_bb69_26b59b7e19a5.slice/crio-d92b98d23ad1439f3da7f78264ce124b7d6f22419047f2d5e0c75324eaef5a0a WatchSource:0}: Error finding container d92b98d23ad1439f3da7f78264ce124b7d6f22419047f2d5e0c75324eaef5a0a: Status 404 returned error can't find the container with id d92b98d23ad1439f3da7f78264ce124b7d6f22419047f2d5e0c75324eaef5a0a
Sep 11 04:41:23.642993 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:23.642971    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-dt7d2"]
Sep 11 04:41:23.646665 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:41:23.646641    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb855f12e_f907_45d0_b777_5930f8a8af4c.slice/crio-bb9345c716867fcba8ca61a7f527c7c5c1a0cd702d5f685973db7eb1f53cc38c WatchSource:0}: Error finding container bb9345c716867fcba8ca61a7f527c7c5c1a0cd702d5f685973db7eb1f53cc38c: Status 404 returned error can't find the container with id bb9345c716867fcba8ca61a7f527c7c5c1a0cd702d5f685973db7eb1f53cc38c
Sep 11 04:41:23.695024 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:23.694994    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:23.889682 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:23.889587    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:24.089645 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:24.089601    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:24.289750 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:24.289714    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:24.474598 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:24.474566    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dt7d2" event={"ID":"b855f12e-f907-45d0-b777-5930f8a8af4c","Type":"ContainerStarted","Data":"bb9345c716867fcba8ca61a7f527c7c5c1a0cd702d5f685973db7eb1f53cc38c"}
Sep 11 04:41:24.476599 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:24.476565    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79f69c7884-pz9cm" event={"ID":"01a959b4-ae03-4d51-bb69-26b59b7e19a5","Type":"ContainerStarted","Data":"25d11860c491720ebbe547787f2002707786b34aabc0320e77433cc11b568ef0"}
Sep 11 04:41:24.476753 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:24.476613    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79f69c7884-pz9cm" event={"ID":"01a959b4-ae03-4d51-bb69-26b59b7e19a5","Type":"ContainerStarted","Data":"d92b98d23ad1439f3da7f78264ce124b7d6f22419047f2d5e0c75324eaef5a0a"}
Sep 11 04:41:24.476753 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:24.476714    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:41:24.495633 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:24.495593    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-79f69c7884-pz9cm" podStartSLOduration=98.495579086 podStartE2EDuration="1m38.495579086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:39:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:24.494423546 +0000 UTC m=+98.965887423" watchObservedRunningTime="2026-09-11 04:41:24.495579086 +0000 UTC m=+98.967042943"
Sep 11 04:41:24.907514 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:24.907475    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:25.090314 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:25.090271    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:25.791318 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:25.791236    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9c0d97b-b5e9-4996-9091-84bab1fb8529-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:25.791318 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:25.791287    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9c0d97b-b5e9-4996-9091-84bab1fb8529-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:25.791719 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:25.791702    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9c0d97b-b5e9-4996-9091-84bab1fb8529-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:25.793660 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:25.793643    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9c0d97b-b5e9-4996-9091-84bab1fb8529-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9lj5\" (UID: \"d9c0d97b-b5e9-4996-9091-84bab1fb8529\") " pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:25.938192 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:25.938159    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q9lj5"
Sep 11 04:41:26.205218 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:26.205136    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q9lj5"]
Sep 11 04:41:26.208246 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:41:26.208220    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9c0d97b_b5e9_4996_9091_84bab1fb8529.slice/crio-a22595522d571f7f274cc624909d374a121060eaeeea08c3e12d96f34d4b963b WatchSource:0}: Error finding container a22595522d571f7f274cc624909d374a121060eaeeea08c3e12d96f34d4b963b: Status 404 returned error can't find the container with id a22595522d571f7f274cc624909d374a121060eaeeea08c3e12d96f34d4b963b
Sep 11 04:41:26.484834 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:26.484747    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dt7d2" event={"ID":"b855f12e-f907-45d0-b777-5930f8a8af4c","Type":"ContainerStarted","Data":"d1131f430ccbad4d3329fed935291a2acfcd56249e19af02e5324dfd53ec5f38"}
Sep 11 04:41:26.484834 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:26.484798    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-dt7d2" event={"ID":"b855f12e-f907-45d0-b777-5930f8a8af4c","Type":"ContainerStarted","Data":"6cd446f3f36ec6103a07040583dfa7188133b7e07fdd1c02a69d169325e8ed35"}
Sep 11 04:41:26.485026 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:26.484864    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-dt7d2"
Sep 11 04:41:26.486318 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:26.486297    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q9lj5" event={"ID":"d9c0d97b-b5e9-4996-9091-84bab1fb8529","Type":"ContainerStarted","Data":"a925b99f3cba29010cb8bce2ef70b8527de4bd1608e6bdfe1c48fef8d22fac80"}
Sep 11 04:41:26.486411 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:26.486334    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q9lj5" event={"ID":"d9c0d97b-b5e9-4996-9091-84bab1fb8529","Type":"ContainerStarted","Data":"a22595522d571f7f274cc624909d374a121060eaeeea08c3e12d96f34d4b963b"}
Sep 11 04:41:26.500436 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:26.500392    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-dt7d2" podStartSLOduration=65.620344707 podStartE2EDuration="1m7.500379494s" totalImagesPullingTime="1.880034787s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:40:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:26.499488391 +0000 UTC m=+100.970952267" watchObservedRunningTime="2026-09-11 04:41:26.500379494 +0000 UTC m=+100.971843370"
Sep 11 04:41:27.491409 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:27.491373    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q9lj5" event={"ID":"d9c0d97b-b5e9-4996-9091-84bab1fb8529","Type":"ContainerStarted","Data":"ec513ba628bc5fd134bd65b6ed1fb4db25e0a27bc6a23147ace9613aa130dfe8"}
Sep 11 04:41:27.687088 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:27.687032    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:27.736954 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:27.736915    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:28.392576 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:28.392553    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nbqjz"
Sep 11 04:41:29.501732 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:29.501695    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q9lj5" event={"ID":"d9c0d97b-b5e9-4996-9091-84bab1fb8529","Type":"ContainerStarted","Data":"8346f539d59a3bd65519dab3b12f3ee2122c7590d8b50f29e22f582e633d6c2f"}
Sep 11 04:41:29.522436 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:29.522394    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-q9lj5" podStartSLOduration=17.450365603 podStartE2EDuration="19.522380854s" totalImagesPullingTime="2.072015251s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 04:41:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:29.522027941 +0000 UTC m=+103.993491817" watchObservedRunningTime="2026-09-11 04:41:29.522380854 +0000 UTC m=+103.993844729"
Sep 11 04:41:33.193969 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:33.193933    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f89c5f85b-7hxwz"]
Sep 11 04:41:33.196623 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:33.196595    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f89c5f85b-7hxwz"
Sep 11 04:41:33.200147 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:33.200120    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 11 04:41:33.200277 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:33.200126    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 11 04:41:33.200277 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:33.200169    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 11 04:41:33.200277 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:33.200209    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-lvxrf\""
Sep 11 04:41:33.200277 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:33.200224    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 11 04:41:33.210922 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:33.210904    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f89c5f85b-7hxwz"]
Sep 11 04:41:33.298858 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:33.298828    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:33.339976 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:33.339944    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zrcp\" (UniqueName: \"kubernetes.io/projected/6879acba-cacd-4e6d-9878-32cad1a6fd1d-kube-api-access-9zrcp\") pod \"managed-serviceaccount-addon-agent-5f89c5f85b-7hxwz\" (UID: \"6879acba-cacd-4e6d-9878-32cad1a6fd1d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f89c5f85b-7hxwz"
Sep 11 04:41:33.340127 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:33.339979    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6879acba-cacd-4e6d-9878-32cad1a6fd1d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5f89c5f85b-7hxwz\" (UID: \"6879acba-cacd-4e6d-9878-32cad1a6fd1d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f89c5f85b-7hxwz"
Sep 11 04:41:33.392748 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:33.392714    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:33.441030 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:33.441005    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zrcp\" (UniqueName: \"kubernetes.io/projected/6879acba-cacd-4e6d-9878-32cad1a6fd1d-kube-api-access-9zrcp\") pod \"managed-serviceaccount-addon-agent-5f89c5f85b-7hxwz\" (UID: \"6879acba-cacd-4e6d-9878-32cad1a6fd1d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f89c5f85b-7hxwz"
Sep 11 04:41:33.441030 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:33.441031    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6879acba-cacd-4e6d-9878-32cad1a6fd1d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5f89c5f85b-7hxwz\" (UID: \"6879acba-cacd-4e6d-9878-32cad1a6fd1d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f89c5f85b-7hxwz"
Sep 11 04:41:33.443553 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:33.443532    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6879acba-cacd-4e6d-9878-32cad1a6fd1d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5f89c5f85b-7hxwz\" (UID: \"6879acba-cacd-4e6d-9878-32cad1a6fd1d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f89c5f85b-7hxwz"
Sep 11 04:41:33.449383 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:33.449327    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zrcp\" (UniqueName: \"kubernetes.io/projected/6879acba-cacd-4e6d-9878-32cad1a6fd1d-kube-api-access-9zrcp\") pod \"managed-serviceaccount-addon-agent-5f89c5f85b-7hxwz\" (UID: \"6879acba-cacd-4e6d-9878-32cad1a6fd1d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f89c5f85b-7hxwz"
Sep 11 04:41:33.513538 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:33.513512    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f89c5f85b-7hxwz"
Sep 11 04:41:33.780059 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:33.780029    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f89c5f85b-7hxwz"]
Sep 11 04:41:33.782981 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:41:33.782952    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6879acba_cacd_4e6d_9878_32cad1a6fd1d.slice/crio-bcac739e0c297e8dc4f64759c0055930545699c46ecba7a103a5b0bb2d8e45cc WatchSource:0}: Error finding container bcac739e0c297e8dc4f64759c0055930545699c46ecba7a103a5b0bb2d8e45cc: Status 404 returned error can't find the container with id bcac739e0c297e8dc4f64759c0055930545699c46ecba7a103a5b0bb2d8e45cc
Sep 11 04:41:34.520487 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:34.520451    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f89c5f85b-7hxwz" event={"ID":"6879acba-cacd-4e6d-9878-32cad1a6fd1d","Type":"ContainerStarted","Data":"bcac739e0c297e8dc4f64759c0055930545699c46ecba7a103a5b0bb2d8e45cc"}
Sep 11 04:41:35.290833 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:35.290797    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:35.348943 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:35.348914    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:36.493582 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:36.493501    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-dt7d2"
Sep 11 04:41:36.528284 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:36.528245    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f89c5f85b-7hxwz" event={"ID":"6879acba-cacd-4e6d-9878-32cad1a6fd1d","Type":"ContainerStarted","Data":"44ad6154585a5b6c0d1067d2ddf878779aa3163f0afa91ffa32eed7761006b5a"}
Sep 11 04:41:36.546016 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:36.545973    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f89c5f85b-7hxwz" podStartSLOduration=1.2197891140000001 podStartE2EDuration="3.545958846s" totalImagesPullingTime="2.326169732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:41:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:36.545252029 +0000 UTC m=+111.016715906" watchObservedRunningTime="2026-09-11 04:41:36.545958846 +0000 UTC m=+111.017422722"
Sep 11 04:41:39.186182 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:39.186150    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:39.282044 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:39.282010    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:39.341485 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:39.341451    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pm2cn"]
Sep 11 04:41:39.344614 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:39.344593    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pm2cn"
Sep 11 04:41:39.345650 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:39.345617    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:39.346785 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:39.346760    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 11 04:41:39.346885 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:39.346810    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-6jt4t\""
Sep 11 04:41:39.352000 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:39.351978    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pm2cn"]
Sep 11 04:41:39.491648 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:39.491555    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/543c405c-52a2-4808-a21a-ea6141fbeb1e-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-pm2cn\" (UID: \"543c405c-52a2-4808-a21a-ea6141fbeb1e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pm2cn"
Sep 11 04:41:39.592599 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:39.592566    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/543c405c-52a2-4808-a21a-ea6141fbeb1e-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-pm2cn\" (UID: \"543c405c-52a2-4808-a21a-ea6141fbeb1e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pm2cn"
Sep 11 04:41:39.592769 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:39.592677    2567 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 11 04:41:39.592769 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:39.592727    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/543c405c-52a2-4808-a21a-ea6141fbeb1e-tls-certificates podName:543c405c-52a2-4808-a21a-ea6141fbeb1e nodeName:}" failed. No retries permitted until 2026-09-11 04:41:40.092714084 +0000 UTC m=+114.564177938 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/543c405c-52a2-4808-a21a-ea6141fbeb1e-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-pm2cn" (UID: "543c405c-52a2-4808-a21a-ea6141fbeb1e") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 11 04:41:40.096889 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:40.096852    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/543c405c-52a2-4808-a21a-ea6141fbeb1e-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-pm2cn\" (UID: \"543c405c-52a2-4808-a21a-ea6141fbeb1e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pm2cn"
Sep 11 04:41:40.097107 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:40.096978    2567 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 11 04:41:40.097107 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:40.097038    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/543c405c-52a2-4808-a21a-ea6141fbeb1e-tls-certificates podName:543c405c-52a2-4808-a21a-ea6141fbeb1e nodeName:}" failed. No retries permitted until 2026-09-11 04:41:41.097025456 +0000 UTC m=+115.568489311 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/543c405c-52a2-4808-a21a-ea6141fbeb1e-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-pm2cn" (UID: "543c405c-52a2-4808-a21a-ea6141fbeb1e") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 11 04:41:40.343556 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:40.343522    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:41.103441 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:41.103391    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/543c405c-52a2-4808-a21a-ea6141fbeb1e-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-pm2cn\" (UID: \"543c405c-52a2-4808-a21a-ea6141fbeb1e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pm2cn"
Sep 11 04:41:41.105949 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:41.105918    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/543c405c-52a2-4808-a21a-ea6141fbeb1e-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-pm2cn\" (UID: \"543c405c-52a2-4808-a21a-ea6141fbeb1e\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pm2cn"
Sep 11 04:41:41.154084 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:41.154037    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pm2cn"
Sep 11 04:41:41.420545 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:41.420473    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pm2cn"]
Sep 11 04:41:41.423176 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:41:41.423152    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod543c405c_52a2_4808_a21a_ea6141fbeb1e.slice/crio-740a8087aa7db6eb3586e267052604a2b6db8c53a9d42581ba7f8b3256303bcd WatchSource:0}: Error finding container 740a8087aa7db6eb3586e267052604a2b6db8c53a9d42581ba7f8b3256303bcd: Status 404 returned error can't find the container with id 740a8087aa7db6eb3586e267052604a2b6db8c53a9d42581ba7f8b3256303bcd
Sep 11 04:41:41.544054 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:41.544022    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pm2cn" event={"ID":"543c405c-52a2-4808-a21a-ea6141fbeb1e","Type":"ContainerStarted","Data":"740a8087aa7db6eb3586e267052604a2b6db8c53a9d42581ba7f8b3256303bcd"}
Sep 11 04:41:43.352903 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:43.352871    2567 patch_prober.go:28] interesting pod/image-registry-79f69c7884-pz9cm container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 11 04:41:43.353285 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:43.352921    2567 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-79f69c7884-pz9cm" podUID="01a959b4-ae03-4d51-bb69-26b59b7e19a5" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 04:41:43.551423 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:43.551388    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pm2cn" event={"ID":"543c405c-52a2-4808-a21a-ea6141fbeb1e","Type":"ContainerStarted","Data":"db4400784c2b8b6d9749ad56dc991a82afeabb479d82fa141ea3afe1efc7e142"}
Sep 11 04:41:43.551615 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:43.551592    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pm2cn"
Sep 11 04:41:43.556039 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:43.556015    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pm2cn"
Sep 11 04:41:43.566505 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:43.566467    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pm2cn" podStartSLOduration=3.453822106 podStartE2EDuration="4.566454345s" totalImagesPullingTime="1.112632239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:41:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:43.566206691 +0000 UTC m=+118.037670568" watchObservedRunningTime="2026-09-11 04:41:43.566454345 +0000 UTC m=+118.037918221"
Sep 11 04:41:44.365883 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:44.365850    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:44.430395 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:44.430355    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:44.504964 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:44.504926    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:45.484690 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:45.484666    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:41:45.629939 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:45.629899    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:45.685633 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:45.685606    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:45.757911 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:45.757883    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:46.539532 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:46.539495    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:47.869834 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:47.869805    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:47.971674 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:47.971640    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:48.496702 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.496664    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:48.705138 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.705105    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:48.726811 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.726779    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-7jp2s"]
Sep 11 04:41:48.731470 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.731454    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jp2s"
Sep 11 04:41:48.733999 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.733980    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 11 04:41:48.734917 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.734899    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 04:41:48.735043 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.734902    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 04:41:48.735182 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.734920    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-9t9xg\""
Sep 11 04:41:48.735283 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.734971    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 04:41:48.735377 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.734984    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 11 04:41:48.741464 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.741433    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-7jp2s"]
Sep 11 04:41:48.753974 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.753918    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lxknm"]
Sep 11 04:41:48.758254 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.758235    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.760892 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.760873    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 04:41:48.761006 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.760910    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 04:41:48.761006 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.760990    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 04:41:48.761617 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.761598    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hr4hw\""
Sep 11 04:41:48.817580 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.817547    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:48.859670 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.859627    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgdvn\" (UniqueName: \"kubernetes.io/projected/205359f4-7e46-436d-8118-7fc925558ef5-kube-api-access-fgdvn\") pod \"openshift-state-metrics-67dcf8c979-7jp2s\" (UID: \"205359f4-7e46-436d-8118-7fc925558ef5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jp2s"
Sep 11 04:41:48.859855 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.859677    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/205359f4-7e46-436d-8118-7fc925558ef5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-7jp2s\" (UID: \"205359f4-7e46-436d-8118-7fc925558ef5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jp2s"
Sep 11 04:41:48.859855 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.859772    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d396c1c-8868-4964-9a46-b02a0d5ffc42-metrics-client-ca\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.859855 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.859813    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1d396c1c-8868-4964-9a46-b02a0d5ffc42-node-exporter-textfile\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.859997 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.859866    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1d396c1c-8868-4964-9a46-b02a0d5ffc42-node-exporter-tls\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.859997 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.859922    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/205359f4-7e46-436d-8118-7fc925558ef5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-7jp2s\" (UID: \"205359f4-7e46-436d-8118-7fc925558ef5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jp2s"
Sep 11 04:41:48.860098 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.859992    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1d396c1c-8868-4964-9a46-b02a0d5ffc42-node-exporter-accelerators-collector-config\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.860098 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.860026    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1d396c1c-8868-4964-9a46-b02a0d5ffc42-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.860098 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.860049    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfmv8\" (UniqueName: \"kubernetes.io/projected/1d396c1c-8868-4964-9a46-b02a0d5ffc42-kube-api-access-pfmv8\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.860216 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.860110    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1d396c1c-8868-4964-9a46-b02a0d5ffc42-root\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.860216 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.860154    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d396c1c-8868-4964-9a46-b02a0d5ffc42-sys\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.860216 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.860188    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/205359f4-7e46-436d-8118-7fc925558ef5-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-7jp2s\" (UID: \"205359f4-7e46-436d-8118-7fc925558ef5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jp2s"
Sep 11 04:41:48.860216 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.860212    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1d396c1c-8868-4964-9a46-b02a0d5ffc42-node-exporter-wtmp\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.882987 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.882960    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:48.951977 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.951947    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:48.960681 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.960655    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1d396c1c-8868-4964-9a46-b02a0d5ffc42-node-exporter-textfile\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.960811 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.960702    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1d396c1c-8868-4964-9a46-b02a0d5ffc42-node-exporter-tls\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.960811 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.960756    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/205359f4-7e46-436d-8118-7fc925558ef5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-7jp2s\" (UID: \"205359f4-7e46-436d-8118-7fc925558ef5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jp2s"
Sep 11 04:41:48.960908 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.960832    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1d396c1c-8868-4964-9a46-b02a0d5ffc42-node-exporter-accelerators-collector-config\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.960908 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.960868    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1d396c1c-8868-4964-9a46-b02a0d5ffc42-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.961002 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:48.960955    2567 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Sep 11 04:41:48.961051 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:48.961032    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/205359f4-7e46-436d-8118-7fc925558ef5-openshift-state-metrics-tls podName:205359f4-7e46-436d-8118-7fc925558ef5 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:49.461010153 +0000 UTC m=+123.932474022 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/205359f4-7e46-436d-8118-7fc925558ef5-openshift-state-metrics-tls") pod "openshift-state-metrics-67dcf8c979-7jp2s" (UID: "205359f4-7e46-436d-8118-7fc925558ef5") : secret "openshift-state-metrics-tls" not found
Sep 11 04:41:48.961310 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.961255    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfmv8\" (UniqueName: \"kubernetes.io/projected/1d396c1c-8868-4964-9a46-b02a0d5ffc42-kube-api-access-pfmv8\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.961310 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.961299    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1d396c1c-8868-4964-9a46-b02a0d5ffc42-root\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.961471 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.961328    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d396c1c-8868-4964-9a46-b02a0d5ffc42-sys\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.961471 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.961351    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1d396c1c-8868-4964-9a46-b02a0d5ffc42-node-exporter-textfile\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.961471 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.961354    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/205359f4-7e46-436d-8118-7fc925558ef5-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-7jp2s\" (UID: \"205359f4-7e46-436d-8118-7fc925558ef5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jp2s"
Sep 11 04:41:48.961471 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.961386    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1d396c1c-8868-4964-9a46-b02a0d5ffc42-node-exporter-wtmp\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.961471 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.961412    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgdvn\" (UniqueName: \"kubernetes.io/projected/205359f4-7e46-436d-8118-7fc925558ef5-kube-api-access-fgdvn\") pod \"openshift-state-metrics-67dcf8c979-7jp2s\" (UID: \"205359f4-7e46-436d-8118-7fc925558ef5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jp2s"
Sep 11 04:41:48.961471 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.961424    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1d396c1c-8868-4964-9a46-b02a0d5ffc42-root\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.961471 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.961447    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/205359f4-7e46-436d-8118-7fc925558ef5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-7jp2s\" (UID: \"205359f4-7e46-436d-8118-7fc925558ef5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jp2s"
Sep 11 04:41:48.961800 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.961488    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d396c1c-8868-4964-9a46-b02a0d5ffc42-metrics-client-ca\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.961800 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.961550    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1d396c1c-8868-4964-9a46-b02a0d5ffc42-node-exporter-accelerators-collector-config\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.961800 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.961697    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1d396c1c-8868-4964-9a46-b02a0d5ffc42-node-exporter-wtmp\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.961800 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.961697    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1d396c1c-8868-4964-9a46-b02a0d5ffc42-sys\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.962001 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.961962    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1d396c1c-8868-4964-9a46-b02a0d5ffc42-metrics-client-ca\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.962571 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.962550    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/205359f4-7e46-436d-8118-7fc925558ef5-metrics-client-ca\") pod \"openshift-state-metrics-67dcf8c979-7jp2s\" (UID: \"205359f4-7e46-436d-8118-7fc925558ef5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jp2s"
Sep 11 04:41:48.964106 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.964061    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1d396c1c-8868-4964-9a46-b02a0d5ffc42-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.964515 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.964494    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/205359f4-7e46-436d-8118-7fc925558ef5-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-67dcf8c979-7jp2s\" (UID: \"205359f4-7e46-436d-8118-7fc925558ef5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jp2s"
Sep 11 04:41:48.964595 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.964536    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1d396c1c-8868-4964-9a46-b02a0d5ffc42-node-exporter-tls\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.970609 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.970567    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfmv8\" (UniqueName: \"kubernetes.io/projected/1d396c1c-8868-4964-9a46-b02a0d5ffc42-kube-api-access-pfmv8\") pod \"node-exporter-lxknm\" (UID: \"1d396c1c-8868-4964-9a46-b02a0d5ffc42\") " pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:48.971523 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:48.971500    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgdvn\" (UniqueName: \"kubernetes.io/projected/205359f4-7e46-436d-8118-7fc925558ef5-kube-api-access-fgdvn\") pod \"openshift-state-metrics-67dcf8c979-7jp2s\" (UID: \"205359f4-7e46-436d-8118-7fc925558ef5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jp2s"
Sep 11 04:41:49.008813 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.008755    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:49.069624 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.069597    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lxknm"
Sep 11 04:41:49.094024 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:41:49.093988    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d396c1c_8868_4964_9a46_b02a0d5ffc42.slice/crio-1c3657a30a59774ea4f9d766ad0586a866326c68d286671b9b1508de36f54930 WatchSource:0}: Error finding container 1c3657a30a59774ea4f9d766ad0586a866326c68d286671b9b1508de36f54930: Status 404 returned error can't find the container with id 1c3657a30a59774ea4f9d766ad0586a866326c68d286671b9b1508de36f54930
Sep 11 04:41:49.109421 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.109378    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:49.209722 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.209694    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:49.332000 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.331916    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:49.465328 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.465295    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/205359f4-7e46-436d-8118-7fc925558ef5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-7jp2s\" (UID: \"205359f4-7e46-436d-8118-7fc925558ef5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jp2s"
Sep 11 04:41:49.467769 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.467741    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/205359f4-7e46-436d-8118-7fc925558ef5-openshift-state-metrics-tls\") pod \"openshift-state-metrics-67dcf8c979-7jp2s\" (UID: \"205359f4-7e46-436d-8118-7fc925558ef5\") " pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jp2s"
Sep 11 04:41:49.485839 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.485802    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:49.570551 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.570514    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxknm" event={"ID":"1d396c1c-8868-4964-9a46-b02a0d5ffc42","Type":"ContainerStarted","Data":"1c3657a30a59774ea4f9d766ad0586a866326c68d286671b9b1508de36f54930"}
Sep 11 04:41:49.620626 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.620534    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:49.642234 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.642210    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jp2s"
Sep 11 04:41:49.727530 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.727394    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:49.807283 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.807161    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:49.814192 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.814167    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 04:41:49.820179 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.820156    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:49.822698 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.822674    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 11 04:41:49.823008 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.822985    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 11 04:41:49.823152 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.823016    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 11 04:41:49.823408 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.823276    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 11 04:41:49.823408 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.823289    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 11 04:41:49.823408 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.823374    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 11 04:41:49.823408 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.823406    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7ctts\""
Sep 11 04:41:49.823816 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.823678    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 11 04:41:49.823816 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.823727    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 11 04:41:49.823954 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.823928    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 11 04:41:49.834640 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.834622    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 04:41:49.923971 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.923899    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-67dcf8c979-7jp2s"]
Sep 11 04:41:49.926877 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.926849    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:49.969716 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.969687    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cc0d0076-7e12-4f8d-8f11-f076e57defa4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:49.969828 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.969734    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:49.969828 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.969768    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:49.969828 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.969785    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhxmf\" (UniqueName: \"kubernetes.io/projected/cc0d0076-7e12-4f8d-8f11-f076e57defa4-kube-api-access-mhxmf\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:49.969989 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.969873    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-config-volume\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:49.969989 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.969908    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cc0d0076-7e12-4f8d-8f11-f076e57defa4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:49.969989 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.969934    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cc0d0076-7e12-4f8d-8f11-f076e57defa4-config-out\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:49.970149 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.969983    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:49.970149 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.970054    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc0d0076-7e12-4f8d-8f11-f076e57defa4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:49.970149 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.970105    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:49.970149 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.970140    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-web-config\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:49.970323 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.970211    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc0d0076-7e12-4f8d-8f11-f076e57defa4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:49.970323 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:49.970241    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.051665 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:41:50.051634    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod205359f4_7e46_436d_8118_7fc925558ef5.slice/crio-7115262f7513f66658db4e1fcb0ee9e4d520eea61f87e5a13b72918746f6179a WatchSource:0}: Error finding container 7115262f7513f66658db4e1fcb0ee9e4d520eea61f87e5a13b72918746f6179a: Status 404 returned error can't find the container with id 7115262f7513f66658db4e1fcb0ee9e4d520eea61f87e5a13b72918746f6179a
Sep 11 04:41:50.070888 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.070857    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc0d0076-7e12-4f8d-8f11-f076e57defa4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.071008 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.070903    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.071008 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.070928    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cc0d0076-7e12-4f8d-8f11-f076e57defa4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.071160 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.071106    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.071218 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.071161    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.071218 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.071191    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhxmf\" (UniqueName: \"kubernetes.io/projected/cc0d0076-7e12-4f8d-8f11-f076e57defa4-kube-api-access-mhxmf\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.071317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.071229    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-config-volume\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.071317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.071253    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cc0d0076-7e12-4f8d-8f11-f076e57defa4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.071317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.071277    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cc0d0076-7e12-4f8d-8f11-f076e57defa4-config-out\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.071317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.071299    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.071317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.071305    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cc0d0076-7e12-4f8d-8f11-f076e57defa4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.071568 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.071326    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc0d0076-7e12-4f8d-8f11-f076e57defa4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.071568 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.071368    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.071568 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.071394    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-web-config\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.072470 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.072399    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc0d0076-7e12-4f8d-8f11-f076e57defa4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.072645 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.072538    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc0d0076-7e12-4f8d-8f11-f076e57defa4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.074206 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.074179    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.074698 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.074674    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-web-config\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.074781 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.074737    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cc0d0076-7e12-4f8d-8f11-f076e57defa4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.074852 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.074777    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.075185 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.075162    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.075291 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.075223    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.075451 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.075432    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-config-volume\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.076342 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.076324    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cc0d0076-7e12-4f8d-8f11-f076e57defa4-config-out\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.076610 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.076592    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.079703 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.079685    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhxmf\" (UniqueName: \"kubernetes.io/projected/cc0d0076-7e12-4f8d-8f11-f076e57defa4-kube-api-access-mhxmf\") pod \"alertmanager-main-0\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.132523 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.132496    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:41:50.414090 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.414051    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 04:41:50.416399 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:41:50.416371    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc0d0076_7e12_4f8d_8f11_f076e57defa4.slice/crio-50cb7c3ee289a8dd1e545a333a8c70f7e2f8bbc62fc01222ce20e3b59d978520 WatchSource:0}: Error finding container 50cb7c3ee289a8dd1e545a333a8c70f7e2f8bbc62fc01222ce20e3b59d978520: Status 404 returned error can't find the container with id 50cb7c3ee289a8dd1e545a333a8c70f7e2f8bbc62fc01222ce20e3b59d978520
Sep 11 04:41:50.575209 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.575173    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jp2s" event={"ID":"205359f4-7e46-436d-8118-7fc925558ef5","Type":"ContainerStarted","Data":"a7af6a9a8c136ddf8f396db377274ca59ce10b0c3b7aef3d3184c1d5f192f10d"}
Sep 11 04:41:50.575364 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.575233    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jp2s" event={"ID":"205359f4-7e46-436d-8118-7fc925558ef5","Type":"ContainerStarted","Data":"190f11ec0e23dc885365a22e02476fb7528650a0039cf90ea947cc943849e342"}
Sep 11 04:41:50.575364 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.575260    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jp2s" event={"ID":"205359f4-7e46-436d-8118-7fc925558ef5","Type":"ContainerStarted","Data":"7115262f7513f66658db4e1fcb0ee9e4d520eea61f87e5a13b72918746f6179a"}
Sep 11 04:41:50.576769 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.576746    2567 generic.go:356] "Generic (PLEG): container finished" podID="1d396c1c-8868-4964-9a46-b02a0d5ffc42" containerID="64dfeb6be164f5a46d8eb88055dfd86698343044cfb7c5dcbea06287581918eb" exitCode=0
Sep 11 04:41:50.576857 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.576821    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxknm" event={"ID":"1d396c1c-8868-4964-9a46-b02a0d5ffc42","Type":"ContainerDied","Data":"64dfeb6be164f5a46d8eb88055dfd86698343044cfb7c5dcbea06287581918eb"}
Sep 11 04:41:50.578118 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:50.578097    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc0d0076-7e12-4f8d-8f11-f076e57defa4","Type":"ContainerStarted","Data":"50cb7c3ee289a8dd1e545a333a8c70f7e2f8bbc62fc01222ce20e3b59d978520"}
Sep 11 04:41:51.584953 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.584925    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxknm" event={"ID":"1d396c1c-8868-4964-9a46-b02a0d5ffc42","Type":"ContainerStarted","Data":"350724a90bb8e6900b66b91ce82abdf0784fb61f3004bf17c143fdbcd142cb59"}
Sep 11 04:41:51.585324 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.584975    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxknm" event={"ID":"1d396c1c-8868-4964-9a46-b02a0d5ffc42","Type":"ContainerStarted","Data":"b17701ca62b71c921b967790109e8ce269767453202e4f384a1505f3eef8e9e5"}
Sep 11 04:41:51.611233 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.611169    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lxknm" podStartSLOduration=2.524000285 podStartE2EDuration="3.611148666s" totalImagesPullingTime="1.00486792s" totalInitContainerRuntime="82.280461ms" isStatefulPod=true podCreationTimestamp="2026-09-11 04:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:51.608911015 +0000 UTC m=+126.080374891" watchObservedRunningTime="2026-09-11 04:41:51.611148666 +0000 UTC m=+126.082612544"
Sep 11 04:41:51.673999 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.673963    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:51.731686 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.731661    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"]
Sep 11 04:41:51.735078 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.735051    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.737506 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.737483    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 11 04:41:51.737633 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.737554    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-77ov28n951qad\""
Sep 11 04:41:51.737633 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.737581    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 11 04:41:51.737779 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.737494    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 11 04:41:51.737779 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.737486    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 11 04:41:51.737779 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.737716    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 11 04:41:51.737779 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.737728    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-8l9sd\""
Sep 11 04:41:51.747802 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.747780    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"]
Sep 11 04:41:51.753668 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.753634    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:51.887252 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.887178    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b8b4da4-4461-4b19-a007-9031ae775cde-secret-grpc-tls\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.887252 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.887211    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc6dp\" (UniqueName: \"kubernetes.io/projected/9b8b4da4-4461-4b19-a007-9031ae775cde-kube-api-access-fc6dp\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.887252 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.887229    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9b8b4da4-4461-4b19-a007-9031ae775cde-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.887252 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.887247    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9b8b4da4-4461-4b19-a007-9031ae775cde-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.887502 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.887318    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b8b4da4-4461-4b19-a007-9031ae775cde-metrics-client-ca\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.887502 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.887344    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9b8b4da4-4461-4b19-a007-9031ae775cde-secret-thanos-querier-tls\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.887502 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.887384    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b8b4da4-4461-4b19-a007-9031ae775cde-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.887502 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.887404    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b8b4da4-4461-4b19-a007-9031ae775cde-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.938162 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.938114    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:51.988108 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.988057    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b8b4da4-4461-4b19-a007-9031ae775cde-secret-grpc-tls\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.988243 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.988120    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc6dp\" (UniqueName: \"kubernetes.io/projected/9b8b4da4-4461-4b19-a007-9031ae775cde-kube-api-access-fc6dp\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.988243 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.988154    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9b8b4da4-4461-4b19-a007-9031ae775cde-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.988243 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.988179    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9b8b4da4-4461-4b19-a007-9031ae775cde-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.988401 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.988243    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b8b4da4-4461-4b19-a007-9031ae775cde-metrics-client-ca\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.988401 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.988278    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9b8b4da4-4461-4b19-a007-9031ae775cde-secret-thanos-querier-tls\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.988506 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.988411    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b8b4da4-4461-4b19-a007-9031ae775cde-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.988506 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.988453    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b8b4da4-4461-4b19-a007-9031ae775cde-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.989206 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.989156    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9b8b4da4-4461-4b19-a007-9031ae775cde-metrics-client-ca\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.991185 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.991163    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9b8b4da4-4461-4b19-a007-9031ae775cde-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.991380 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.991360    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9b8b4da4-4461-4b19-a007-9031ae775cde-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.991453 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.991440    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9b8b4da4-4461-4b19-a007-9031ae775cde-secret-grpc-tls\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.991492 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.991461    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9b8b4da4-4461-4b19-a007-9031ae775cde-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.991540 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.991515    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9b8b4da4-4461-4b19-a007-9031ae775cde-secret-thanos-querier-tls\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.991615 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.991596    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9b8b4da4-4461-4b19-a007-9031ae775cde-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:51.999991 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:51.999969    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc6dp\" (UniqueName: \"kubernetes.io/projected/9b8b4da4-4461-4b19-a007-9031ae775cde-kube-api-access-fc6dp\") pod \"thanos-querier-6788cfb4b9-xvtz9\" (UID: \"9b8b4da4-4461-4b19-a007-9031ae775cde\") " pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:52.039655 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:52.039616    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:52.044522 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:52.044504    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:52.316681 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:52.316649    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"]
Sep 11 04:41:52.320219 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:41:52.320196    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b8b4da4_4461_4b19_a007_9031ae775cde.slice/crio-193236b89c6a3debb2fe6f83f75fa459ec5769a14ba0f548f6e407a4290a4e1d WatchSource:0}: Error finding container 193236b89c6a3debb2fe6f83f75fa459ec5769a14ba0f548f6e407a4290a4e1d: Status 404 returned error can't find the container with id 193236b89c6a3debb2fe6f83f75fa459ec5769a14ba0f548f6e407a4290a4e1d
Sep 11 04:41:52.593893 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:52.593807    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9" event={"ID":"9b8b4da4-4461-4b19-a007-9031ae775cde","Type":"ContainerStarted","Data":"193236b89c6a3debb2fe6f83f75fa459ec5769a14ba0f548f6e407a4290a4e1d"}
Sep 11 04:41:52.595801 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:52.595770    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jp2s" event={"ID":"205359f4-7e46-436d-8118-7fc925558ef5","Type":"ContainerStarted","Data":"58fcec3c1ec130e850cb120c690cda7568bcd7d200008878e98f70c544ce2b41"}
Sep 11 04:41:52.597356 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:52.597328    2567 generic.go:356] "Generic (PLEG): container finished" podID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerID="f317d56c3d916518a04b962cb2df147f9461ed74f282520a00ee9a50d56c64da" exitCode=0
Sep 11 04:41:52.597474 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:52.597426    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc0d0076-7e12-4f8d-8f11-f076e57defa4","Type":"ContainerDied","Data":"f317d56c3d916518a04b962cb2df147f9461ed74f282520a00ee9a50d56c64da"}
Sep 11 04:41:52.613702 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:52.613657    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-67dcf8c979-7jp2s" podStartSLOduration=3.302124529 podStartE2EDuration="4.61364323s" totalImagesPullingTime="1.311518701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:41:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:52.612598578 +0000 UTC m=+127.084062454" watchObservedRunningTime="2026-09-11 04:41:52.61364323 +0000 UTC m=+127.085107105"
Sep 11 04:41:53.166240 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.166208    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-67fc898877-796tx"]
Sep 11 04:41:53.169457 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.169430    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:41:53.172233 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.172204    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 11 04:41:53.173377 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.173358    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 11 04:41:53.173484 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.173382    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 11 04:41:53.174205 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.174146    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-7w7m4\""
Sep 11 04:41:53.174442 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.174414    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-98259d3poctlq\""
Sep 11 04:41:53.174685 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.174667    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 11 04:41:53.192543 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.192522    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-67fc898877-796tx"]
Sep 11 04:41:53.298999 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.298848    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/277ab579-b700-4770-ac93-8fe753be5622-secret-metrics-server-client-certs\") pod \"metrics-server-67fc898877-796tx\" (UID: \"277ab579-b700-4770-ac93-8fe753be5622\") " pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:41:53.298999 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.298894    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z8fh\" (UniqueName: \"kubernetes.io/projected/277ab579-b700-4770-ac93-8fe753be5622-kube-api-access-6z8fh\") pod \"metrics-server-67fc898877-796tx\" (UID: \"277ab579-b700-4770-ac93-8fe753be5622\") " pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:41:53.298999 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.298920    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/277ab579-b700-4770-ac93-8fe753be5622-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-67fc898877-796tx\" (UID: \"277ab579-b700-4770-ac93-8fe753be5622\") " pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:41:53.298999 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.298941    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/277ab579-b700-4770-ac93-8fe753be5622-client-ca-bundle\") pod \"metrics-server-67fc898877-796tx\" (UID: \"277ab579-b700-4770-ac93-8fe753be5622\") " pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:41:53.298999 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.298984    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/277ab579-b700-4770-ac93-8fe753be5622-audit-log\") pod \"metrics-server-67fc898877-796tx\" (UID: \"277ab579-b700-4770-ac93-8fe753be5622\") " pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:41:53.299338 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.299144    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/277ab579-b700-4770-ac93-8fe753be5622-secret-metrics-server-tls\") pod \"metrics-server-67fc898877-796tx\" (UID: \"277ab579-b700-4770-ac93-8fe753be5622\") " pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:41:53.299338 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.299187    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/277ab579-b700-4770-ac93-8fe753be5622-metrics-server-audit-profiles\") pod \"metrics-server-67fc898877-796tx\" (UID: \"277ab579-b700-4770-ac93-8fe753be5622\") " pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:41:53.399711 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.399681    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/277ab579-b700-4770-ac93-8fe753be5622-audit-log\") pod \"metrics-server-67fc898877-796tx\" (UID: \"277ab579-b700-4770-ac93-8fe753be5622\") " pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:41:53.399873 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.399792    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/277ab579-b700-4770-ac93-8fe753be5622-secret-metrics-server-tls\") pod \"metrics-server-67fc898877-796tx\" (UID: \"277ab579-b700-4770-ac93-8fe753be5622\") " pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:41:53.399873 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.399826    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/277ab579-b700-4770-ac93-8fe753be5622-metrics-server-audit-profiles\") pod \"metrics-server-67fc898877-796tx\" (UID: \"277ab579-b700-4770-ac93-8fe753be5622\") " pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:41:53.399997 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.399881    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/277ab579-b700-4770-ac93-8fe753be5622-secret-metrics-server-client-certs\") pod \"metrics-server-67fc898877-796tx\" (UID: \"277ab579-b700-4770-ac93-8fe753be5622\") " pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:41:53.399997 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.399916    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z8fh\" (UniqueName: \"kubernetes.io/projected/277ab579-b700-4770-ac93-8fe753be5622-kube-api-access-6z8fh\") pod \"metrics-server-67fc898877-796tx\" (UID: \"277ab579-b700-4770-ac93-8fe753be5622\") " pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:41:53.399997 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.399947    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/277ab579-b700-4770-ac93-8fe753be5622-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-67fc898877-796tx\" (UID: \"277ab579-b700-4770-ac93-8fe753be5622\") " pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:41:53.399997 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.399980    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/277ab579-b700-4770-ac93-8fe753be5622-client-ca-bundle\") pod \"metrics-server-67fc898877-796tx\" (UID: \"277ab579-b700-4770-ac93-8fe753be5622\") " pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:41:53.400238 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.400144    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/277ab579-b700-4770-ac93-8fe753be5622-audit-log\") pod \"metrics-server-67fc898877-796tx\" (UID: \"277ab579-b700-4770-ac93-8fe753be5622\") " pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:41:53.400923 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.400878    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/277ab579-b700-4770-ac93-8fe753be5622-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-67fc898877-796tx\" (UID: \"277ab579-b700-4770-ac93-8fe753be5622\") " pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:41:53.401171 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.401148    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/277ab579-b700-4770-ac93-8fe753be5622-metrics-server-audit-profiles\") pod \"metrics-server-67fc898877-796tx\" (UID: \"277ab579-b700-4770-ac93-8fe753be5622\") " pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:41:53.402793 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.402770    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/277ab579-b700-4770-ac93-8fe753be5622-secret-metrics-server-tls\") pod \"metrics-server-67fc898877-796tx\" (UID: \"277ab579-b700-4770-ac93-8fe753be5622\") " pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:41:53.402949 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.402923    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/277ab579-b700-4770-ac93-8fe753be5622-secret-metrics-server-client-certs\") pod \"metrics-server-67fc898877-796tx\" (UID: \"277ab579-b700-4770-ac93-8fe753be5622\") " pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:41:53.403674 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.403657    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/277ab579-b700-4770-ac93-8fe753be5622-client-ca-bundle\") pod \"metrics-server-67fc898877-796tx\" (UID: \"277ab579-b700-4770-ac93-8fe753be5622\") " pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:41:53.408870 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.408843    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z8fh\" (UniqueName: \"kubernetes.io/projected/277ab579-b700-4770-ac93-8fe753be5622-kube-api-access-6z8fh\") pod \"metrics-server-67fc898877-796tx\" (UID: \"277ab579-b700-4770-ac93-8fe753be5622\") " pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:41:53.480676 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.480593    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:41:53.525097 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.525006    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-69vlz"]
Sep 11 04:41:53.529790 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.529768    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-69vlz"
Sep 11 04:41:53.533336 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.533224    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:53.533801 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.533780    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 11 04:41:53.537462 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.535768    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-br5rg\""
Sep 11 04:41:53.537462 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.536792    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-69vlz"]
Sep 11 04:41:53.703426 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.703386    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhjzj\" (UniqueName: \"kubernetes.io/projected/0b9dee47-fd70-4c08-ae4b-4aa8580e0614-kube-api-access-zhjzj\") pod \"monitoring-plugin-5d85778b9d-69vlz\" (UID: \"0b9dee47-fd70-4c08-ae4b-4aa8580e0614\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-69vlz"
Sep 11 04:41:53.703877 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.703507    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0b9dee47-fd70-4c08-ae4b-4aa8580e0614-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-69vlz\" (UID: \"0b9dee47-fd70-4c08-ae4b-4aa8580e0614\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-69vlz"
Sep 11 04:41:53.774460 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.774212    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-67fc898877-796tx"]
Sep 11 04:41:53.804642 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.804614    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0b9dee47-fd70-4c08-ae4b-4aa8580e0614-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-69vlz\" (UID: \"0b9dee47-fd70-4c08-ae4b-4aa8580e0614\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-69vlz"
Sep 11 04:41:53.804800 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.804725    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhjzj\" (UniqueName: \"kubernetes.io/projected/0b9dee47-fd70-4c08-ae4b-4aa8580e0614-kube-api-access-zhjzj\") pod \"monitoring-plugin-5d85778b9d-69vlz\" (UID: \"0b9dee47-fd70-4c08-ae4b-4aa8580e0614\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-69vlz"
Sep 11 04:41:53.804800 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:53.804781    2567 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 11 04:41:53.804897 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:41:53.804861    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0b9dee47-fd70-4c08-ae4b-4aa8580e0614-monitoring-plugin-cert podName:0b9dee47-fd70-4c08-ae4b-4aa8580e0614 nodeName:}" failed. No retries permitted until 2026-09-11 04:41:54.304840003 +0000 UTC m=+128.776303865 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/0b9dee47-fd70-4c08-ae4b-4aa8580e0614-monitoring-plugin-cert") pod "monitoring-plugin-5d85778b9d-69vlz" (UID: "0b9dee47-fd70-4c08-ae4b-4aa8580e0614") : secret "monitoring-plugin-cert" not found
Sep 11 04:41:53.813892 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.813860    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhjzj\" (UniqueName: \"kubernetes.io/projected/0b9dee47-fd70-4c08-ae4b-4aa8580e0614-kube-api-access-zhjzj\") pod \"monitoring-plugin-5d85778b9d-69vlz\" (UID: \"0b9dee47-fd70-4c08-ae4b-4aa8580e0614\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-69vlz"
Sep 11 04:41:53.946866 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.946831    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:53.983726 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.983699    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"]
Sep 11 04:41:53.987178 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.987157    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:53.989907 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.989662    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 11 04:41:53.989907 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.989662    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 11 04:41:53.989907 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.989744    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 11 04:41:53.989907 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.989751    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 11 04:41:53.990204 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.989995    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-9jtd6\""
Sep 11 04:41:53.990204 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.989999    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 11 04:41:53.994225 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:53.994205    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 11 04:41:54.002441 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.002414    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"]
Sep 11 04:41:54.107202 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.107121    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-federate-client-tls\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.107202 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.107165    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.107410 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.107248    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-serving-certs-ca-bundle\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.107410 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.107294    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9vgk\" (UniqueName: \"kubernetes.io/projected/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-kube-api-access-d9vgk\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.107410 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.107326    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.107410 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.107376    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-telemeter-client-tls\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.107588 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.107440    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-metrics-client-ca\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.107588 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.107509    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-secret-telemeter-client\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.126950 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:41:54.126922    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod277ab579_b700_4770_ac93_8fe753be5622.slice/crio-f61ea7fd0909526a56983d767a4590830199a4d58c1c95d303c118e13c534ba8 WatchSource:0}: Error finding container f61ea7fd0909526a56983d767a4590830199a4d58c1c95d303c118e13c534ba8: Status 404 returned error can't find the container with id f61ea7fd0909526a56983d767a4590830199a4d58c1c95d303c118e13c534ba8
Sep 11 04:41:54.208388 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.208354    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-telemeter-client-tls\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.208543 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.208407    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-metrics-client-ca\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.208543 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.208452    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-secret-telemeter-client\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.208543 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.208478    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-federate-client-tls\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.208543 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.208540    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.208761 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.208608    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-serving-certs-ca-bundle\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.208761 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.208660    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9vgk\" (UniqueName: \"kubernetes.io/projected/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-kube-api-access-d9vgk\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.208761 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.208692    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.209392 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.209366    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-metrics-client-ca\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.209838 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.209806    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-serving-certs-ca-bundle\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.210010 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.209960    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.211513 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.211488    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.211616 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.211597    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-telemeter-client-tls\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.211928 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.211887    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-federate-client-tls\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.212094 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.212050    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-secret-telemeter-client\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.216759 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.216736    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9vgk\" (UniqueName: \"kubernetes.io/projected/cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a-kube-api-access-d9vgk\") pod \"telemeter-client-576f54fd8d-4gjnr\" (UID: \"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a\") " pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.298292 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.298247    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"
Sep 11 04:41:54.309932 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.309906    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0b9dee47-fd70-4c08-ae4b-4aa8580e0614-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-69vlz\" (UID: \"0b9dee47-fd70-4c08-ae4b-4aa8580e0614\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-69vlz"
Sep 11 04:41:54.312909 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.312882    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/0b9dee47-fd70-4c08-ae4b-4aa8580e0614-monitoring-plugin-cert\") pod \"monitoring-plugin-5d85778b9d-69vlz\" (UID: \"0b9dee47-fd70-4c08-ae4b-4aa8580e0614\") " pod="openshift-monitoring/monitoring-plugin-5d85778b9d-69vlz"
Sep 11 04:41:54.446944 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.446856    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-69vlz"
Sep 11 04:41:54.606224 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.606046    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-67fc898877-796tx" event={"ID":"277ab579-b700-4770-ac93-8fe753be5622","Type":"ContainerStarted","Data":"f61ea7fd0909526a56983d767a4590830199a4d58c1c95d303c118e13c534ba8"}
Sep 11 04:41:54.608478 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.608360    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9" event={"ID":"9b8b4da4-4461-4b19-a007-9031ae775cde","Type":"ContainerStarted","Data":"f26df66477925651b7fcebc1dcc6a535093257c636b78e284eeaa9410e9c54a2"}
Sep 11 04:41:54.795032 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.794543    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5d85778b9d-69vlz"]
Sep 11 04:41:54.798638 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.798551    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-576f54fd8d-4gjnr"]
Sep 11 04:41:54.916012 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.915986    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs\") pod \"network-metrics-daemon-dfv69\" (UID: \"0dc2bea8-03da-45f3-ac2b-d428b89a7542\") " pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:41:54.918883 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.918858    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0dc2bea8-03da-45f3-ac2b-d428b89a7542-metrics-certs\") pod \"network-metrics-daemon-dfv69\" (UID: \"0dc2bea8-03da-45f3-ac2b-d428b89a7542\") " pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:41:54.925017 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.924996    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-rs4qk\""
Sep 11 04:41:54.933036 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.932873    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 04:41:54.933036 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.932930    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dfv69"
Sep 11 04:41:54.936781 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.936764    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:54.939374 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.939351    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 04:41:54.939495 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.939477    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 04:41:54.940862 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.940841    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 04:41:54.940995 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.940977    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 04:41:54.941517 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.941234    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 04:41:54.941517 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.941426    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 04:41:54.941654 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.941618    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 04:41:54.941705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.941658    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 04:41:54.942156 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.941994    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-v7lvc\""
Sep 11 04:41:54.942156 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.942006    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 04:41:54.942156 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.942006    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 04:41:54.942846 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.942613    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 04:41:54.942846 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.942613    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 04:41:54.942846 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.942759    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 04:41:54.943734 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.943714    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d0sr6ln40s2k\""
Sep 11 04:41:54.944603 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.944582    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 04:41:54.959281 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.959201    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 04:41:54.963973 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:54.963936    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:55.118386 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.118165    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.118386 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.118229    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.118386 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.118264    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.118386 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.118301    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ckp8\" (UniqueName: \"kubernetes.io/projected/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-kube-api-access-4ckp8\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.118386 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.118346    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-web-config\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.118781 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.118403    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-config-out\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.118781 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.118451    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-config\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.118781 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.118492    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.118781 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.118524    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.118781 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.118549    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.118781 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.118571    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.118781 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.118630    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.118781 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.118660    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.118781 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.118683    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.118781 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.118732    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.118781 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.118759    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.118781 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.118782    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.119270 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.118823    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.119270 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.118865    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.119270 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.118922    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.221411 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.221021    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.221411 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.221092    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.223165 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.221635    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.223165 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.221687    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.223165 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.221724    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.223165 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.221757    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.223165 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.221782    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ckp8\" (UniqueName: \"kubernetes.io/projected/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-kube-api-access-4ckp8\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.223165 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.221824    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-web-config\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.223165 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.221857    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-config-out\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.223165 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.221896    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-config\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.223165 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.221924    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.223165 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.221953    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.223165 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.221983    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.223165 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.222011    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.223165 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.222056    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.223165 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.222105    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.223165 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.222135    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.223165 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.222194    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.223165 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.222227    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.224290 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.222256    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.224290 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.222896    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dfv69"]
Sep 11 04:41:55.224980 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.224577    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.228165 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.226623    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-79f69c7884-pz9cm"]
Sep 11 04:41:55.228165 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.227034    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.228165 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.227278    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.228366 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.228345    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.229125 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.229098    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.230800 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.230770    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-config\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.234381 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.231880    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.234381 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.232273    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.234381 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.232447    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.234381 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.233173    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.234381 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.233247    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.234381 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.233847    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.234381 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.233914    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.234381 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.234371    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.234875 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.234767    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.234875 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.234793    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-config-out\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.236153 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.236127    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.236933 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.236867    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.237467 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.237423    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-web-config\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.242494 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.242474    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ckp8\" (UniqueName: \"kubernetes.io/projected/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-kube-api-access-4ckp8\") pod \"prometheus-k8s-0\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.250012 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.249344    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:41:55.305724 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.305670    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:55.391900 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.391845    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:55.491759 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.491673    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:55.615907 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.615824    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr" event={"ID":"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a","Type":"ContainerStarted","Data":"1664515fa87ec4d968af2510c3c56c928739bae14c94c59d5398785699537582"}
Sep 11 04:41:55.620293 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.620264    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc0d0076-7e12-4f8d-8f11-f076e57defa4","Type":"ContainerStarted","Data":"5ab94724c62a7e59fcfb01d81cce4650b2e7800b00e461092fa2bbc424776b8d"}
Sep 11 04:41:55.620436 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.620371    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc0d0076-7e12-4f8d-8f11-f076e57defa4","Type":"ContainerStarted","Data":"bceb9d50bd3c869634514c2711aeff1a1729cb0f88ed4439db708cca38739fc6"}
Sep 11 04:41:55.620514 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.620435    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc0d0076-7e12-4f8d-8f11-f076e57defa4","Type":"ContainerStarted","Data":"bdfd41657a2233ac1a17dba2fcde36df2df83464331a2b8f23279bdf7299dd64"}
Sep 11 04:41:55.620514 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.620498    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc0d0076-7e12-4f8d-8f11-f076e57defa4","Type":"ContainerStarted","Data":"8b9ac9e90898dbb3cab6e31b1875b10c288bce40a63d8ca9486bf65d13067e83"}
Sep 11 04:41:55.620623 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.620561    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc0d0076-7e12-4f8d-8f11-f076e57defa4","Type":"ContainerStarted","Data":"db1bae133f91984ea4ffaed655a856a34ad27614a3fddf50276ef3f82157d092"}
Sep 11 04:41:55.623610 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.623581    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9" event={"ID":"9b8b4da4-4461-4b19-a007-9031ae775cde","Type":"ContainerStarted","Data":"754c00bbc33abfaa9bc76328d2d3e6138f3055b6a3e69735b48b8c8cf73adb16"}
Sep 11 04:41:55.623732 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.623655    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9" event={"ID":"9b8b4da4-4461-4b19-a007-9031ae775cde","Type":"ContainerStarted","Data":"aff72a35ddeaf47fefb478567761288d5d48f9a4c4bdfa7a7f6bfcef0389629e"}
Sep 11 04:41:55.625251 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:55.625203    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-69vlz" event={"ID":"0b9dee47-fd70-4c08-ae4b-4aa8580e0614","Type":"ContainerStarted","Data":"b057995f9a374733c3cd44a3dbcb47e43fdd90e26ee901dfc12a9102e4a62c4a"}
Sep 11 04:41:55.894782 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:41:55.894743    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0dc2bea8_03da_45f3_ac2b_d428b89a7542.slice/crio-25bb089092d91c1ba743c002322e2f4d33934446344a4ee8a4d7ce3efa3bd4ea WatchSource:0}: Error finding container 25bb089092d91c1ba743c002322e2f4d33934446344a4ee8a4d7ce3efa3bd4ea: Status 404 returned error can't find the container with id 25bb089092d91c1ba743c002322e2f4d33934446344a4ee8a4d7ce3efa3bd4ea
Sep 11 04:41:56.575167 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:56.575137    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 04:41:56.631181 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:56.631143    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dfv69" event={"ID":"0dc2bea8-03da-45f3-ac2b-d428b89a7542","Type":"ContainerStarted","Data":"25bb089092d91c1ba743c002322e2f4d33934446344a4ee8a4d7ce3efa3bd4ea"}
Sep 11 04:41:56.821996 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:41:56.821962    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcea99d18_0ff7_41b6_b7e3_e7fc6f35f003.slice/crio-5f8f5e08163d59fe63528f50208ef0fddcbac7c9b8610d4378c527ef2e7fedc4 WatchSource:0}: Error finding container 5f8f5e08163d59fe63528f50208ef0fddcbac7c9b8610d4378c527ef2e7fedc4: Status 404 returned error can't find the container with id 5f8f5e08163d59fe63528f50208ef0fddcbac7c9b8610d4378c527ef2e7fedc4
Sep 11 04:41:56.940443 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:56.940403    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:56.998308 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:56.998271    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:57.638977 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.638876    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9" event={"ID":"9b8b4da4-4461-4b19-a007-9031ae775cde","Type":"ContainerStarted","Data":"ba475b6cef8bfdd5008dd6a04917acf9d6a9737a815074d186cd823579824f7f"}
Sep 11 04:41:57.639167 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.638984    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9" event={"ID":"9b8b4da4-4461-4b19-a007-9031ae775cde","Type":"ContainerStarted","Data":"63414a3585b0f23093b9226e5a61fceab8c75ae5482b69de7e991e45183cda90"}
Sep 11 04:41:57.639167 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.639015    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9" event={"ID":"9b8b4da4-4461-4b19-a007-9031ae775cde","Type":"ContainerStarted","Data":"f2080596cca4565c86fb31199bdf2f234714ee43e969716eaf3201e4d22c522f"}
Sep 11 04:41:57.639167 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.639044    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:41:57.640573 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.640549    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-69vlz" event={"ID":"0b9dee47-fd70-4c08-ae4b-4aa8580e0614","Type":"ContainerStarted","Data":"ff5c8810e17dbac2dd4434131c4f94eb1a5e5e922dc723f2e811877d193dea4a"}
Sep 11 04:41:57.640742 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.640722    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-69vlz"
Sep 11 04:41:57.642719 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.642691    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dfv69" event={"ID":"0dc2bea8-03da-45f3-ac2b-d428b89a7542","Type":"ContainerStarted","Data":"5b495962d9a11396bf1f92882ea90a3ba4ce041a33f9d71cbcc0ee9ac4ab062f"}
Sep 11 04:41:57.642821 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.642737    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dfv69" event={"ID":"0dc2bea8-03da-45f3-ac2b-d428b89a7542","Type":"ContainerStarted","Data":"2fb3887be8b1d22e222d0ac3842c3db48afe10ea6e354c90802ca5e1d6268235"}
Sep 11 04:41:57.645133 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.645105    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr" event={"ID":"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a","Type":"ContainerStarted","Data":"6f43a47dceaa27253669bd17b92a091592692871a336a3fd9c577fc779136156"}
Sep 11 04:41:57.645230 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.645156    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr" event={"ID":"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a","Type":"ContainerStarted","Data":"61086b436f3fc1c279fb2e1d9450c0d67ce178feac3c78072cbfbd80df268d04"}
Sep 11 04:41:57.645230 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.645183    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr" event={"ID":"cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a","Type":"ContainerStarted","Data":"c7ac38554f9fe8daa4a70bccd02283edf11a3f1feefd079dd90b8ecedac4747b"}
Sep 11 04:41:57.646497 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.646475    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-69vlz"
Sep 11 04:41:57.646949 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.646927    2567 generic.go:356] "Generic (PLEG): container finished" podID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerID="602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2" exitCode=0
Sep 11 04:41:57.647045 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.646999    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003","Type":"ContainerDied","Data":"602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2"}
Sep 11 04:41:57.647116 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.647045    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003","Type":"ContainerStarted","Data":"5f8f5e08163d59fe63528f50208ef0fddcbac7c9b8610d4378c527ef2e7fedc4"}
Sep 11 04:41:57.651349 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.651318    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc0d0076-7e12-4f8d-8f11-f076e57defa4","Type":"ContainerStarted","Data":"bb791e652527700a53a97f6321e313ab8df8d3603d0b77fc84c724bdc1d34b2f"}
Sep 11 04:41:57.652913 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.652871    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-67fc898877-796tx" event={"ID":"277ab579-b700-4770-ac93-8fe753be5622","Type":"ContainerStarted","Data":"0d154c150253c841e46eb5b75e572e9622bb59eb90a93852024e6eb3594fffda"}
Sep 11 04:41:57.665724 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.665654    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9" podStartSLOduration=2.35180239 podStartE2EDuration="6.665640503s" totalImagesPullingTime="4.313838113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:41:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:57.663419235 +0000 UTC m=+132.134883146" watchObservedRunningTime="2026-09-11 04:41:57.665640503 +0000 UTC m=+132.137104378"
Sep 11 04:41:57.683153 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.683109    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5d85778b9d-69vlz" podStartSLOduration=2.411904462 podStartE2EDuration="4.683097953s" totalImagesPullingTime="2.271193491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:57.681731384 +0000 UTC m=+132.153195275" watchObservedRunningTime="2026-09-11 04:41:57.683097953 +0000 UTC m=+132.154561829"
Sep 11 04:41:57.691027 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.690999    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:57.704147 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.704097    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-576f54fd8d-4gjnr" podStartSLOduration=2.437527206 podStartE2EDuration="4.704060119s" totalImagesPullingTime="2.266532913s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:57.702594528 +0000 UTC m=+132.174058401" watchObservedRunningTime="2026-09-11 04:41:57.704060119 +0000 UTC m=+132.175523995"
Sep 11 04:41:57.749534 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.749483    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-67fc898877-796tx" podStartSLOduration=1.815221707 podStartE2EDuration="4.749468891s" totalImagesPullingTime="2.934247184s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:41:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:57.747892019 +0000 UTC m=+132.219355896" watchObservedRunningTime="2026-09-11 04:41:57.749468891 +0000 UTC m=+132.220932768"
Sep 11 04:41:57.753298 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.753273    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:57.765664 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.765602    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dfv69" podStartSLOduration=130.572572521 podStartE2EDuration="2m11.765587561s" totalImagesPullingTime="1.19301504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:39:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:57.764365233 +0000 UTC m=+132.235829124" watchObservedRunningTime="2026-09-11 04:41:57.765587561 +0000 UTC m=+132.237051437"
Sep 11 04:41:57.814051 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.814007    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.5883209799999998 podStartE2EDuration="8.813992039s" totalImagesPullingTime="5.208033272s" totalInitContainerRuntime="17.637787ms" isStatefulPod=false podCreationTimestamp="2026-09-11 04:41:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:41:57.811481822 +0000 UTC m=+132.282945713" watchObservedRunningTime="2026-09-11 04:41:57.813992039 +0000 UTC m=+132.285455915"
Sep 11 04:41:57.969595 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:57.969507    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:41:58.067460 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:41:58.067428    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:00.669918 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:00.669887    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003","Type":"ContainerStarted","Data":"0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f"}
Sep 11 04:42:01.682860 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:01.682827    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003","Type":"ContainerStarted","Data":"0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0"}
Sep 11 04:42:01.683274 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:01.682910    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003","Type":"ContainerStarted","Data":"68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597"}
Sep 11 04:42:01.683274 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:01.682936    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003","Type":"ContainerStarted","Data":"4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6"}
Sep 11 04:42:01.683274 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:01.682960    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003","Type":"ContainerStarted","Data":"0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029"}
Sep 11 04:42:01.683274 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:01.682982    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003","Type":"ContainerStarted","Data":"8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc"}
Sep 11 04:42:01.714855 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:01.714764    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=4.820846225 podStartE2EDuration="7.714748315s" totalImagesPullingTime="2.860260063s" totalInitContainerRuntime="33.642027ms" isStatefulPod=false podCreationTimestamp="2026-09-11 04:41:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:42:01.713354445 +0000 UTC m=+136.184818321" watchObservedRunningTime="2026-09-11 04:42:01.714748315 +0000 UTC m=+136.186212191"
Sep 11 04:42:02.774085 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:02.774040    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:02.846117 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:02.846059    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:02.900785 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:02.900741    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:03.064036 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:03.063933    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:03.445754 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:03.445670    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:03.498921 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:03.498886    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:03.663973 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:03.663949    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6788cfb4b9-xvtz9"
Sep 11 04:42:03.847341 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:03.847286    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:04.069127 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:04.069055    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:04.123155 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:04.123047    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:04.281779 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:04.281743    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:04.380227 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:04.380142    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:04.480087 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:04.480034    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:04.653584 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:04.653505    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:04.772515 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:04.772485    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:05.250708 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:05.250676    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:42:05.859566 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:05.859534    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:05.910718 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:05.910683    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:09.107781 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:09.107738    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:09.167987 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:09.167947    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:12.680139 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:12.680055    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:12.734887 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:12.734846    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:13.480789 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:13.480750    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:42:13.480936 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:13.480814    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:42:14.874599 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:14.874559    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:14.946450 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:14.946424    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:15.116218 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:15.116191    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2t5fm_9285e03b-0880-4ae3-b25d-3d4cd81ba474/serve-healthcheck-canary/0.log"
Sep 11 04:42:16.097309 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:16.097275    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:16.153527 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:16.153494    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:17.967719 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:17.967687    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:18.067866 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:18.067826    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:18.666782 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:18.666747    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:18.719195 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:18.719160    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:19.894710 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:19.894660    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:19.995369 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:19.995337    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:20.096213 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.096164    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:20.254261 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.254168    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-79f69c7884-pz9cm" podUID="01a959b4-ae03-4d51-bb69-26b59b7e19a5" containerName="registry" containerID="cri-o://25d11860c491720ebbe547787f2002707786b34aabc0320e77433cc11b568ef0" gracePeriod=30
Sep 11 04:42:20.500676 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.500646    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:42:20.549200 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.549174    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-bound-sa-token\") pod \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") "
Sep 11 04:42:20.549371 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.549213    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/01a959b4-ae03-4d51-bb69-26b59b7e19a5-trusted-ca\") pod \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") "
Sep 11 04:42:20.549371 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.549228    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls\") pod \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") "
Sep 11 04:42:20.549371 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.549245    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/01a959b4-ae03-4d51-bb69-26b59b7e19a5-installation-pull-secrets\") pod \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") "
Sep 11 04:42:20.549533 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.549428    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/01a959b4-ae03-4d51-bb69-26b59b7e19a5-ca-trust-extracted\") pod \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") "
Sep 11 04:42:20.549533 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.549513    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-certificates\") pod \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") "
Sep 11 04:42:20.549639 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.549549    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/01a959b4-ae03-4d51-bb69-26b59b7e19a5-image-registry-private-configuration\") pod \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") "
Sep 11 04:42:20.549639 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.549578    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wr5mz\" (UniqueName: \"kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-kube-api-access-wr5mz\") pod \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\" (UID: \"01a959b4-ae03-4d51-bb69-26b59b7e19a5\") "
Sep 11 04:42:20.549639 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.549625    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01a959b4-ae03-4d51-bb69-26b59b7e19a5-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "01a959b4-ae03-4d51-bb69-26b59b7e19a5" (UID: "01a959b4-ae03-4d51-bb69-26b59b7e19a5"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:42:20.549893 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.549872    2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/01a959b4-ae03-4d51-bb69-26b59b7e19a5-trusted-ca\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:42:20.550353 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.550321    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "01a959b4-ae03-4d51-bb69-26b59b7e19a5" (UID: "01a959b4-ae03-4d51-bb69-26b59b7e19a5"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:42:20.552014 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.551986    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "01a959b4-ae03-4d51-bb69-26b59b7e19a5" (UID: "01a959b4-ae03-4d51-bb69-26b59b7e19a5"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:42:20.552014 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.551992    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "01a959b4-ae03-4d51-bb69-26b59b7e19a5" (UID: "01a959b4-ae03-4d51-bb69-26b59b7e19a5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:42:20.552196 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.552106    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01a959b4-ae03-4d51-bb69-26b59b7e19a5-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "01a959b4-ae03-4d51-bb69-26b59b7e19a5" (UID: "01a959b4-ae03-4d51-bb69-26b59b7e19a5"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:42:20.552452 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.552428    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-kube-api-access-wr5mz" (OuterVolumeSpecName: "kube-api-access-wr5mz") pod "01a959b4-ae03-4d51-bb69-26b59b7e19a5" (UID: "01a959b4-ae03-4d51-bb69-26b59b7e19a5"). InnerVolumeSpecName "kube-api-access-wr5mz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:42:20.552452 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.552433    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01a959b4-ae03-4d51-bb69-26b59b7e19a5-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "01a959b4-ae03-4d51-bb69-26b59b7e19a5" (UID: "01a959b4-ae03-4d51-bb69-26b59b7e19a5"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:42:20.558282 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.558258    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01a959b4-ae03-4d51-bb69-26b59b7e19a5-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "01a959b4-ae03-4d51-bb69-26b59b7e19a5" (UID: "01a959b4-ae03-4d51-bb69-26b59b7e19a5"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:42:20.650607 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.650580    2567 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/01a959b4-ae03-4d51-bb69-26b59b7e19a5-ca-trust-extracted\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:42:20.650607 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.650604    2567 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-certificates\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:42:20.650751 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.650615    2567 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/01a959b4-ae03-4d51-bb69-26b59b7e19a5-image-registry-private-configuration\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:42:20.650751 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.650624    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wr5mz\" (UniqueName: \"kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-kube-api-access-wr5mz\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:42:20.650751 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.650633    2567 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-bound-sa-token\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:42:20.650751 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.650642    2567 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/01a959b4-ae03-4d51-bb69-26b59b7e19a5-registry-tls\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:42:20.650751 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.650650    2567 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/01a959b4-ae03-4d51-bb69-26b59b7e19a5-installation-pull-secrets\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:42:20.762906 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.762879    2567 generic.go:356] "Generic (PLEG): container finished" podID="01a959b4-ae03-4d51-bb69-26b59b7e19a5" containerID="25d11860c491720ebbe547787f2002707786b34aabc0320e77433cc11b568ef0" exitCode=0
Sep 11 04:42:20.763053 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.762926    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79f69c7884-pz9cm"
Sep 11 04:42:20.763053 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.762936    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79f69c7884-pz9cm" event={"ID":"01a959b4-ae03-4d51-bb69-26b59b7e19a5","Type":"ContainerDied","Data":"25d11860c491720ebbe547787f2002707786b34aabc0320e77433cc11b568ef0"}
Sep 11 04:42:20.763053 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.762977    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79f69c7884-pz9cm" event={"ID":"01a959b4-ae03-4d51-bb69-26b59b7e19a5","Type":"ContainerDied","Data":"d92b98d23ad1439f3da7f78264ce124b7d6f22419047f2d5e0c75324eaef5a0a"}
Sep 11 04:42:20.763053 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.762995    2567 scope.go:117] "RemoveContainer" containerID="25d11860c491720ebbe547787f2002707786b34aabc0320e77433cc11b568ef0"
Sep 11 04:42:20.772292 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.772274    2567 scope.go:117] "RemoveContainer" containerID="25d11860c491720ebbe547787f2002707786b34aabc0320e77433cc11b568ef0"
Sep 11 04:42:20.772592 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:42:20.772571    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25d11860c491720ebbe547787f2002707786b34aabc0320e77433cc11b568ef0\": container with ID starting with 25d11860c491720ebbe547787f2002707786b34aabc0320e77433cc11b568ef0 not found: ID does not exist" containerID="25d11860c491720ebbe547787f2002707786b34aabc0320e77433cc11b568ef0"
Sep 11 04:42:20.772678 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.772597    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25d11860c491720ebbe547787f2002707786b34aabc0320e77433cc11b568ef0"} err="failed to get container status \"25d11860c491720ebbe547787f2002707786b34aabc0320e77433cc11b568ef0\": rpc error: code = NotFound desc = could not find container \"25d11860c491720ebbe547787f2002707786b34aabc0320e77433cc11b568ef0\": container with ID starting with 25d11860c491720ebbe547787f2002707786b34aabc0320e77433cc11b568ef0 not found: ID does not exist"
Sep 11 04:42:20.784507 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.784486    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-79f69c7884-pz9cm"]
Sep 11 04:42:20.787411 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:20.787391    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-79f69c7884-pz9cm"]
Sep 11 04:42:22.017032 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:22.016989    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01a959b4-ae03-4d51-bb69-26b59b7e19a5" path="/var/lib/kubelet/pods/01a959b4-ae03-4d51-bb69-26b59b7e19a5/volumes"
Sep 11 04:42:23.275834 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:23.275794    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:23.483956 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:23.483922    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:23.532621 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:23.532532    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:23.588281 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:23.588252    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:24.282610 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:24.282580    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:24.339514 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:24.339482    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:27.686924 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:27.686838    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:27.738031 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:27.738001    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:28.072791 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:28.072752    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:28.195484 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:28.195371    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:28.306758 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:28.306672    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:33.385715 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:33.385682    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:33.486191 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:33.486169    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:42:33.489979 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:33.489958    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-67fc898877-796tx"
Sep 11 04:42:36.568831 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:36.568794    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:36.623937 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:36.623910    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:42:55.250613 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:55.250561    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:42:55.270226 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:55.270194    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:42:55.922545 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:42:55.922519    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:04.830614 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:04.830581    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:43:04.897330 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:04.897296    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:43:09.140153 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:09.140113    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 04:43:09.140642 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:09.140421    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="alertmanager" containerID="cri-o://db1bae133f91984ea4ffaed655a856a34ad27614a3fddf50276ef3f82157d092" gracePeriod=120
Sep 11 04:43:09.140642 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:09.140467    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="kube-rbac-proxy-metric" containerID="cri-o://5ab94724c62a7e59fcfb01d81cce4650b2e7800b00e461092fa2bbc424776b8d" gracePeriod=120
Sep 11 04:43:09.140642 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:09.140570    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="prom-label-proxy" containerID="cri-o://bb791e652527700a53a97f6321e313ab8df8d3603d0b77fc84c724bdc1d34b2f" gracePeriod=120
Sep 11 04:43:09.140642 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:09.140543    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="kube-rbac-proxy-web" containerID="cri-o://bdfd41657a2233ac1a17dba2fcde36df2df83464331a2b8f23279bdf7299dd64" gracePeriod=120
Sep 11 04:43:09.140858 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:09.140568    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="kube-rbac-proxy" containerID="cri-o://bceb9d50bd3c869634514c2711aeff1a1729cb0f88ed4439db708cca38739fc6" gracePeriod=120
Sep 11 04:43:09.140858 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:09.140590    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="config-reloader" containerID="cri-o://8b9ac9e90898dbb3cab6e31b1875b10c288bce40a63d8ca9486bf65d13067e83" gracePeriod=120
Sep 11 04:43:09.966871 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:09.966809    2567 generic.go:356] "Generic (PLEG): container finished" podID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerID="bb791e652527700a53a97f6321e313ab8df8d3603d0b77fc84c724bdc1d34b2f" exitCode=0
Sep 11 04:43:09.966871 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:09.966864    2567 generic.go:356] "Generic (PLEG): container finished" podID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerID="bceb9d50bd3c869634514c2711aeff1a1729cb0f88ed4439db708cca38739fc6" exitCode=0
Sep 11 04:43:09.966871 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:09.966876    2567 generic.go:356] "Generic (PLEG): container finished" podID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerID="8b9ac9e90898dbb3cab6e31b1875b10c288bce40a63d8ca9486bf65d13067e83" exitCode=0
Sep 11 04:43:09.967113 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:09.966886    2567 generic.go:356] "Generic (PLEG): container finished" podID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerID="db1bae133f91984ea4ffaed655a856a34ad27614a3fddf50276ef3f82157d092" exitCode=0
Sep 11 04:43:09.967113 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:09.966885    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc0d0076-7e12-4f8d-8f11-f076e57defa4","Type":"ContainerDied","Data":"bb791e652527700a53a97f6321e313ab8df8d3603d0b77fc84c724bdc1d34b2f"}
Sep 11 04:43:09.967113 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:09.966955    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc0d0076-7e12-4f8d-8f11-f076e57defa4","Type":"ContainerDied","Data":"bceb9d50bd3c869634514c2711aeff1a1729cb0f88ed4439db708cca38739fc6"}
Sep 11 04:43:09.967113 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:09.966985    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc0d0076-7e12-4f8d-8f11-f076e57defa4","Type":"ContainerDied","Data":"8b9ac9e90898dbb3cab6e31b1875b10c288bce40a63d8ca9486bf65d13067e83"}
Sep 11 04:43:09.967113 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:09.967009    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc0d0076-7e12-4f8d-8f11-f076e57defa4","Type":"ContainerDied","Data":"db1bae133f91984ea4ffaed655a856a34ad27614a3fddf50276ef3f82157d092"}
Sep 11 04:43:10.388120 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.388095    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:10.469818 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.469790    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-config-volume\") pod \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") "
Sep 11 04:43:10.469966 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.469836    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-kube-rbac-proxy-web\") pod \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") "
Sep 11 04:43:10.469966 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.469862    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-main-tls\") pod \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") "
Sep 11 04:43:10.469966 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.469900    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc0d0076-7e12-4f8d-8f11-f076e57defa4-metrics-client-ca\") pod \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") "
Sep 11 04:43:10.469966 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.469939    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-web-config\") pod \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") "
Sep 11 04:43:10.469966 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.469966    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mhxmf\" (UniqueName: \"kubernetes.io/projected/cc0d0076-7e12-4f8d-8f11-f076e57defa4-kube-api-access-mhxmf\") pod \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") "
Sep 11 04:43:10.470248 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.469996    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc0d0076-7e12-4f8d-8f11-f076e57defa4-alertmanager-trusted-ca-bundle\") pod \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") "
Sep 11 04:43:10.470248 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.470024    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") "
Sep 11 04:43:10.470248 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.470091    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cc0d0076-7e12-4f8d-8f11-f076e57defa4-tls-assets\") pod \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") "
Sep 11 04:43:10.470248 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.470125    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cc0d0076-7e12-4f8d-8f11-f076e57defa4-alertmanager-main-db\") pod \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") "
Sep 11 04:43:10.470248 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.470150    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-cluster-tls-config\") pod \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") "
Sep 11 04:43:10.470248 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.470192    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cc0d0076-7e12-4f8d-8f11-f076e57defa4-config-out\") pod \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") "
Sep 11 04:43:10.470248 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.470230    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-kube-rbac-proxy\") pod \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\" (UID: \"cc0d0076-7e12-4f8d-8f11-f076e57defa4\") "
Sep 11 04:43:10.470764 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.470361    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc0d0076-7e12-4f8d-8f11-f076e57defa4-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "cc0d0076-7e12-4f8d-8f11-f076e57defa4" (UID: "cc0d0076-7e12-4f8d-8f11-f076e57defa4"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:43:10.470764 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.470593    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc0d0076-7e12-4f8d-8f11-f076e57defa4-metrics-client-ca\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:10.471198 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.471150    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc0d0076-7e12-4f8d-8f11-f076e57defa4-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "cc0d0076-7e12-4f8d-8f11-f076e57defa4" (UID: "cc0d0076-7e12-4f8d-8f11-f076e57defa4"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:43:10.471330 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.471195    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc0d0076-7e12-4f8d-8f11-f076e57defa4-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "cc0d0076-7e12-4f8d-8f11-f076e57defa4" (UID: "cc0d0076-7e12-4f8d-8f11-f076e57defa4"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:43:10.473400 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.473359    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "cc0d0076-7e12-4f8d-8f11-f076e57defa4" (UID: "cc0d0076-7e12-4f8d-8f11-f076e57defa4"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:43:10.473737 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.473698    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-config-volume" (OuterVolumeSpecName: "config-volume") pod "cc0d0076-7e12-4f8d-8f11-f076e57defa4" (UID: "cc0d0076-7e12-4f8d-8f11-f076e57defa4"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:43:10.474034 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.474002    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "cc0d0076-7e12-4f8d-8f11-f076e57defa4" (UID: "cc0d0076-7e12-4f8d-8f11-f076e57defa4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:43:10.474176 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.474087    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc0d0076-7e12-4f8d-8f11-f076e57defa4-kube-api-access-mhxmf" (OuterVolumeSpecName: "kube-api-access-mhxmf") pod "cc0d0076-7e12-4f8d-8f11-f076e57defa4" (UID: "cc0d0076-7e12-4f8d-8f11-f076e57defa4"). InnerVolumeSpecName "kube-api-access-mhxmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:43:10.474176 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.474136    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc0d0076-7e12-4f8d-8f11-f076e57defa4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "cc0d0076-7e12-4f8d-8f11-f076e57defa4" (UID: "cc0d0076-7e12-4f8d-8f11-f076e57defa4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:43:10.474665 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.474642    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "cc0d0076-7e12-4f8d-8f11-f076e57defa4" (UID: "cc0d0076-7e12-4f8d-8f11-f076e57defa4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:43:10.474866 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.474841    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "cc0d0076-7e12-4f8d-8f11-f076e57defa4" (UID: "cc0d0076-7e12-4f8d-8f11-f076e57defa4"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:43:10.474944 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.474928    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc0d0076-7e12-4f8d-8f11-f076e57defa4-config-out" (OuterVolumeSpecName: "config-out") pod "cc0d0076-7e12-4f8d-8f11-f076e57defa4" (UID: "cc0d0076-7e12-4f8d-8f11-f076e57defa4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:43:10.477980 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.477959    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "cc0d0076-7e12-4f8d-8f11-f076e57defa4" (UID: "cc0d0076-7e12-4f8d-8f11-f076e57defa4"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:43:10.485335 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.485315    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-web-config" (OuterVolumeSpecName: "web-config") pod "cc0d0076-7e12-4f8d-8f11-f076e57defa4" (UID: "cc0d0076-7e12-4f8d-8f11-f076e57defa4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:43:10.571730 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.571700    2567 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cc0d0076-7e12-4f8d-8f11-f076e57defa4-tls-assets\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:10.571730 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.571724    2567 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/cc0d0076-7e12-4f8d-8f11-f076e57defa4-alertmanager-main-db\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:10.571730 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.571735    2567 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-cluster-tls-config\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:10.571910 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.571744    2567 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cc0d0076-7e12-4f8d-8f11-f076e57defa4-config-out\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:10.571910 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.571753    2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:10.571910 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.571762    2567 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:10.571910 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.571770    2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:10.571910 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.571778    2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-main-tls\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:10.571910 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.571787    2567 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-web-config\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:10.571910 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.571795    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhxmf\" (UniqueName: \"kubernetes.io/projected/cc0d0076-7e12-4f8d-8f11-f076e57defa4-kube-api-access-mhxmf\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:10.571910 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.571805    2567 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cc0d0076-7e12-4f8d-8f11-f076e57defa4-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:10.571910 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.571814    2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/cc0d0076-7e12-4f8d-8f11-f076e57defa4-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:10.973645 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.973558    2567 generic.go:356] "Generic (PLEG): container finished" podID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerID="5ab94724c62a7e59fcfb01d81cce4650b2e7800b00e461092fa2bbc424776b8d" exitCode=0
Sep 11 04:43:10.973645 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.973584    2567 generic.go:356] "Generic (PLEG): container finished" podID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerID="bdfd41657a2233ac1a17dba2fcde36df2df83464331a2b8f23279bdf7299dd64" exitCode=0
Sep 11 04:43:10.973645 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.973626    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:10.973844 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.973659    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc0d0076-7e12-4f8d-8f11-f076e57defa4","Type":"ContainerDied","Data":"5ab94724c62a7e59fcfb01d81cce4650b2e7800b00e461092fa2bbc424776b8d"}
Sep 11 04:43:10.973844 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.973716    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc0d0076-7e12-4f8d-8f11-f076e57defa4","Type":"ContainerDied","Data":"bdfd41657a2233ac1a17dba2fcde36df2df83464331a2b8f23279bdf7299dd64"}
Sep 11 04:43:10.973844 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.973743    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"cc0d0076-7e12-4f8d-8f11-f076e57defa4","Type":"ContainerDied","Data":"50cb7c3ee289a8dd1e545a333a8c70f7e2f8bbc62fc01222ce20e3b59d978520"}
Sep 11 04:43:10.973844 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.973772    2567 scope.go:117] "RemoveContainer" containerID="bb791e652527700a53a97f6321e313ab8df8d3603d0b77fc84c724bdc1d34b2f"
Sep 11 04:43:10.982062 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.982045    2567 scope.go:117] "RemoveContainer" containerID="5ab94724c62a7e59fcfb01d81cce4650b2e7800b00e461092fa2bbc424776b8d"
Sep 11 04:43:10.988940 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.988923    2567 scope.go:117] "RemoveContainer" containerID="bceb9d50bd3c869634514c2711aeff1a1729cb0f88ed4439db708cca38739fc6"
Sep 11 04:43:10.995864 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.995845    2567 scope.go:117] "RemoveContainer" containerID="bdfd41657a2233ac1a17dba2fcde36df2df83464331a2b8f23279bdf7299dd64"
Sep 11 04:43:10.998619 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:10.998596    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 04:43:11.002510 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.002484    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 04:43:11.003994 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.003978    2567 scope.go:117] "RemoveContainer" containerID="8b9ac9e90898dbb3cab6e31b1875b10c288bce40a63d8ca9486bf65d13067e83"
Sep 11 04:43:11.011397 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.011381    2567 scope.go:117] "RemoveContainer" containerID="db1bae133f91984ea4ffaed655a856a34ad27614a3fddf50276ef3f82157d092"
Sep 11 04:43:11.018359 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.018343    2567 scope.go:117] "RemoveContainer" containerID="f317d56c3d916518a04b962cb2df147f9461ed74f282520a00ee9a50d56c64da"
Sep 11 04:43:11.028592 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.028573    2567 scope.go:117] "RemoveContainer" containerID="bb791e652527700a53a97f6321e313ab8df8d3603d0b77fc84c724bdc1d34b2f"
Sep 11 04:43:11.029094 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:43:11.029036    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb791e652527700a53a97f6321e313ab8df8d3603d0b77fc84c724bdc1d34b2f\": container with ID starting with bb791e652527700a53a97f6321e313ab8df8d3603d0b77fc84c724bdc1d34b2f not found: ID does not exist" containerID="bb791e652527700a53a97f6321e313ab8df8d3603d0b77fc84c724bdc1d34b2f"
Sep 11 04:43:11.029232 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.029099    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb791e652527700a53a97f6321e313ab8df8d3603d0b77fc84c724bdc1d34b2f"} err="failed to get container status \"bb791e652527700a53a97f6321e313ab8df8d3603d0b77fc84c724bdc1d34b2f\": rpc error: code = NotFound desc = could not find container \"bb791e652527700a53a97f6321e313ab8df8d3603d0b77fc84c724bdc1d34b2f\": container with ID starting with bb791e652527700a53a97f6321e313ab8df8d3603d0b77fc84c724bdc1d34b2f not found: ID does not exist"
Sep 11 04:43:11.029523 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.029503    2567 scope.go:117] "RemoveContainer" containerID="5ab94724c62a7e59fcfb01d81cce4650b2e7800b00e461092fa2bbc424776b8d"
Sep 11 04:43:11.029978 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:43:11.029902    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ab94724c62a7e59fcfb01d81cce4650b2e7800b00e461092fa2bbc424776b8d\": container with ID starting with 5ab94724c62a7e59fcfb01d81cce4650b2e7800b00e461092fa2bbc424776b8d not found: ID does not exist" containerID="5ab94724c62a7e59fcfb01d81cce4650b2e7800b00e461092fa2bbc424776b8d"
Sep 11 04:43:11.029978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.029934    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ab94724c62a7e59fcfb01d81cce4650b2e7800b00e461092fa2bbc424776b8d"} err="failed to get container status \"5ab94724c62a7e59fcfb01d81cce4650b2e7800b00e461092fa2bbc424776b8d\": rpc error: code = NotFound desc = could not find container \"5ab94724c62a7e59fcfb01d81cce4650b2e7800b00e461092fa2bbc424776b8d\": container with ID starting with 5ab94724c62a7e59fcfb01d81cce4650b2e7800b00e461092fa2bbc424776b8d not found: ID does not exist"
Sep 11 04:43:11.029978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.029959    2567 scope.go:117] "RemoveContainer" containerID="bceb9d50bd3c869634514c2711aeff1a1729cb0f88ed4439db708cca38739fc6"
Sep 11 04:43:11.030332 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:43:11.030292    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bceb9d50bd3c869634514c2711aeff1a1729cb0f88ed4439db708cca38739fc6\": container with ID starting with bceb9d50bd3c869634514c2711aeff1a1729cb0f88ed4439db708cca38739fc6 not found: ID does not exist" containerID="bceb9d50bd3c869634514c2711aeff1a1729cb0f88ed4439db708cca38739fc6"
Sep 11 04:43:11.030403 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.030343    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bceb9d50bd3c869634514c2711aeff1a1729cb0f88ed4439db708cca38739fc6"} err="failed to get container status \"bceb9d50bd3c869634514c2711aeff1a1729cb0f88ed4439db708cca38739fc6\": rpc error: code = NotFound desc = could not find container \"bceb9d50bd3c869634514c2711aeff1a1729cb0f88ed4439db708cca38739fc6\": container with ID starting with bceb9d50bd3c869634514c2711aeff1a1729cb0f88ed4439db708cca38739fc6 not found: ID does not exist"
Sep 11 04:43:11.030403 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.030366    2567 scope.go:117] "RemoveContainer" containerID="bdfd41657a2233ac1a17dba2fcde36df2df83464331a2b8f23279bdf7299dd64"
Sep 11 04:43:11.030678 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:43:11.030659    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdfd41657a2233ac1a17dba2fcde36df2df83464331a2b8f23279bdf7299dd64\": container with ID starting with bdfd41657a2233ac1a17dba2fcde36df2df83464331a2b8f23279bdf7299dd64 not found: ID does not exist" containerID="bdfd41657a2233ac1a17dba2fcde36df2df83464331a2b8f23279bdf7299dd64"
Sep 11 04:43:11.030731 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.030684    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdfd41657a2233ac1a17dba2fcde36df2df83464331a2b8f23279bdf7299dd64"} err="failed to get container status \"bdfd41657a2233ac1a17dba2fcde36df2df83464331a2b8f23279bdf7299dd64\": rpc error: code = NotFound desc = could not find container \"bdfd41657a2233ac1a17dba2fcde36df2df83464331a2b8f23279bdf7299dd64\": container with ID starting with bdfd41657a2233ac1a17dba2fcde36df2df83464331a2b8f23279bdf7299dd64 not found: ID does not exist"
Sep 11 04:43:11.030731 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.030699    2567 scope.go:117] "RemoveContainer" containerID="8b9ac9e90898dbb3cab6e31b1875b10c288bce40a63d8ca9486bf65d13067e83"
Sep 11 04:43:11.030945 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:43:11.030927    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b9ac9e90898dbb3cab6e31b1875b10c288bce40a63d8ca9486bf65d13067e83\": container with ID starting with 8b9ac9e90898dbb3cab6e31b1875b10c288bce40a63d8ca9486bf65d13067e83 not found: ID does not exist" containerID="8b9ac9e90898dbb3cab6e31b1875b10c288bce40a63d8ca9486bf65d13067e83"
Sep 11 04:43:11.031004 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.030948    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9ac9e90898dbb3cab6e31b1875b10c288bce40a63d8ca9486bf65d13067e83"} err="failed to get container status \"8b9ac9e90898dbb3cab6e31b1875b10c288bce40a63d8ca9486bf65d13067e83\": rpc error: code = NotFound desc = could not find container \"8b9ac9e90898dbb3cab6e31b1875b10c288bce40a63d8ca9486bf65d13067e83\": container with ID starting with 8b9ac9e90898dbb3cab6e31b1875b10c288bce40a63d8ca9486bf65d13067e83 not found: ID does not exist"
Sep 11 04:43:11.031004 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.030961    2567 scope.go:117] "RemoveContainer" containerID="db1bae133f91984ea4ffaed655a856a34ad27614a3fddf50276ef3f82157d092"
Sep 11 04:43:11.031257 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:43:11.031239    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db1bae133f91984ea4ffaed655a856a34ad27614a3fddf50276ef3f82157d092\": container with ID starting with db1bae133f91984ea4ffaed655a856a34ad27614a3fddf50276ef3f82157d092 not found: ID does not exist" containerID="db1bae133f91984ea4ffaed655a856a34ad27614a3fddf50276ef3f82157d092"
Sep 11 04:43:11.031314 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.031260    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db1bae133f91984ea4ffaed655a856a34ad27614a3fddf50276ef3f82157d092"} err="failed to get container status \"db1bae133f91984ea4ffaed655a856a34ad27614a3fddf50276ef3f82157d092\": rpc error: code = NotFound desc = could not find container \"db1bae133f91984ea4ffaed655a856a34ad27614a3fddf50276ef3f82157d092\": container with ID starting with db1bae133f91984ea4ffaed655a856a34ad27614a3fddf50276ef3f82157d092 not found: ID does not exist"
Sep 11 04:43:11.031314 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.031274    2567 scope.go:117] "RemoveContainer" containerID="f317d56c3d916518a04b962cb2df147f9461ed74f282520a00ee9a50d56c64da"
Sep 11 04:43:11.031476 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:43:11.031460    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f317d56c3d916518a04b962cb2df147f9461ed74f282520a00ee9a50d56c64da\": container with ID starting with f317d56c3d916518a04b962cb2df147f9461ed74f282520a00ee9a50d56c64da not found: ID does not exist" containerID="f317d56c3d916518a04b962cb2df147f9461ed74f282520a00ee9a50d56c64da"
Sep 11 04:43:11.031527 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.031479    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f317d56c3d916518a04b962cb2df147f9461ed74f282520a00ee9a50d56c64da"} err="failed to get container status \"f317d56c3d916518a04b962cb2df147f9461ed74f282520a00ee9a50d56c64da\": rpc error: code = NotFound desc = could not find container \"f317d56c3d916518a04b962cb2df147f9461ed74f282520a00ee9a50d56c64da\": container with ID starting with f317d56c3d916518a04b962cb2df147f9461ed74f282520a00ee9a50d56c64da not found: ID does not exist"
Sep 11 04:43:11.031527 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.031491    2567 scope.go:117] "RemoveContainer" containerID="bb791e652527700a53a97f6321e313ab8df8d3603d0b77fc84c724bdc1d34b2f"
Sep 11 04:43:11.031625 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.031520    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 04:43:11.031709 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.031692    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb791e652527700a53a97f6321e313ab8df8d3603d0b77fc84c724bdc1d34b2f"} err="failed to get container status \"bb791e652527700a53a97f6321e313ab8df8d3603d0b77fc84c724bdc1d34b2f\": rpc error: code = NotFound desc = could not find container \"bb791e652527700a53a97f6321e313ab8df8d3603d0b77fc84c724bdc1d34b2f\": container with ID starting with bb791e652527700a53a97f6321e313ab8df8d3603d0b77fc84c724bdc1d34b2f not found: ID does not exist"
Sep 11 04:43:11.031817 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.031709    2567 scope.go:117] "RemoveContainer" containerID="5ab94724c62a7e59fcfb01d81cce4650b2e7800b00e461092fa2bbc424776b8d"
Sep 11 04:43:11.031937 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.031917    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ab94724c62a7e59fcfb01d81cce4650b2e7800b00e461092fa2bbc424776b8d"} err="failed to get container status \"5ab94724c62a7e59fcfb01d81cce4650b2e7800b00e461092fa2bbc424776b8d\": rpc error: code = NotFound desc = could not find container \"5ab94724c62a7e59fcfb01d81cce4650b2e7800b00e461092fa2bbc424776b8d\": container with ID starting with 5ab94724c62a7e59fcfb01d81cce4650b2e7800b00e461092fa2bbc424776b8d not found: ID does not exist"
Sep 11 04:43:11.031985 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.031941    2567 scope.go:117] "RemoveContainer" containerID="bceb9d50bd3c869634514c2711aeff1a1729cb0f88ed4439db708cca38739fc6"
Sep 11 04:43:11.031985 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.031926    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="init-config-reloader"
Sep 11 04:43:11.032095 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.031989    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="init-config-reloader"
Sep 11 04:43:11.032095 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032003    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="alertmanager"
Sep 11 04:43:11.032095 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032008    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="alertmanager"
Sep 11 04:43:11.032095 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032015    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="prom-label-proxy"
Sep 11 04:43:11.032095 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032020    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="prom-label-proxy"
Sep 11 04:43:11.032095 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032027    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="kube-rbac-proxy-web"
Sep 11 04:43:11.032095 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032033    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="kube-rbac-proxy-web"
Sep 11 04:43:11.032095 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032040    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="config-reloader"
Sep 11 04:43:11.032095 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032049    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="config-reloader"
Sep 11 04:43:11.032095 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032096    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="kube-rbac-proxy"
Sep 11 04:43:11.032398 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032103    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="kube-rbac-proxy"
Sep 11 04:43:11.032398 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032114    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="kube-rbac-proxy-metric"
Sep 11 04:43:11.032398 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032119    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="kube-rbac-proxy-metric"
Sep 11 04:43:11.032398 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032127    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01a959b4-ae03-4d51-bb69-26b59b7e19a5" containerName="registry"
Sep 11 04:43:11.032398 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032135    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="01a959b4-ae03-4d51-bb69-26b59b7e19a5" containerName="registry"
Sep 11 04:43:11.032398 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032198    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bceb9d50bd3c869634514c2711aeff1a1729cb0f88ed4439db708cca38739fc6"} err="failed to get container status \"bceb9d50bd3c869634514c2711aeff1a1729cb0f88ed4439db708cca38739fc6\": rpc error: code = NotFound desc = could not find container \"bceb9d50bd3c869634514c2711aeff1a1729cb0f88ed4439db708cca38739fc6\": container with ID starting with bceb9d50bd3c869634514c2711aeff1a1729cb0f88ed4439db708cca38739fc6 not found: ID does not exist"
Sep 11 04:43:11.032398 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032211    2567 scope.go:117] "RemoveContainer" containerID="bdfd41657a2233ac1a17dba2fcde36df2df83464331a2b8f23279bdf7299dd64"
Sep 11 04:43:11.032398 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032216    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="kube-rbac-proxy-metric"
Sep 11 04:43:11.032398 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032227    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="01a959b4-ae03-4d51-bb69-26b59b7e19a5" containerName="registry"
Sep 11 04:43:11.032398 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032233    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="kube-rbac-proxy"
Sep 11 04:43:11.032398 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032240    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="kube-rbac-proxy-web"
Sep 11 04:43:11.032398 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032246    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="config-reloader"
Sep 11 04:43:11.032398 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032253    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="alertmanager"
Sep 11 04:43:11.032398 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032258    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" containerName="prom-label-proxy"
Sep 11 04:43:11.032398 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032388    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdfd41657a2233ac1a17dba2fcde36df2df83464331a2b8f23279bdf7299dd64"} err="failed to get container status \"bdfd41657a2233ac1a17dba2fcde36df2df83464331a2b8f23279bdf7299dd64\": rpc error: code = NotFound desc = could not find container \"bdfd41657a2233ac1a17dba2fcde36df2df83464331a2b8f23279bdf7299dd64\": container with ID starting with bdfd41657a2233ac1a17dba2fcde36df2df83464331a2b8f23279bdf7299dd64 not found: ID does not exist"
Sep 11 04:43:11.032398 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032400    2567 scope.go:117] "RemoveContainer" containerID="8b9ac9e90898dbb3cab6e31b1875b10c288bce40a63d8ca9486bf65d13067e83"
Sep 11 04:43:11.032953 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032675    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9ac9e90898dbb3cab6e31b1875b10c288bce40a63d8ca9486bf65d13067e83"} err="failed to get container status \"8b9ac9e90898dbb3cab6e31b1875b10c288bce40a63d8ca9486bf65d13067e83\": rpc error: code = NotFound desc = could not find container \"8b9ac9e90898dbb3cab6e31b1875b10c288bce40a63d8ca9486bf65d13067e83\": container with ID starting with 8b9ac9e90898dbb3cab6e31b1875b10c288bce40a63d8ca9486bf65d13067e83 not found: ID does not exist"
Sep 11 04:43:11.032953 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032699    2567 scope.go:117] "RemoveContainer" containerID="db1bae133f91984ea4ffaed655a856a34ad27614a3fddf50276ef3f82157d092"
Sep 11 04:43:11.032953 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032908    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db1bae133f91984ea4ffaed655a856a34ad27614a3fddf50276ef3f82157d092"} err="failed to get container status \"db1bae133f91984ea4ffaed655a856a34ad27614a3fddf50276ef3f82157d092\": rpc error: code = NotFound desc = could not find container \"db1bae133f91984ea4ffaed655a856a34ad27614a3fddf50276ef3f82157d092\": container with ID starting with db1bae133f91984ea4ffaed655a856a34ad27614a3fddf50276ef3f82157d092 not found: ID does not exist"
Sep 11 04:43:11.032953 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.032920    2567 scope.go:117] "RemoveContainer" containerID="f317d56c3d916518a04b962cb2df147f9461ed74f282520a00ee9a50d56c64da"
Sep 11 04:43:11.033151 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.033128    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f317d56c3d916518a04b962cb2df147f9461ed74f282520a00ee9a50d56c64da"} err="failed to get container status \"f317d56c3d916518a04b962cb2df147f9461ed74f282520a00ee9a50d56c64da\": rpc error: code = NotFound desc = could not find container \"f317d56c3d916518a04b962cb2df147f9461ed74f282520a00ee9a50d56c64da\": container with ID starting with f317d56c3d916518a04b962cb2df147f9461ed74f282520a00ee9a50d56c64da not found: ID does not exist"
Sep 11 04:43:11.037285 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.037270    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.039727 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.039707    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 11 04:43:11.039800 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.039707    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7ctts\""
Sep 11 04:43:11.039800 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.039745    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 11 04:43:11.040058 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.040037    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 11 04:43:11.040188 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.040090    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 11 04:43:11.040188 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.040037    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 11 04:43:11.040188 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.040061    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 11 04:43:11.040424 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.040405    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 11 04:43:11.040508 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.040416    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 11 04:43:11.046729 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.046713    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 11 04:43:11.049786 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.049764    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 04:43:11.176271 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.176238    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/45fd11dc-75ca-4420-a997-263d7f5f75d3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.176420 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.176277    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/45fd11dc-75ca-4420-a997-263d7f5f75d3-config-volume\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.176420 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.176311    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/45fd11dc-75ca-4420-a997-263d7f5f75d3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.176420 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.176333    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/45fd11dc-75ca-4420-a997-263d7f5f75d3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.176420 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.176350    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/45fd11dc-75ca-4420-a997-263d7f5f75d3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.176420 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.176407    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/45fd11dc-75ca-4420-a997-263d7f5f75d3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.176611 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.176430    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kv2x\" (UniqueName: \"kubernetes.io/projected/45fd11dc-75ca-4420-a997-263d7f5f75d3-kube-api-access-8kv2x\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.176611 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.176471    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/45fd11dc-75ca-4420-a997-263d7f5f75d3-config-out\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.176611 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.176502    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45fd11dc-75ca-4420-a997-263d7f5f75d3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.176611 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.176522    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/45fd11dc-75ca-4420-a997-263d7f5f75d3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.176611 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.176569    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/45fd11dc-75ca-4420-a997-263d7f5f75d3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.176611 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.176591    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/45fd11dc-75ca-4420-a997-263d7f5f75d3-web-config\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.176805 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.176612    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45fd11dc-75ca-4420-a997-263d7f5f75d3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.276982 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.276958    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45fd11dc-75ca-4420-a997-263d7f5f75d3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.277144 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.276994    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/45fd11dc-75ca-4420-a997-263d7f5f75d3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.277144 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.277019    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/45fd11dc-75ca-4420-a997-263d7f5f75d3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.277144 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.277035    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/45fd11dc-75ca-4420-a997-263d7f5f75d3-web-config\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.277144 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.277051    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45fd11dc-75ca-4420-a997-263d7f5f75d3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.277144 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.277086    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/45fd11dc-75ca-4420-a997-263d7f5f75d3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.277144 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.277112    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/45fd11dc-75ca-4420-a997-263d7f5f75d3-config-volume\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.277460 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.277263    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/45fd11dc-75ca-4420-a997-263d7f5f75d3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.277460 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.277296    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/45fd11dc-75ca-4420-a997-263d7f5f75d3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.277460 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.277319    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/45fd11dc-75ca-4420-a997-263d7f5f75d3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.277460 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.277380    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/45fd11dc-75ca-4420-a997-263d7f5f75d3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.277460 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.277407    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kv2x\" (UniqueName: \"kubernetes.io/projected/45fd11dc-75ca-4420-a997-263d7f5f75d3-kube-api-access-8kv2x\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.277460 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.277450    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/45fd11dc-75ca-4420-a997-263d7f5f75d3-config-out\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.278520 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.278455    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45fd11dc-75ca-4420-a997-263d7f5f75d3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.278709 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.278685    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/45fd11dc-75ca-4420-a997-263d7f5f75d3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.279916 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.279122    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/45fd11dc-75ca-4420-a997-263d7f5f75d3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.280296 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.280256    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/45fd11dc-75ca-4420-a997-263d7f5f75d3-config-out\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.280398 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.280326    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/45fd11dc-75ca-4420-a997-263d7f5f75d3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.280604 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.280546    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/45fd11dc-75ca-4420-a997-263d7f5f75d3-config-volume\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.280680 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.280600    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/45fd11dc-75ca-4420-a997-263d7f5f75d3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.280831 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.280812    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/45fd11dc-75ca-4420-a997-263d7f5f75d3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.281253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.281231    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/45fd11dc-75ca-4420-a997-263d7f5f75d3-web-config\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.281954 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.281936    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/45fd11dc-75ca-4420-a997-263d7f5f75d3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.282421 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.282394    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/45fd11dc-75ca-4420-a997-263d7f5f75d3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.282525 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.282504    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/45fd11dc-75ca-4420-a997-263d7f5f75d3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.287393 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.287375    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kv2x\" (UniqueName: \"kubernetes.io/projected/45fd11dc-75ca-4420-a997-263d7f5f75d3-kube-api-access-8kv2x\") pod \"alertmanager-main-0\" (UID: \"45fd11dc-75ca-4420-a997-263d7f5f75d3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.347026 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.347002    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 11 04:43:11.626675 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.626651    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 11 04:43:11.628469 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:43:11.628444    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45fd11dc_75ca_4420_a997_263d7f5f75d3.slice/crio-660f3f72943279d0c00d69c0844b0b94568cd80e09811b6e5108c3c0976bce2b WatchSource:0}: Error finding container 660f3f72943279d0c00d69c0844b0b94568cd80e09811b6e5108c3c0976bce2b: Status 404 returned error can't find the container with id 660f3f72943279d0c00d69c0844b0b94568cd80e09811b6e5108c3c0976bce2b
Sep 11 04:43:11.979553 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.979461    2567 generic.go:356] "Generic (PLEG): container finished" podID="45fd11dc-75ca-4420-a997-263d7f5f75d3" containerID="eb13cb669ad22a08bacc057e1288f7b37ae135f6702ab8144ea9d455ad838d25" exitCode=0
Sep 11 04:43:11.979719 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.979557    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45fd11dc-75ca-4420-a997-263d7f5f75d3","Type":"ContainerDied","Data":"eb13cb669ad22a08bacc057e1288f7b37ae135f6702ab8144ea9d455ad838d25"}
Sep 11 04:43:11.979719 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:11.979641    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45fd11dc-75ca-4420-a997-263d7f5f75d3","Type":"ContainerStarted","Data":"660f3f72943279d0c00d69c0844b0b94568cd80e09811b6e5108c3c0976bce2b"}
Sep 11 04:43:12.017021 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:12.016990    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc0d0076-7e12-4f8d-8f11-f076e57defa4" path="/var/lib/kubelet/pods/cc0d0076-7e12-4f8d-8f11-f076e57defa4/volumes"
Sep 11 04:43:12.987948 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:12.987914    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45fd11dc-75ca-4420-a997-263d7f5f75d3","Type":"ContainerStarted","Data":"d6fb464d25b2cc0e81a66bf42b7de5ae6d70305d569481523c06a5adb555b587"}
Sep 11 04:43:12.988407 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:12.987974    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45fd11dc-75ca-4420-a997-263d7f5f75d3","Type":"ContainerStarted","Data":"f8c576d0670e8eccfde110328fd8d6688e36d529da96e90f52a8053aaf257a05"}
Sep 11 04:43:12.988407 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:12.987999    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45fd11dc-75ca-4420-a997-263d7f5f75d3","Type":"ContainerStarted","Data":"f1c01bd2fc47a39573b5f2abb90bf3b79cdbc0008a5589eafd6c2cdbb2a1f2ea"}
Sep 11 04:43:12.988407 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:12.988025    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45fd11dc-75ca-4420-a997-263d7f5f75d3","Type":"ContainerStarted","Data":"f7b720c20f0b8cbc2cdb8b6e66fe844519f1fae0267626517412cf6e87371db5"}
Sep 11 04:43:12.988407 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:12.988049    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45fd11dc-75ca-4420-a997-263d7f5f75d3","Type":"ContainerStarted","Data":"01a20e6b6894bada058fb8b77b9c4490f166bae8eb0fbc86e4d748a046bb8070"}
Sep 11 04:43:12.988407 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:12.988103    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"45fd11dc-75ca-4420-a997-263d7f5f75d3","Type":"ContainerStarted","Data":"56cd8416f57cb9bf136fcf2ec5bf43d4e9d385a7076329e99bf3ad1e66dcc0f1"}
Sep 11 04:43:13.016253 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.016203    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.001519477 podStartE2EDuration="2.016186284s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.666807ms" isStatefulPod=false podCreationTimestamp="2026-09-11 04:43:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:43:13.01577127 +0000 UTC m=+207.487235166" watchObservedRunningTime="2026-09-11 04:43:13.016186284 +0000 UTC m=+207.487650160"
Sep 11 04:43:13.357923 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.357891    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 04:43:13.358203 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.358172    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="prometheus" containerID="cri-o://0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f" gracePeriod=600
Sep 11 04:43:13.358376 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.358201    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="config-reloader" containerID="cri-o://8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc" gracePeriod=600
Sep 11 04:43:13.358376 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.358243    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="kube-rbac-proxy" containerID="cri-o://68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597" gracePeriod=600
Sep 11 04:43:13.358376 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.358196    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="kube-rbac-proxy-web" containerID="cri-o://4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6" gracePeriod=600
Sep 11 04:43:13.358534 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.358240    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="kube-rbac-proxy-thanos" containerID="cri-o://0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0" gracePeriod=600
Sep 11 04:43:13.358534 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.358197    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="thanos-sidecar" containerID="cri-o://0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029" gracePeriod=600
Sep 11 04:43:13.593994 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.593969    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:13.698567 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.698496    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ckp8\" (UniqueName: \"kubernetes.io/projected/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-kube-api-access-4ckp8\") pod \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") "
Sep 11 04:43:13.698567 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.698550    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-web-config\") pod \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") "
Sep 11 04:43:13.698735 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.698572    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-configmap-kubelet-serving-ca-bundle\") pod \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") "
Sep 11 04:43:13.698735 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.698671    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-config\") pod \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") "
Sep 11 04:43:13.698735 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.698722    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-thanos-prometheus-http-client-file\") pod \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") "
Sep 11 04:43:13.698858 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.698760    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-rulefiles-0\") pod \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") "
Sep 11 04:43:13.698858 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.698791    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-grpc-tls\") pod \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") "
Sep 11 04:43:13.698858 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.698830    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-config-out\") pod \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") "
Sep 11 04:43:13.699007 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.698906    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" (UID: "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:43:13.699007 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.698948    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-tls-assets\") pod \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") "
Sep 11 04:43:13.699007 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.698978    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-prometheus-k8s-tls\") pod \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") "
Sep 11 04:43:13.699182 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.699009    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-db\") pod \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") "
Sep 11 04:43:13.699182 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.699151    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") "
Sep 11 04:43:13.699284 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.699188    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-trusted-ca-bundle\") pod \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") "
Sep 11 04:43:13.699284 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.699221    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-configmap-serving-certs-ca-bundle\") pod \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") "
Sep 11 04:43:13.699284 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.699246    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-rulefiles-1\") pod \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") "
Sep 11 04:43:13.699441 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.699287    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") "
Sep 11 04:43:13.699441 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.699318    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-rulefiles-2\") pod \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") "
Sep 11 04:43:13.699441 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.699366    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-configmap-metrics-client-ca\") pod \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") "
Sep 11 04:43:13.699441 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.699393    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-kube-rbac-proxy\") pod \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") "
Sep 11 04:43:13.699631 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.699562    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-metrics-client-certs\") pod \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\" (UID: \"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003\") "
Sep 11 04:43:13.699924 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.699882    2567 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:13.700913 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.700409    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" (UID: "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:43:13.701740 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.701125    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" (UID: "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:43:13.701740 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.701326    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" (UID: "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:43:13.701740 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.701682    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" (UID: "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:43:13.701740 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.701720    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" (UID: "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:43:13.701999 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.701789    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" (UID: "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:43:13.703054 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.702277    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-kube-api-access-4ckp8" (OuterVolumeSpecName: "kube-api-access-4ckp8") pod "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" (UID: "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003"). InnerVolumeSpecName "kube-api-access-4ckp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:43:13.703054 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.702474    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" (UID: "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:43:13.703355 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.703325    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" (UID: "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:43:13.703690 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.703664    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" (UID: "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:43:13.703789 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.703689    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-config-out" (OuterVolumeSpecName: "config-out") pod "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" (UID: "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 04:43:13.703789 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.703692    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-config" (OuterVolumeSpecName: "config") pod "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" (UID: "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:43:13.703789 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.703717    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" (UID: "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:43:13.704131 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.704104    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" (UID: "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:43:13.704294 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.704276    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" (UID: "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:43:13.704370 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.704344    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" (UID: "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:43:13.705334 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.705301    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" (UID: "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:43:13.705334 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.705327    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" (UID: "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:43:13.712987 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.712966    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-web-config" (OuterVolumeSpecName: "web-config") pod "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" (UID: "cea99d18-0ff7-41b6-b7e3-e7fc6f35f003"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:43:13.800507 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.800482    2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:13.800507 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.800503    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:13.800507 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.800512    2567 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-configmap-metrics-client-ca\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:13.800661 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.800520    2567 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-kube-rbac-proxy\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:13.800661 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.800528    2567 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-metrics-client-certs\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:13.800661 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.800537    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ckp8\" (UniqueName: \"kubernetes.io/projected/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-kube-api-access-4ckp8\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:13.800661 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.800545    2567 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-web-config\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:13.800661 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.800553    2567 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-config\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:13.800661 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.800560    2567 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:13.800661 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.800569    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:13.800661 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.800578    2567 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-grpc-tls\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:13.800661 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.800585    2567 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-config-out\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:13.800661 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.800594    2567 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-tls-assets\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:13.800661 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.800601    2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:13.800661 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.800608    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-db\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:13.800661 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.800618    2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:13.800661 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.800627    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:13.800661 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.800635    2567 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:13.800661 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.800643    2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:43:13.995125 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.995041    2567 generic.go:356] "Generic (PLEG): container finished" podID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerID="0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0" exitCode=0
Sep 11 04:43:13.995125 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.995079    2567 generic.go:356] "Generic (PLEG): container finished" podID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerID="68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597" exitCode=0
Sep 11 04:43:13.995125 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.995090    2567 generic.go:356] "Generic (PLEG): container finished" podID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerID="4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6" exitCode=0
Sep 11 04:43:13.995125 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.995095    2567 generic.go:356] "Generic (PLEG): container finished" podID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerID="0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029" exitCode=0
Sep 11 04:43:13.995125 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.995101    2567 generic.go:356] "Generic (PLEG): container finished" podID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerID="8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc" exitCode=0
Sep 11 04:43:13.995125 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.995106    2567 generic.go:356] "Generic (PLEG): container finished" podID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerID="0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f" exitCode=0
Sep 11 04:43:13.995125 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.995101    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003","Type":"ContainerDied","Data":"0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0"}
Sep 11 04:43:13.995700 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.995154    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:13.995700 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.995215    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003","Type":"ContainerDied","Data":"68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597"}
Sep 11 04:43:13.995700 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.995261    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003","Type":"ContainerDied","Data":"4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6"}
Sep 11 04:43:13.995700 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.995285    2567 scope.go:117] "RemoveContainer" containerID="0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0"
Sep 11 04:43:13.995700 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.995303    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003","Type":"ContainerDied","Data":"0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029"}
Sep 11 04:43:13.995700 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.995345    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003","Type":"ContainerDied","Data":"8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc"}
Sep 11 04:43:13.995700 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.995388    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003","Type":"ContainerDied","Data":"0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f"}
Sep 11 04:43:13.995700 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:13.995429    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"cea99d18-0ff7-41b6-b7e3-e7fc6f35f003","Type":"ContainerDied","Data":"5f8f5e08163d59fe63528f50208ef0fddcbac7c9b8610d4378c527ef2e7fedc4"}
Sep 11 04:43:14.004053 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.004035    2567 scope.go:117] "RemoveContainer" containerID="68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597"
Sep 11 04:43:14.011011 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.010994    2567 scope.go:117] "RemoveContainer" containerID="4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6"
Sep 11 04:43:14.018135 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.018118    2567 scope.go:117] "RemoveContainer" containerID="0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029"
Sep 11 04:43:14.018720 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.018696    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 04:43:14.023016 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.022998    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 04:43:14.025403 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.025389    2567 scope.go:117] "RemoveContainer" containerID="8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc"
Sep 11 04:43:14.032107 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.032088    2567 scope.go:117] "RemoveContainer" containerID="0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f"
Sep 11 04:43:14.039097 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.039082    2567 scope.go:117] "RemoveContainer" containerID="602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2"
Sep 11 04:43:14.045624 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.045610    2567 scope.go:117] "RemoveContainer" containerID="0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0"
Sep 11 04:43:14.045899 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:43:14.045879    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0\": container with ID starting with 0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0 not found: ID does not exist" containerID="0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0"
Sep 11 04:43:14.045973 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.045913    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0"} err="failed to get container status \"0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0\": rpc error: code = NotFound desc = could not find container \"0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0\": container with ID starting with 0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0 not found: ID does not exist"
Sep 11 04:43:14.045973 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.045937    2567 scope.go:117] "RemoveContainer" containerID="68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597"
Sep 11 04:43:14.046239 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:43:14.046221    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597\": container with ID starting with 68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597 not found: ID does not exist" containerID="68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597"
Sep 11 04:43:14.046285 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.046246    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597"} err="failed to get container status \"68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597\": rpc error: code = NotFound desc = could not find container \"68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597\": container with ID starting with 68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597 not found: ID does not exist"
Sep 11 04:43:14.046285 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.046261    2567 scope.go:117] "RemoveContainer" containerID="4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6"
Sep 11 04:43:14.046494 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:43:14.046475    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6\": container with ID starting with 4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6 not found: ID does not exist" containerID="4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6"
Sep 11 04:43:14.046557 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.046501    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6"} err="failed to get container status \"4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6\": rpc error: code = NotFound desc = could not find container \"4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6\": container with ID starting with 4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6 not found: ID does not exist"
Sep 11 04:43:14.046557 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.046522    2567 scope.go:117] "RemoveContainer" containerID="0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029"
Sep 11 04:43:14.046741 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:43:14.046726    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029\": container with ID starting with 0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029 not found: ID does not exist" containerID="0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029"
Sep 11 04:43:14.046776 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.046745    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029"} err="failed to get container status \"0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029\": rpc error: code = NotFound desc = could not find container \"0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029\": container with ID starting with 0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029 not found: ID does not exist"
Sep 11 04:43:14.046776 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.046757    2567 scope.go:117] "RemoveContainer" containerID="8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc"
Sep 11 04:43:14.046973 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:43:14.046956    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc\": container with ID starting with 8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc not found: ID does not exist" containerID="8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc"
Sep 11 04:43:14.047033 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.046980    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc"} err="failed to get container status \"8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc\": rpc error: code = NotFound desc = could not find container \"8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc\": container with ID starting with 8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc not found: ID does not exist"
Sep 11 04:43:14.047033 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.046997    2567 scope.go:117] "RemoveContainer" containerID="0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f"
Sep 11 04:43:14.047350 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:43:14.047331    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f\": container with ID starting with 0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f not found: ID does not exist" containerID="0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f"
Sep 11 04:43:14.047398 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.047354    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f"} err="failed to get container status \"0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f\": rpc error: code = NotFound desc = could not find container \"0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f\": container with ID starting with 0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f not found: ID does not exist"
Sep 11 04:43:14.047398 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.047369    2567 scope.go:117] "RemoveContainer" containerID="602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2"
Sep 11 04:43:14.047581 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:43:14.047565    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2\": container with ID starting with 602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2 not found: ID does not exist" containerID="602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2"
Sep 11 04:43:14.047617 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.047586    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2"} err="failed to get container status \"602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2\": rpc error: code = NotFound desc = could not find container \"602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2\": container with ID starting with 602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2 not found: ID does not exist"
Sep 11 04:43:14.047617 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.047599    2567 scope.go:117] "RemoveContainer" containerID="0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0"
Sep 11 04:43:14.047807 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.047787    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0"} err="failed to get container status \"0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0\": rpc error: code = NotFound desc = could not find container \"0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0\": container with ID starting with 0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0 not found: ID does not exist"
Sep 11 04:43:14.047878 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.047809    2567 scope.go:117] "RemoveContainer" containerID="68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597"
Sep 11 04:43:14.048086 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.048048    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597"} err="failed to get container status \"68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597\": rpc error: code = NotFound desc = could not find container \"68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597\": container with ID starting with 68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597 not found: ID does not exist"
Sep 11 04:43:14.048144 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.048091    2567 scope.go:117] "RemoveContainer" containerID="4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6"
Sep 11 04:43:14.048284 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.048265    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6"} err="failed to get container status \"4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6\": rpc error: code = NotFound desc = could not find container \"4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6\": container with ID starting with 4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6 not found: ID does not exist"
Sep 11 04:43:14.048350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.048284    2567 scope.go:117] "RemoveContainer" containerID="0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029"
Sep 11 04:43:14.048472 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.048456    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029"} err="failed to get container status \"0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029\": rpc error: code = NotFound desc = could not find container \"0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029\": container with ID starting with 0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029 not found: ID does not exist"
Sep 11 04:43:14.048537 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.048474    2567 scope.go:117] "RemoveContainer" containerID="8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc"
Sep 11 04:43:14.048712 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.048695    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc"} err="failed to get container status \"8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc\": rpc error: code = NotFound desc = could not find container \"8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc\": container with ID starting with 8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc not found: ID does not exist"
Sep 11 04:43:14.048766 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.048714    2567 scope.go:117] "RemoveContainer" containerID="0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f"
Sep 11 04:43:14.048899 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.048881    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f"} err="failed to get container status \"0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f\": rpc error: code = NotFound desc = could not find container \"0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f\": container with ID starting with 0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f not found: ID does not exist"
Sep 11 04:43:14.049001 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.048901    2567 scope.go:117] "RemoveContainer" containerID="602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2"
Sep 11 04:43:14.049147 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.049130    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2"} err="failed to get container status \"602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2\": rpc error: code = NotFound desc = could not find container \"602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2\": container with ID starting with 602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2 not found: ID does not exist"
Sep 11 04:43:14.049210 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.049149    2567 scope.go:117] "RemoveContainer" containerID="0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0"
Sep 11 04:43:14.049380 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.049361    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0"} err="failed to get container status \"0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0\": rpc error: code = NotFound desc = could not find container \"0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0\": container with ID starting with 0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0 not found: ID does not exist"
Sep 11 04:43:14.049439 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.049380    2567 scope.go:117] "RemoveContainer" containerID="68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597"
Sep 11 04:43:14.049583 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.049565    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597"} err="failed to get container status \"68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597\": rpc error: code = NotFound desc = could not find container \"68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597\": container with ID starting with 68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597 not found: ID does not exist"
Sep 11 04:43:14.049648 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.049583    2567 scope.go:117] "RemoveContainer" containerID="4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6"
Sep 11 04:43:14.049803 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.049785    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6"} err="failed to get container status \"4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6\": rpc error: code = NotFound desc = could not find container \"4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6\": container with ID starting with 4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6 not found: ID does not exist"
Sep 11 04:43:14.049861 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.049803    2567 scope.go:117] "RemoveContainer" containerID="0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029"
Sep 11 04:43:14.050001 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.049986    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029"} err="failed to get container status \"0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029\": rpc error: code = NotFound desc = could not find container \"0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029\": container with ID starting with 0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029 not found: ID does not exist"
Sep 11 04:43:14.050042 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.050002    2567 scope.go:117] "RemoveContainer" containerID="8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc"
Sep 11 04:43:14.050235 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.050218    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc"} err="failed to get container status \"8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc\": rpc error: code = NotFound desc = could not find container \"8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc\": container with ID starting with 8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc not found: ID does not exist"
Sep 11 04:43:14.050286 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.050235    2567 scope.go:117] "RemoveContainer" containerID="0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f"
Sep 11 04:43:14.050440 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.050424    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f"} err="failed to get container status \"0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f\": rpc error: code = NotFound desc = could not find container \"0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f\": container with ID starting with 0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f not found: ID does not exist"
Sep 11 04:43:14.050483 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.050441    2567 scope.go:117] "RemoveContainer" containerID="602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2"
Sep 11 04:43:14.050621 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.050606    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2"} err="failed to get container status \"602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2\": rpc error: code = NotFound desc = could not find container \"602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2\": container with ID starting with 602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2 not found: ID does not exist"
Sep 11 04:43:14.050662 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.050621    2567 scope.go:117] "RemoveContainer" containerID="0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0"
Sep 11 04:43:14.050823 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.050806    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0"} err="failed to get container status \"0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0\": rpc error: code = NotFound desc = could not find container \"0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0\": container with ID starting with 0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0 not found: ID does not exist"
Sep 11 04:43:14.050885 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.050825    2567 scope.go:117] "RemoveContainer" containerID="68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597"
Sep 11 04:43:14.051031 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.051016    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597"} err="failed to get container status \"68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597\": rpc error: code = NotFound desc = could not find container \"68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597\": container with ID starting with 68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597 not found: ID does not exist"
Sep 11 04:43:14.051101 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.051031    2567 scope.go:117] "RemoveContainer" containerID="4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6"
Sep 11 04:43:14.051292 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.051277    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6"} err="failed to get container status \"4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6\": rpc error: code = NotFound desc = could not find container \"4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6\": container with ID starting with 4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6 not found: ID does not exist"
Sep 11 04:43:14.051334 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.051294    2567 scope.go:117] "RemoveContainer" containerID="0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029"
Sep 11 04:43:14.051503 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.051487    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029"} err="failed to get container status \"0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029\": rpc error: code = NotFound desc = could not find container \"0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029\": container with ID starting with 0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029 not found: ID does not exist"
Sep 11 04:43:14.051554 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.051505    2567 scope.go:117] "RemoveContainer" containerID="8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc"
Sep 11 04:43:14.051706 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.051692    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc"} err="failed to get container status \"8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc\": rpc error: code = NotFound desc = could not find container \"8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc\": container with ID starting with 8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc not found: ID does not exist"
Sep 11 04:43:14.051759 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.051707    2567 scope.go:117] "RemoveContainer" containerID="0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f"
Sep 11 04:43:14.051889 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.051875    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f"} err="failed to get container status \"0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f\": rpc error: code = NotFound desc = could not find container \"0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f\": container with ID starting with 0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f not found: ID does not exist"
Sep 11 04:43:14.051941 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.051888    2567 scope.go:117] "RemoveContainer" containerID="602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2"
Sep 11 04:43:14.052088 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.052051    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2"} err="failed to get container status \"602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2\": rpc error: code = NotFound desc = could not find container \"602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2\": container with ID starting with 602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2 not found: ID does not exist"
Sep 11 04:43:14.052088 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.052081    2567 scope.go:117] "RemoveContainer" containerID="0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0"
Sep 11 04:43:14.052295 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.052276    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0"} err="failed to get container status \"0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0\": rpc error: code = NotFound desc = could not find container \"0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0\": container with ID starting with 0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0 not found: ID does not exist"
Sep 11 04:43:14.052337 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.052297    2567 scope.go:117] "RemoveContainer" containerID="68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597"
Sep 11 04:43:14.052497 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.052482    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597"} err="failed to get container status \"68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597\": rpc error: code = NotFound desc = could not find container \"68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597\": container with ID starting with 68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597 not found: ID does not exist"
Sep 11 04:43:14.052539 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.052500    2567 scope.go:117] "RemoveContainer" containerID="4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6"
Sep 11 04:43:14.052707 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.052691    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6"} err="failed to get container status \"4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6\": rpc error: code = NotFound desc = could not find container \"4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6\": container with ID starting with 4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6 not found: ID does not exist"
Sep 11 04:43:14.052753 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.052709    2567 scope.go:117] "RemoveContainer" containerID="0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029"
Sep 11 04:43:14.052994 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.052966    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029"} err="failed to get container status \"0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029\": rpc error: code = NotFound desc = could not find container \"0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029\": container with ID starting with 0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029 not found: ID does not exist"
Sep 11 04:43:14.052994 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.052995    2567 scope.go:117] "RemoveContainer" containerID="8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc"
Sep 11 04:43:14.053342 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.053308    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc"} err="failed to get container status \"8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc\": rpc error: code = NotFound desc = could not find container \"8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc\": container with ID starting with 8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc not found: ID does not exist"
Sep 11 04:43:14.053342 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.053343    2567 scope.go:117] "RemoveContainer" containerID="0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f"
Sep 11 04:43:14.053629 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.053608    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f"} err="failed to get container status \"0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f\": rpc error: code = NotFound desc = could not find container \"0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f\": container with ID starting with 0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f not found: ID does not exist"
Sep 11 04:43:14.053685 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.053631    2567 scope.go:117] "RemoveContainer" containerID="602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2"
Sep 11 04:43:14.054006 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.053911    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2"} err="failed to get container status \"602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2\": rpc error: code = NotFound desc = could not find container \"602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2\": container with ID starting with 602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2 not found: ID does not exist"
Sep 11 04:43:14.054006 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.053938    2567 scope.go:117] "RemoveContainer" containerID="0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0"
Sep 11 04:43:14.054274 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.054255    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0"} err="failed to get container status \"0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0\": rpc error: code = NotFound desc = could not find container \"0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0\": container with ID starting with 0930925a6e0618a12dc460163d40926f433529f20e37af746a073c7ed7d1eda0 not found: ID does not exist"
Sep 11 04:43:14.054340 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.054278    2567 scope.go:117] "RemoveContainer" containerID="68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597"
Sep 11 04:43:14.054590 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.054569    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597"} err="failed to get container status \"68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597\": rpc error: code = NotFound desc = could not find container \"68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597\": container with ID starting with 68688193b9594799c2c793c9352213069aa058bec31efcc3009c1b0f2fa0f597 not found: ID does not exist"
Sep 11 04:43:14.054654 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.054592    2567 scope.go:117] "RemoveContainer" containerID="4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6"
Sep 11 04:43:14.054878 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.054848    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6"} err="failed to get container status \"4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6\": rpc error: code = NotFound desc = could not find container \"4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6\": container with ID starting with 4bd9bf483e2e76e9b215673e9188e26ac341367ff6474330b29f3668aa4c4bf6 not found: ID does not exist"
Sep 11 04:43:14.054878 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.054869    2567 scope.go:117] "RemoveContainer" containerID="0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029"
Sep 11 04:43:14.055061 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055041    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 04:43:14.055177 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055152    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029"} err="failed to get container status \"0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029\": rpc error: code = NotFound desc = could not find container \"0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029\": container with ID starting with 0ba64141d724e26a13741f2382fd5164caaa309972173bb874a8908886cbb029 not found: ID does not exist"
Sep 11 04:43:14.055220 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055177    2567 scope.go:117] "RemoveContainer" containerID="8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc"
Sep 11 04:43:14.055378 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055364    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="config-reloader"
Sep 11 04:43:14.055428 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055380    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="config-reloader"
Sep 11 04:43:14.055428 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055391    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="init-config-reloader"
Sep 11 04:43:14.055428 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055400    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="init-config-reloader"
Sep 11 04:43:14.055428 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055412    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="kube-rbac-proxy-web"
Sep 11 04:43:14.055428 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055419    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="kube-rbac-proxy-web"
Sep 11 04:43:14.055575 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055430    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="kube-rbac-proxy-thanos"
Sep 11 04:43:14.055575 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055437    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="kube-rbac-proxy-thanos"
Sep 11 04:43:14.055575 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055448    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="prometheus"
Sep 11 04:43:14.055575 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055446    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc"} err="failed to get container status \"8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc\": rpc error: code = NotFound desc = could not find container \"8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc\": container with ID starting with 8f52aece48a42f4922ea16f83c08600fff0219a21fa598487b007c884eb261fc not found: ID does not exist"
Sep 11 04:43:14.055575 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055466    2567 scope.go:117] "RemoveContainer" containerID="0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f"
Sep 11 04:43:14.055575 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055456    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="prometheus"
Sep 11 04:43:14.055575 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055524    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="thanos-sidecar"
Sep 11 04:43:14.055575 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055532    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="thanos-sidecar"
Sep 11 04:43:14.055575 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055544    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="kube-rbac-proxy"
Sep 11 04:43:14.055575 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055549    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="kube-rbac-proxy"
Sep 11 04:43:14.055932 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055656    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="kube-rbac-proxy-thanos"
Sep 11 04:43:14.055932 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055666    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="kube-rbac-proxy-web"
Sep 11 04:43:14.055932 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055674    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="prometheus"
Sep 11 04:43:14.055932 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055682    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="thanos-sidecar"
Sep 11 04:43:14.055932 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055687    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="config-reloader"
Sep 11 04:43:14.055932 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055693    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" containerName="kube-rbac-proxy"
Sep 11 04:43:14.055932 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055716    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f"} err="failed to get container status \"0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f\": rpc error: code = NotFound desc = could not find container \"0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f\": container with ID starting with 0a96f96ff3ecbd54ff303646b25b3b317ea46c5824db6ad2927815168e24e91f not found: ID does not exist"
Sep 11 04:43:14.055932 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055735    2567 scope.go:117] "RemoveContainer" containerID="602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2"
Sep 11 04:43:14.056202 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.055968    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2"} err="failed to get container status \"602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2\": rpc error: code = NotFound desc = could not find container \"602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2\": container with ID starting with 602bedc18088d03bf3bea9c0013b200ccca7a15a2d6983a9abf94d4512d303b2 not found: ID does not exist"
Sep 11 04:43:14.061045 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.061028    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.063802 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.063782    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 11 04:43:14.064896 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.064871    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-v7lvc\""
Sep 11 04:43:14.064981 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.064903    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 11 04:43:14.064981 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.064934    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 11 04:43:14.065125 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.065003    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 11 04:43:14.065125 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.064920    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 11 04:43:14.065125 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.065027    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 11 04:43:14.065269 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.064881    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 11 04:43:14.065269 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.065131    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-d0sr6ln40s2k\""
Sep 11 04:43:14.065269 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.065193    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 11 04:43:14.065269 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.065194    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 11 04:43:14.065269 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.065253    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 11 04:43:14.065269 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.065264    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 11 04:43:14.065512 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.065376    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 11 04:43:14.067160 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.067136    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 11 04:43:14.069272 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.069256    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 11 04:43:14.075816 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.075792    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 04:43:14.204700 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.204667    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.204700 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.204700    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.204889 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.204718    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.204889 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.204759    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.204889 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.204778    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.204889 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.204808    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.204889 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.204830    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.204889 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.204844    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.204889 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.204866    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r2jv\" (UniqueName: \"kubernetes.io/projected/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-kube-api-access-8r2jv\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.205120 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.204905    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.205120 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.204935    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.205120 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.204961    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.205120 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.205009    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.205120 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.205036    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.205120 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.205056    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-config\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.205294 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.205129    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-config-out\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.205294 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.205145    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.205294 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.205177    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-web-config\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.205294 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.205198    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.205294 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.205235    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.306583 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.306554    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.306583 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.306584    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.306743 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.306602    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.306743 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.306617    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.306743 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.306634    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.306743 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.306659    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.306950 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.306799    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.306950 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.306843    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.306950 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.306872    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.306950 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.306902    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r2jv\" (UniqueName: \"kubernetes.io/projected/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-kube-api-access-8r2jv\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.307174 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.306951    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.307174 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.306983    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.307174 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.307015    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.307174 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.307048    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.307174 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.307097    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.307174 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.307126    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-config\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.307439 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.307189    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-config-out\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.307439 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.307213    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.307439 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.307241    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-web-config\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.307439 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.307244    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.307439 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.307270    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.307439 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.307344    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.307439 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.307375    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.307862 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.307842    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.308208 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.308182    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.309000 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.308972    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.310149 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.310126    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.310247 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.310163    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.310247 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.310176    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.310377 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.310327    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.310377 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.310353    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.310538 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.310518    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.311003 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.310976    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.312035 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.312008    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-config\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.312161 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.312058    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.312445 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.312424    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.312769 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.312733    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-config-out\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.312867 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.312842    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-web-config\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.313707 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.313690    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.315176 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.315156    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r2jv\" (UniqueName: \"kubernetes.io/projected/2261ebde-fc80-4c2b-b741-6db6b80bbfb7-kube-api-access-8r2jv\") pod \"prometheus-k8s-0\" (UID: \"2261ebde-fc80-4c2b-b741-6db6b80bbfb7\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.372217 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.372183    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:14.658049 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:14.656481    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 11 04:43:14.658376 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:43:14.658339    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2261ebde_fc80_4c2b_b741_6db6b80bbfb7.slice/crio-5ab8b27a6cc598ee648b098fff2115a4ffeadd5991005814f1617781ca097bed WatchSource:0}: Error finding container 5ab8b27a6cc598ee648b098fff2115a4ffeadd5991005814f1617781ca097bed: Status 404 returned error can't find the container with id 5ab8b27a6cc598ee648b098fff2115a4ffeadd5991005814f1617781ca097bed
Sep 11 04:43:15.001852 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:15.001752    2567 generic.go:356] "Generic (PLEG): container finished" podID="2261ebde-fc80-4c2b-b741-6db6b80bbfb7" containerID="bd381404cd54d16bbbe0715a7d9346dac19d8d0482a012921ce42eb38dbda28c" exitCode=0
Sep 11 04:43:15.002271 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:15.001859    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2261ebde-fc80-4c2b-b741-6db6b80bbfb7","Type":"ContainerDied","Data":"bd381404cd54d16bbbe0715a7d9346dac19d8d0482a012921ce42eb38dbda28c"}
Sep 11 04:43:15.002271 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:15.001938    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2261ebde-fc80-4c2b-b741-6db6b80bbfb7","Type":"ContainerStarted","Data":"5ab8b27a6cc598ee648b098fff2115a4ffeadd5991005814f1617781ca097bed"}
Sep 11 04:43:16.010326 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:16.010282    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2261ebde-fc80-4c2b-b741-6db6b80bbfb7","Type":"ContainerStarted","Data":"151bcfe56420fdb0ec476f1fa0e896612d7fb42d73e1e953904ca067f555890c"}
Sep 11 04:43:16.010832 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:16.010405    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2261ebde-fc80-4c2b-b741-6db6b80bbfb7","Type":"ContainerStarted","Data":"9627459b9e34addb9c65b450250d63aa0ad3bc3b8faffb57c4024811b1da9f8a"}
Sep 11 04:43:16.010832 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:16.010453    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2261ebde-fc80-4c2b-b741-6db6b80bbfb7","Type":"ContainerStarted","Data":"484ec80e69c27cd87a1260a6730d5fb212ad1ffceebeef72a5489b8c4c2e12a1"}
Sep 11 04:43:16.010832 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:16.010494    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2261ebde-fc80-4c2b-b741-6db6b80bbfb7","Type":"ContainerStarted","Data":"18f070deeb4eec1e25270783a16396ced777a898ea4063125f99348c36842db5"}
Sep 11 04:43:16.010832 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:16.010530    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2261ebde-fc80-4c2b-b741-6db6b80bbfb7","Type":"ContainerStarted","Data":"235a4b4108455484499fe176fdb61efcb084e71367a9717b5b4f1d5c2bacda5f"}
Sep 11 04:43:16.010832 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:16.010579    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"2261ebde-fc80-4c2b-b741-6db6b80bbfb7","Type":"ContainerStarted","Data":"4d40636494cff1542f15211368a2a498dd99e451b8a8aa6ce7a69ced747d8fc2"}
Sep 11 04:43:16.016932 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:16.016898    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cea99d18-0ff7-41b6-b7e3-e7fc6f35f003" path="/var/lib/kubelet/pods/cea99d18-0ff7-41b6-b7e3-e7fc6f35f003/volumes"
Sep 11 04:43:16.040658 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:16.040610    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.022458492 podStartE2EDuration="2.040595764s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.137272ms" isStatefulPod=false podCreationTimestamp="2026-09-11 04:43:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:43:16.037863968 +0000 UTC m=+210.509327867" watchObservedRunningTime="2026-09-11 04:43:16.040595764 +0000 UTC m=+210.512059675"
Sep 11 04:43:16.751022 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:16.750986    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:43:16.851591 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:16.851545    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:43:19.372485 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:19.372449    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:43:27.718184 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:27.718123    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:43:27.817725 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:27.817691    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:43:45.320727 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:45.320674    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:43:45.376855 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:45.376824    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:43:45.740139 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:45.740037    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:43:45.804686 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:45.804654    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:43:45.864439 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:45.864403    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:43:45.917988 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:45.917959    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:43:46.467136 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:46.467100    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:43:56.369362 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:56.369321    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:43:56.469827 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:56.469797    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:43:56.528284 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:43:56.528249    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:44:14.372685 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:44:14.372597    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:44:14.388872 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:44:14.388848    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:44:15.255447 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:44:15.255421    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 11 04:44:21.219531 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:44:21.219480    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:44:21.311744 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:44:21.311709    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:44:45.963451 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:44:45.963420    2567 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 04:44:48.076619 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:44:48.076585    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:44:48.878376 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:44:48.878341    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:44:51.861317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:44:51.861288    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:44:51.962023 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:44:51.961993    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:44:54.049149 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:44:54.049113    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:44:54.151119 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:44:54.151083    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:44:54.372113 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:44:54.372006    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:44:54.474984 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:44:54.474946    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:44:56.270986 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:44:56.270947    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:44:56.455151 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:44:56.455117    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:44:58.685029 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:44:58.684998    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:44:58.741733 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:44:58.741699    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:04.669740 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:04.669704    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:04.736916 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:04.736884    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-z4c9t"]
Sep 11 04:45:04.740427 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:04.740406    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-z4c9t"
Sep 11 04:45:04.742666 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:04.742645    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 11 04:45:04.742790 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:04.742773    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 11 04:45:04.743662 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:04.743647    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-wgqfq\""
Sep 11 04:45:04.748815 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:04.748793    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-z4c9t"]
Sep 11 04:45:04.781720 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:04.781692    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2c4t\" (UniqueName: \"kubernetes.io/projected/afdd57ac-e9c1-4117-b25c-b06e9f78ac32-kube-api-access-b2c4t\") pod \"cert-manager-55b4d954d5-z4c9t\" (UID: \"afdd57ac-e9c1-4117-b25c-b06e9f78ac32\") " pod="cert-manager/cert-manager-55b4d954d5-z4c9t"
Sep 11 04:45:04.781828 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:04.781742    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/afdd57ac-e9c1-4117-b25c-b06e9f78ac32-bound-sa-token\") pod \"cert-manager-55b4d954d5-z4c9t\" (UID: \"afdd57ac-e9c1-4117-b25c-b06e9f78ac32\") " pod="cert-manager/cert-manager-55b4d954d5-z4c9t"
Sep 11 04:45:04.883178 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:04.883151    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2c4t\" (UniqueName: \"kubernetes.io/projected/afdd57ac-e9c1-4117-b25c-b06e9f78ac32-kube-api-access-b2c4t\") pod \"cert-manager-55b4d954d5-z4c9t\" (UID: \"afdd57ac-e9c1-4117-b25c-b06e9f78ac32\") " pod="cert-manager/cert-manager-55b4d954d5-z4c9t"
Sep 11 04:45:04.883315 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:04.883202    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/afdd57ac-e9c1-4117-b25c-b06e9f78ac32-bound-sa-token\") pod \"cert-manager-55b4d954d5-z4c9t\" (UID: \"afdd57ac-e9c1-4117-b25c-b06e9f78ac32\") " pod="cert-manager/cert-manager-55b4d954d5-z4c9t"
Sep 11 04:45:04.892442 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:04.892408    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/afdd57ac-e9c1-4117-b25c-b06e9f78ac32-bound-sa-token\") pod \"cert-manager-55b4d954d5-z4c9t\" (UID: \"afdd57ac-e9c1-4117-b25c-b06e9f78ac32\") " pod="cert-manager/cert-manager-55b4d954d5-z4c9t"
Sep 11 04:45:04.892544 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:04.892514    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2c4t\" (UniqueName: \"kubernetes.io/projected/afdd57ac-e9c1-4117-b25c-b06e9f78ac32-kube-api-access-b2c4t\") pod \"cert-manager-55b4d954d5-z4c9t\" (UID: \"afdd57ac-e9c1-4117-b25c-b06e9f78ac32\") " pod="cert-manager/cert-manager-55b4d954d5-z4c9t"
Sep 11 04:45:05.050694 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:05.050670    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-z4c9t"
Sep 11 04:45:05.124540 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:05.124509    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:05.220130 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:05.220094    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:05.319653 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:05.319632    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-z4c9t"]
Sep 11 04:45:05.321704 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:45:05.321677    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podafdd57ac_e9c1_4117_b25c_b06e9f78ac32.slice/crio-91a4f26226246efcdaf105d3bee8de2bea5f10174c9b31e361545cd9321fe50a WatchSource:0}: Error finding container 91a4f26226246efcdaf105d3bee8de2bea5f10174c9b31e361545cd9321fe50a: Status 404 returned error can't find the container with id 91a4f26226246efcdaf105d3bee8de2bea5f10174c9b31e361545cd9321fe50a
Sep 11 04:45:05.323561 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:05.323541    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 04:45:05.449142 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:05.449103    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-z4c9t" event={"ID":"afdd57ac-e9c1-4117-b25c-b06e9f78ac32","Type":"ContainerStarted","Data":"91a4f26226246efcdaf105d3bee8de2bea5f10174c9b31e361545cd9321fe50a"}
Sep 11 04:45:06.144028 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:06.143991    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:06.203618 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:06.203560    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:08.466589 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:08.466499    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-z4c9t" event={"ID":"afdd57ac-e9c1-4117-b25c-b06e9f78ac32","Type":"ContainerStarted","Data":"26b18ef3345d969434424cdafd298a9d554e5a048c8b9fff629b851f2b4be61d"}
Sep 11 04:45:08.480896 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:08.480817    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-z4c9t" podStartSLOduration=1.757991188 podStartE2EDuration="4.480800898s" totalImagesPullingTime="2.72280971s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:45:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:45:08.480440217 +0000 UTC m=+322.951904093" watchObservedRunningTime="2026-09-11 04:45:08.480800898 +0000 UTC m=+322.952264774"
Sep 11 04:45:12.166571 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:12.166538    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:12.270430 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:12.270393    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:14.089019 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:14.088977    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:25.617138 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:25.617098    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:25.713009 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:25.712977    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:26.233772 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.233737    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:26.528231 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.528200    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:26.669842 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.669809    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g"]
Sep 11 04:45:26.719762 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.719736    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g"]
Sep 11 04:45:26.719925 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.719852    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g"
Sep 11 04:45:26.722561 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.722538    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4g22g\""
Sep 11 04:45:26.722706 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.722574    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 04:45:26.722706 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.722590    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 04:45:26.722706 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.722585    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 04:45:26.722891 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.722875    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 04:45:26.766329 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.766299    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ece7b42e-bc90-4471-bb64-202955606c18-manifests\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-bht4g\" (UID: \"ece7b42e-bc90-4471-bb64-202955606c18\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g"
Sep 11 04:45:26.766469 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.766338    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ece7b42e-bc90-4471-bb64-202955606c18-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-bht4g\" (UID: \"ece7b42e-bc90-4471-bb64-202955606c18\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g"
Sep 11 04:45:26.766469 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.766368    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ece7b42e-bc90-4471-bb64-202955606c18-tmp\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-bht4g\" (UID: \"ece7b42e-bc90-4471-bb64-202955606c18\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g"
Sep 11 04:45:26.766469 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.766389    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ece7b42e-bc90-4471-bb64-202955606c18-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-bht4g\" (UID: \"ece7b42e-bc90-4471-bb64-202955606c18\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g"
Sep 11 04:45:26.766469 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.766449    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99v5c\" (UniqueName: \"kubernetes.io/projected/ece7b42e-bc90-4471-bb64-202955606c18-kube-api-access-99v5c\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-bht4g\" (UID: \"ece7b42e-bc90-4471-bb64-202955606c18\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g"
Sep 11 04:45:26.867390 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.867306    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ece7b42e-bc90-4471-bb64-202955606c18-tmp\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-bht4g\" (UID: \"ece7b42e-bc90-4471-bb64-202955606c18\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g"
Sep 11 04:45:26.867390 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.867344    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ece7b42e-bc90-4471-bb64-202955606c18-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-bht4g\" (UID: \"ece7b42e-bc90-4471-bb64-202955606c18\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g"
Sep 11 04:45:26.867390 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.867367    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99v5c\" (UniqueName: \"kubernetes.io/projected/ece7b42e-bc90-4471-bb64-202955606c18-kube-api-access-99v5c\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-bht4g\" (UID: \"ece7b42e-bc90-4471-bb64-202955606c18\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g"
Sep 11 04:45:26.867601 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.867420    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ece7b42e-bc90-4471-bb64-202955606c18-manifests\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-bht4g\" (UID: \"ece7b42e-bc90-4471-bb64-202955606c18\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g"
Sep 11 04:45:26.867601 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.867445    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ece7b42e-bc90-4471-bb64-202955606c18-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-bht4g\" (UID: \"ece7b42e-bc90-4471-bb64-202955606c18\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g"
Sep 11 04:45:26.867879 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.867858    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ece7b42e-bc90-4471-bb64-202955606c18-manifests\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-bht4g\" (UID: \"ece7b42e-bc90-4471-bb64-202955606c18\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g"
Sep 11 04:45:26.869800 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.869772    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ece7b42e-bc90-4471-bb64-202955606c18-tmp\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-bht4g\" (UID: \"ece7b42e-bc90-4471-bb64-202955606c18\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g"
Sep 11 04:45:26.869999 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.869981    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ece7b42e-bc90-4471-bb64-202955606c18-webhook-cert\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-bht4g\" (UID: \"ece7b42e-bc90-4471-bb64-202955606c18\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g"
Sep 11 04:45:26.870040 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.869998    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ece7b42e-bc90-4471-bb64-202955606c18-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-bht4g\" (UID: \"ece7b42e-bc90-4471-bb64-202955606c18\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g"
Sep 11 04:45:26.875432 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:26.875411    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99v5c\" (UniqueName: \"kubernetes.io/projected/ece7b42e-bc90-4471-bb64-202955606c18-kube-api-access-99v5c\") pod \"opendatahub-operator-controller-manager-7fbc9b44c4-bht4g\" (UID: \"ece7b42e-bc90-4471-bb64-202955606c18\") " pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g"
Sep 11 04:45:27.030406 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:27.030370    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g"
Sep 11 04:45:27.303724 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:27.303694    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g"]
Sep 11 04:45:27.306349 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:45:27.306323    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podece7b42e_bc90_4471_bb64_202955606c18.slice/crio-399b59d55f3d2f6caa4f0777ece6c87e463c1347eb845d86ef38d57dd38e03d6 WatchSource:0}: Error finding container 399b59d55f3d2f6caa4f0777ece6c87e463c1347eb845d86ef38d57dd38e03d6: Status 404 returned error can't find the container with id 399b59d55f3d2f6caa4f0777ece6c87e463c1347eb845d86ef38d57dd38e03d6
Sep 11 04:45:27.544807 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:27.544771    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g" event={"ID":"ece7b42e-bc90-4471-bb64-202955606c18","Type":"ContainerStarted","Data":"399b59d55f3d2f6caa4f0777ece6c87e463c1347eb845d86ef38d57dd38e03d6"}
Sep 11 04:45:27.566353 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:27.566266    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:30.562524 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:30.562483    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g" event={"ID":"ece7b42e-bc90-4471-bb64-202955606c18","Type":"ContainerStarted","Data":"2c624059e79871194c56399e945c5f7e5779e392ee4adbb99eb990dc2bf16282"}
Sep 11 04:45:31.568196 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:31.568168    2567 generic.go:356] "Generic (PLEG): container finished" podID="ece7b42e-bc90-4471-bb64-202955606c18" containerID="2c624059e79871194c56399e945c5f7e5779e392ee4adbb99eb990dc2bf16282" exitCode=0
Sep 11 04:45:31.568505 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:31.568246    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g" event={"ID":"ece7b42e-bc90-4471-bb64-202955606c18","Type":"ContainerDied","Data":"2c624059e79871194c56399e945c5f7e5779e392ee4adbb99eb990dc2bf16282"}
Sep 11 04:45:32.574862 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:32.574826    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g" event={"ID":"ece7b42e-bc90-4471-bb64-202955606c18","Type":"ContainerStarted","Data":"17b36ecc3c212a028fb9523d135123c4994450f5bfd4b8c0bde0adb3f4aba590"}
Sep 11 04:45:32.575298 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:32.575078    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g"
Sep 11 04:45:32.597485 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:32.597445    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g" podStartSLOduration=1.9526578159999999 podStartE2EDuration="6.597430693s" totalImagesPullingTime="3.558809379s" totalInitContainerRuntime="1.085963498s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:45:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:45:32.595253478 +0000 UTC m=+347.066717381" watchObservedRunningTime="2026-09-11 04:45:32.597430693 +0000 UTC m=+347.068894569"
Sep 11 04:45:38.670993 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:38.670906    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:38.767170 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:38.767125    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:43.580924 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:43.580892    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7fbc9b44c4-bht4g"
Sep 11 04:45:45.745492 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:45.745461    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:45.888123 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:45.888092    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:46.060449 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:46.060409    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:46.308723 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:46.308660    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:46.408944 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:46.408842    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:47.510738 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:47.510702    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:47.607505 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:47.607472    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:48.336364 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:48.336333    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:48.436890 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:48.436851    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:48.537293 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:48.537263    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:52.537765 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:52.537730    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:52.638234 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:52.638206    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:52.852258 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:52.852150    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:53.585440 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:53.585403    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:53.639335 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:53.639310    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:56.804174 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:56.804125    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:56.861456 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:56.861416    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:56.958769 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:56.958742    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:57.299965 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:57.299924    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:45:57.369898 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:45:57.369858    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:46:04.162327 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.162291    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"]
Sep 11 04:46:04.240501 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.240461    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"]
Sep 11 04:46:04.240686 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.240573    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:04.243339 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.243310    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 04:46:04.243498 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.243426    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 11 04:46:04.243498 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.243355    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 11 04:46:04.243498 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.243391    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 11 04:46:04.243498 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.243391    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 04:46:04.243720 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.243355    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-2brz8\""
Sep 11 04:46:04.243770 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.243747    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 04:46:04.286829 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.286806    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cbd8223-30c6-4ea7-b72f-017985855258-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-k9cf4\" (UID: \"4cbd8223-30c6-4ea7-b72f-017985855258\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:04.286950 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.286835    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cbd8223-30c6-4ea7-b72f-017985855258-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-k9cf4\" (UID: \"4cbd8223-30c6-4ea7-b72f-017985855258\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:04.286950 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.286860    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsn8w\" (UniqueName: \"kubernetes.io/projected/4cbd8223-30c6-4ea7-b72f-017985855258-kube-api-access-dsn8w\") pod \"istiod-openshift-gateway-5677ccb49b-k9cf4\" (UID: \"4cbd8223-30c6-4ea7-b72f-017985855258\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:04.286950 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.286877    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4cbd8223-30c6-4ea7-b72f-017985855258-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-k9cf4\" (UID: \"4cbd8223-30c6-4ea7-b72f-017985855258\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:04.287156 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.286990    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4cbd8223-30c6-4ea7-b72f-017985855258-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-k9cf4\" (UID: \"4cbd8223-30c6-4ea7-b72f-017985855258\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:04.287156 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.287029    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4cbd8223-30c6-4ea7-b72f-017985855258-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-k9cf4\" (UID: \"4cbd8223-30c6-4ea7-b72f-017985855258\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:04.287156 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.287061    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4cbd8223-30c6-4ea7-b72f-017985855258-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-k9cf4\" (UID: \"4cbd8223-30c6-4ea7-b72f-017985855258\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:04.388232 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.388196    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cbd8223-30c6-4ea7-b72f-017985855258-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-k9cf4\" (UID: \"4cbd8223-30c6-4ea7-b72f-017985855258\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:04.388232 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.388233    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cbd8223-30c6-4ea7-b72f-017985855258-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-k9cf4\" (UID: \"4cbd8223-30c6-4ea7-b72f-017985855258\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:04.388464 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.388262    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsn8w\" (UniqueName: \"kubernetes.io/projected/4cbd8223-30c6-4ea7-b72f-017985855258-kube-api-access-dsn8w\") pod \"istiod-openshift-gateway-5677ccb49b-k9cf4\" (UID: \"4cbd8223-30c6-4ea7-b72f-017985855258\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:04.388464 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.388372    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4cbd8223-30c6-4ea7-b72f-017985855258-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-k9cf4\" (UID: \"4cbd8223-30c6-4ea7-b72f-017985855258\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:04.388464 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.388439    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4cbd8223-30c6-4ea7-b72f-017985855258-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-k9cf4\" (UID: \"4cbd8223-30c6-4ea7-b72f-017985855258\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:04.388464 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.388463    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4cbd8223-30c6-4ea7-b72f-017985855258-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-k9cf4\" (UID: \"4cbd8223-30c6-4ea7-b72f-017985855258\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:04.388692 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.388490    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4cbd8223-30c6-4ea7-b72f-017985855258-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-k9cf4\" (UID: \"4cbd8223-30c6-4ea7-b72f-017985855258\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:04.389222 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.389197    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/4cbd8223-30c6-4ea7-b72f-017985855258-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-k9cf4\" (UID: \"4cbd8223-30c6-4ea7-b72f-017985855258\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:04.390679 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.390650    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/4cbd8223-30c6-4ea7-b72f-017985855258-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-k9cf4\" (UID: \"4cbd8223-30c6-4ea7-b72f-017985855258\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:04.390900 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.390882    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/4cbd8223-30c6-4ea7-b72f-017985855258-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-k9cf4\" (UID: \"4cbd8223-30c6-4ea7-b72f-017985855258\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:04.391282 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.391262    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/4cbd8223-30c6-4ea7-b72f-017985855258-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-k9cf4\" (UID: \"4cbd8223-30c6-4ea7-b72f-017985855258\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:04.391388 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.391368    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4cbd8223-30c6-4ea7-b72f-017985855258-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-k9cf4\" (UID: \"4cbd8223-30c6-4ea7-b72f-017985855258\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:04.404019 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.403996    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4cbd8223-30c6-4ea7-b72f-017985855258-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-k9cf4\" (UID: \"4cbd8223-30c6-4ea7-b72f-017985855258\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:04.404554 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.404535    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsn8w\" (UniqueName: \"kubernetes.io/projected/4cbd8223-30c6-4ea7-b72f-017985855258-kube-api-access-dsn8w\") pod \"istiod-openshift-gateway-5677ccb49b-k9cf4\" (UID: \"4cbd8223-30c6-4ea7-b72f-017985855258\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:04.549607 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.549575    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:04.826184 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:04.826114    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"]
Sep 11 04:46:04.829424 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:46:04.829402    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cbd8223_30c6_4ea7_b72f_017985855258.slice/crio-e1993720bac46323392c89b655e0051463ed2698331847c4e2e6193406e488eb WatchSource:0}: Error finding container e1993720bac46323392c89b655e0051463ed2698331847c4e2e6193406e488eb: Status 404 returned error can't find the container with id e1993720bac46323392c89b655e0051463ed2698331847c4e2e6193406e488eb
Sep 11 04:46:05.092442 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.092365    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6f84bb6f84-hlxct"]
Sep 11 04:46:05.095301 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.095286    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-hlxct"
Sep 11 04:46:05.097766 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.097747    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 11 04:46:05.098677 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.098662    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 11 04:46:05.098755 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.098704    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 11 04:46:05.099008 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.098989    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-6dbkv\""
Sep 11 04:46:05.099008 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.099001    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 11 04:46:05.099140 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.098994    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 11 04:46:05.113467 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.113443    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6f84bb6f84-hlxct"]
Sep 11 04:46:05.196587 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.196561    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/84168371-431c-43d4-aa16-06e00e771aeb-manager-config\") pod \"lws-controller-manager-6f84bb6f84-hlxct\" (UID: \"84168371-431c-43d4-aa16-06e00e771aeb\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-hlxct"
Sep 11 04:46:05.196959 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.196625    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4p89\" (UniqueName: \"kubernetes.io/projected/84168371-431c-43d4-aa16-06e00e771aeb-kube-api-access-n4p89\") pod \"lws-controller-manager-6f84bb6f84-hlxct\" (UID: \"84168371-431c-43d4-aa16-06e00e771aeb\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-hlxct"
Sep 11 04:46:05.196959 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.196682    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/84168371-431c-43d4-aa16-06e00e771aeb-cert\") pod \"lws-controller-manager-6f84bb6f84-hlxct\" (UID: \"84168371-431c-43d4-aa16-06e00e771aeb\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-hlxct"
Sep 11 04:46:05.196959 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.196713    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/84168371-431c-43d4-aa16-06e00e771aeb-metrics-cert\") pod \"lws-controller-manager-6f84bb6f84-hlxct\" (UID: \"84168371-431c-43d4-aa16-06e00e771aeb\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-hlxct"
Sep 11 04:46:05.297254 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.297217    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4p89\" (UniqueName: \"kubernetes.io/projected/84168371-431c-43d4-aa16-06e00e771aeb-kube-api-access-n4p89\") pod \"lws-controller-manager-6f84bb6f84-hlxct\" (UID: \"84168371-431c-43d4-aa16-06e00e771aeb\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-hlxct"
Sep 11 04:46:05.297425 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.297269    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/84168371-431c-43d4-aa16-06e00e771aeb-cert\") pod \"lws-controller-manager-6f84bb6f84-hlxct\" (UID: \"84168371-431c-43d4-aa16-06e00e771aeb\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-hlxct"
Sep 11 04:46:05.297425 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.297288    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/84168371-431c-43d4-aa16-06e00e771aeb-metrics-cert\") pod \"lws-controller-manager-6f84bb6f84-hlxct\" (UID: \"84168371-431c-43d4-aa16-06e00e771aeb\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-hlxct"
Sep 11 04:46:05.297425 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.297310    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/84168371-431c-43d4-aa16-06e00e771aeb-manager-config\") pod \"lws-controller-manager-6f84bb6f84-hlxct\" (UID: \"84168371-431c-43d4-aa16-06e00e771aeb\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-hlxct"
Sep 11 04:46:05.297998 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.297969    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/84168371-431c-43d4-aa16-06e00e771aeb-manager-config\") pod \"lws-controller-manager-6f84bb6f84-hlxct\" (UID: \"84168371-431c-43d4-aa16-06e00e771aeb\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-hlxct"
Sep 11 04:46:05.299765 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.299745    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/84168371-431c-43d4-aa16-06e00e771aeb-metrics-cert\") pod \"lws-controller-manager-6f84bb6f84-hlxct\" (UID: \"84168371-431c-43d4-aa16-06e00e771aeb\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-hlxct"
Sep 11 04:46:05.299874 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.299816    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/84168371-431c-43d4-aa16-06e00e771aeb-cert\") pod \"lws-controller-manager-6f84bb6f84-hlxct\" (UID: \"84168371-431c-43d4-aa16-06e00e771aeb\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-hlxct"
Sep 11 04:46:05.305757 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.305734    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4p89\" (UniqueName: \"kubernetes.io/projected/84168371-431c-43d4-aa16-06e00e771aeb-kube-api-access-n4p89\") pod \"lws-controller-manager-6f84bb6f84-hlxct\" (UID: \"84168371-431c-43d4-aa16-06e00e771aeb\") " pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-hlxct"
Sep 11 04:46:05.405922 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.405850    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-hlxct"
Sep 11 04:46:05.695420 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.695393    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6f84bb6f84-hlxct"]
Sep 11 04:46:05.729511 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:05.729473    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4" event={"ID":"4cbd8223-30c6-4ea7-b72f-017985855258","Type":"ContainerStarted","Data":"e1993720bac46323392c89b655e0051463ed2698331847c4e2e6193406e488eb"}
Sep 11 04:46:05.760400 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:46:05.760356    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84168371_431c_43d4_aa16_06e00e771aeb.slice/crio-570b4abfba049f6a36e282aec6370c3c65d1f3ad9acc6aa9c8f45c69fa4f608f WatchSource:0}: Error finding container 570b4abfba049f6a36e282aec6370c3c65d1f3ad9acc6aa9c8f45c69fa4f608f: Status 404 returned error can't find the container with id 570b4abfba049f6a36e282aec6370c3c65d1f3ad9acc6aa9c8f45c69fa4f608f
Sep 11 04:46:06.737140 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:06.737091    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-hlxct" event={"ID":"84168371-431c-43d4-aa16-06e00e771aeb","Type":"ContainerStarted","Data":"570b4abfba049f6a36e282aec6370c3c65d1f3ad9acc6aa9c8f45c69fa4f608f"}
Sep 11 04:46:06.740470 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:06.740437    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:46:06.846801 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:06.846720    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:46:07.037632 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:07.037581    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:46:07.180984 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:07.180934    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:46:07.257496 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:07.257461    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:46:07.257658 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:07.257524    2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892148Ki","pods":"250"}
Sep 11 04:46:07.746717 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:07.746279    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4" event={"ID":"4cbd8223-30c6-4ea7-b72f-017985855258","Type":"ContainerStarted","Data":"3f0f282b3ee8f91593719219f9be964bbfc2f65f821d0763768ea52f2581c88d"}
Sep 11 04:46:07.746717 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:07.746389    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:07.772979 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:07.772918    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4" podStartSLOduration=1.34720315 podStartE2EDuration="3.772893974s" totalImagesPullingTime="2.425690824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:46:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:46:07.770694873 +0000 UTC m=+382.242158751" watchObservedRunningTime="2026-09-11 04:46:07.772893974 +0000 UTC m=+382.244357851"
Sep 11 04:46:08.276333 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:08.276298    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:46:08.753495 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:08.753415    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-k9cf4"
Sep 11 04:46:09.756975 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:09.756938    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-hlxct" event={"ID":"84168371-431c-43d4-aa16-06e00e771aeb","Type":"ContainerStarted","Data":"f8fdc883244129541bd48a39c89bc70a43a4da4748961bd7c480a132be68f406"}
Sep 11 04:46:09.757521 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:09.757026    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-hlxct"
Sep 11 04:46:09.778166 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:09.778126    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-hlxct" podStartSLOduration=1.370956699 podStartE2EDuration="4.778113079s" totalImagesPullingTime="3.40715638s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:46:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:46:09.775847389 +0000 UTC m=+384.247311280" watchObservedRunningTime="2026-09-11 04:46:09.778113079 +0000 UTC m=+384.249576952"
Sep 11 04:46:20.763991 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:20.763957    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6f84bb6f84-hlxct"
Sep 11 04:46:29.845569 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:29.845529    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:46:57.716107 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:57.716050    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:46:57.784727 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:46:57.784697    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:11.099743 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:11.099663    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:11.959005 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:11.958973    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:12.697700 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:12.697659    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:15.723960 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:15.723927    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:16.031577 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:16.031540    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:16.152262 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:16.152214    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:16.278161 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:16.278123    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:16.371141 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:16.371023    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:16.846797 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:16.846758    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:16.977408 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:16.977371    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:17.563426 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:17.563390    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:18.730175 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:18.730134    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:18.939672 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:18.939623    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:20.920446 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:20.920408    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:22.758267 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:22.758228    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:23.043755 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:23.043664    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:23.281377 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:23.281346    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:26.124488 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:26.124442    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:26.203783 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:26.203751    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:26.513489 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:26.513454    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:26.614167 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:26.614132    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:26.925622 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:26.925538    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:27.061320 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:27.061289    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:35.048371 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:35.048336    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7"]
Sep 11 04:47:35.051835 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:35.051812    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7"
Sep 11 04:47:35.054433 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:35.054411    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 04:47:35.054524 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:35.054507    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-wvcgs\""
Sep 11 04:47:35.054567 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:35.054540    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 04:47:35.066994 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:35.066971    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7"]
Sep 11 04:47:35.105116 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:35.105087    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcc8e92d-9145-4558-84d0-023f8428b478-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-lfdw7\" (UID: \"fcc8e92d-9145-4558-84d0-023f8428b478\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7"
Sep 11 04:47:35.105245 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:35.105176    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2774\" (UniqueName: \"kubernetes.io/projected/fcc8e92d-9145-4558-84d0-023f8428b478-kube-api-access-h2774\") pod \"kuadrant-operator-controller-manager-5769f99d9f-lfdw7\" (UID: \"fcc8e92d-9145-4558-84d0-023f8428b478\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7"
Sep 11 04:47:35.188048 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:35.188016    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:35.205643 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:35.205618    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcc8e92d-9145-4558-84d0-023f8428b478-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-lfdw7\" (UID: \"fcc8e92d-9145-4558-84d0-023f8428b478\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7"
Sep 11 04:47:35.205799 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:35.205669    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2774\" (UniqueName: \"kubernetes.io/projected/fcc8e92d-9145-4558-84d0-023f8428b478-kube-api-access-h2774\") pod \"kuadrant-operator-controller-manager-5769f99d9f-lfdw7\" (UID: \"fcc8e92d-9145-4558-84d0-023f8428b478\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7"
Sep 11 04:47:35.206035 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:35.206012    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcc8e92d-9145-4558-84d0-023f8428b478-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-lfdw7\" (UID: \"fcc8e92d-9145-4558-84d0-023f8428b478\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7"
Sep 11 04:47:35.215973 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:35.215937    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2774\" (UniqueName: \"kubernetes.io/projected/fcc8e92d-9145-4558-84d0-023f8428b478-kube-api-access-h2774\") pod \"kuadrant-operator-controller-manager-5769f99d9f-lfdw7\" (UID: \"fcc8e92d-9145-4558-84d0-023f8428b478\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7"
Sep 11 04:47:35.292130 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:35.292087    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:35.368031 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:35.367944    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7"
Sep 11 04:47:35.670246 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:35.670219    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7"]
Sep 11 04:47:35.672084 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:47:35.672035    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcc8e92d_9145_4558_84d0_023f8428b478.slice/crio-b451f0e7eb51c580d6868633f893fb6552aa8840fae610748c883525cbadf04c WatchSource:0}: Error finding container b451f0e7eb51c580d6868633f893fb6552aa8840fae610748c883525cbadf04c: Status 404 returned error can't find the container with id b451f0e7eb51c580d6868633f893fb6552aa8840fae610748c883525cbadf04c
Sep 11 04:47:36.147247 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:36.147209    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7" event={"ID":"fcc8e92d-9145-4558-84d0-023f8428b478","Type":"ContainerStarted","Data":"b451f0e7eb51c580d6868633f893fb6552aa8840fae610748c883525cbadf04c"}
Sep 11 04:47:37.591431 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:37.591387    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:37.691785 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:37.691745    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:40.172113 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:40.172061    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7" event={"ID":"fcc8e92d-9145-4558-84d0-023f8428b478","Type":"ContainerStarted","Data":"824e5df02a03ea05b47f045a0e7ec58b1a2144a760d0903f54ec7b16b5bbc793"}
Sep 11 04:47:40.172535 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:40.172316    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7"
Sep 11 04:47:40.192676 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:40.192628    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7" podStartSLOduration=1.338580268 podStartE2EDuration="5.192615552s" totalImagesPullingTime="3.854035284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:47:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:47:40.192316486 +0000 UTC m=+474.663780366" watchObservedRunningTime="2026-09-11 04:47:40.192615552 +0000 UTC m=+474.664079427"
Sep 11 04:47:42.337348 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:42.337306    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:42.438007 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:42.437972    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:47.228156 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:47.228113    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:47.327099 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:47.327019    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:47.426910 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:47.426873    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:51.178570 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:51.178538    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7"
Sep 11 04:47:56.048729 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:56.048693    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:56.144633 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:56.144598    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:58.490509 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:58.490467    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:47:58.548450 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:47:58.548414    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:00.508654 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:00.508585    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:00.558272 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:00.558236    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:03.237732 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:03.237694    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:03.339435 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:03.339399    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:07.537926 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:07.537894    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:07.707785 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:07.707748    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:08.238823 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:08.238766    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:08.292496 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:08.292453    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:08.649526 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:08.649490    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:08.707740 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:08.707707    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:09.041512 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:09.041480    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:09.098568 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:09.098532    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:12.015876 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:12.015836    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:12.105616 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:12.105582    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:12.208737 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:12.208702    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:12.872593 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:12.872559    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:12.989222 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:12.989187    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:17.454564 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:17.454528    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:17.554753 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:17.554721    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:19.912749 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:19.912709    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:19.968434 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:19.968399    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:24.674031 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:24.673988    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:24.778135 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:24.778095    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:26.128053 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:26.128009    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:28.050525 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:28.050477    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:28.148387 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:28.148349    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:28.463057 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:28.462983    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hcp4g"]
Sep 11 04:48:28.467267 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:28.467243    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hcp4g"
Sep 11 04:48:28.470385 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:28.470362    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 04:48:28.471206 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:28.471183    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-927zs\""
Sep 11 04:48:28.471322 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:28.471212    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 11 04:48:28.471322 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:28.471183    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 04:48:28.474015 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:28.473992    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hcp4g"]
Sep 11 04:48:28.494288 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:28.494259    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:28.565633 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:28.565604    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx7r5\" (UniqueName: \"kubernetes.io/projected/e938b91c-8d6a-42ac-8803-8dd2de7002a2-kube-api-access-rx7r5\") pod \"postgres-69fb4594cf-hcp4g\" (UID: \"e938b91c-8d6a-42ac-8803-8dd2de7002a2\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hcp4g"
Sep 11 04:48:28.565842 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:28.565659    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e938b91c-8d6a-42ac-8803-8dd2de7002a2-data\") pod \"postgres-69fb4594cf-hcp4g\" (UID: \"e938b91c-8d6a-42ac-8803-8dd2de7002a2\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hcp4g"
Sep 11 04:48:28.666385 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:28.666353    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx7r5\" (UniqueName: \"kubernetes.io/projected/e938b91c-8d6a-42ac-8803-8dd2de7002a2-kube-api-access-rx7r5\") pod \"postgres-69fb4594cf-hcp4g\" (UID: \"e938b91c-8d6a-42ac-8803-8dd2de7002a2\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hcp4g"
Sep 11 04:48:28.666582 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:28.666410    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e938b91c-8d6a-42ac-8803-8dd2de7002a2-data\") pod \"postgres-69fb4594cf-hcp4g\" (UID: \"e938b91c-8d6a-42ac-8803-8dd2de7002a2\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hcp4g"
Sep 11 04:48:28.666822 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:28.666798    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e938b91c-8d6a-42ac-8803-8dd2de7002a2-data\") pod \"postgres-69fb4594cf-hcp4g\" (UID: \"e938b91c-8d6a-42ac-8803-8dd2de7002a2\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hcp4g"
Sep 11 04:48:28.674474 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:28.674452    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx7r5\" (UniqueName: \"kubernetes.io/projected/e938b91c-8d6a-42ac-8803-8dd2de7002a2-kube-api-access-rx7r5\") pod \"postgres-69fb4594cf-hcp4g\" (UID: \"e938b91c-8d6a-42ac-8803-8dd2de7002a2\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hcp4g"
Sep 11 04:48:28.778718 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:28.778679    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hcp4g"
Sep 11 04:48:29.053982 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:29.053750    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hcp4g"]
Sep 11 04:48:29.056869 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:48:29.056843    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode938b91c_8d6a_42ac_8803_8dd2de7002a2.slice/crio-b922ef68b5cf90a30922b77ced56356790b373199d2ca25316eca612498e719e WatchSource:0}: Error finding container b922ef68b5cf90a30922b77ced56356790b373199d2ca25316eca612498e719e: Status 404 returned error can't find the container with id b922ef68b5cf90a30922b77ced56356790b373199d2ca25316eca612498e719e
Sep 11 04:48:29.406974 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:29.406892    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hcp4g" event={"ID":"e938b91c-8d6a-42ac-8803-8dd2de7002a2","Type":"ContainerStarted","Data":"b922ef68b5cf90a30922b77ced56356790b373199d2ca25316eca612498e719e"}
Sep 11 04:48:34.436875 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:34.436834    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hcp4g" event={"ID":"e938b91c-8d6a-42ac-8803-8dd2de7002a2","Type":"ContainerStarted","Data":"8e4740c69a5fe2b65b013243073a39c5cae75eefcf872aad54a978d508983a22"}
Sep 11 04:48:34.437373 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:34.436964    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hcp4g"
Sep 11 04:48:34.454436 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:34.454383    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hcp4g" podStartSLOduration=1.643322473 podStartE2EDuration="6.454363236s" totalImagesPullingTime="4.811040763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:48:34.453211938 +0000 UTC m=+528.924675813" watchObservedRunningTime="2026-09-11 04:48:34.454363236 +0000 UTC m=+528.925827112"
Sep 11 04:48:38.796835 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:38.796749    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:40.470739 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:40.470713    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hcp4g"
Sep 11 04:48:40.775384 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:40.775354    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:41.172200 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:41.172109    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:47.216638 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:47.216598    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:47.313690 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:47.313652    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:47.408613 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:47.408575    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:47.620892 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:47.620824    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:50.767478 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:50.767437    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:48:50.867471 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:48:50.867419    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:49:24.181369 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:24.181319    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:49:24.291100 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:24.291039    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:49:25.591960 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:25.591927    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:49:25.669283 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:25.669235    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:49:25.819543 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:25.819510    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:49:25.937382 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:25.937306    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:49:26.161351 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:26.161306    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:49:26.249137 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:26.249045    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:49:26.291127 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:26.291091    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-4wnbl"]
Sep 11 04:49:26.294784 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:26.294766    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-4wnbl"
Sep 11 04:49:26.297337 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:26.297314    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-2l6qs\""
Sep 11 04:49:26.297796 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:26.297778    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 11 04:49:26.303803 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:26.303780    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-4wnbl"]
Sep 11 04:49:26.392971 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:26.392938    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d36da1d-c48a-4391-b2eb-5ffc687f1164-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-4wnbl\" (UID: \"9d36da1d-c48a-4391-b2eb-5ffc687f1164\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-4wnbl"
Sep 11 04:49:26.393152 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:26.392980    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbc7r\" (UniqueName: \"kubernetes.io/projected/9d36da1d-c48a-4391-b2eb-5ffc687f1164-kube-api-access-mbc7r\") pod \"payload-processing-5b54b7fbb7-4wnbl\" (UID: \"9d36da1d-c48a-4391-b2eb-5ffc687f1164\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-4wnbl"
Sep 11 04:49:26.494033 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:26.494000    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d36da1d-c48a-4391-b2eb-5ffc687f1164-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-4wnbl\" (UID: \"9d36da1d-c48a-4391-b2eb-5ffc687f1164\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-4wnbl"
Sep 11 04:49:26.494033 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:26.494034    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbc7r\" (UniqueName: \"kubernetes.io/projected/9d36da1d-c48a-4391-b2eb-5ffc687f1164-kube-api-access-mbc7r\") pod \"payload-processing-5b54b7fbb7-4wnbl\" (UID: \"9d36da1d-c48a-4391-b2eb-5ffc687f1164\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-4wnbl"
Sep 11 04:49:26.494805 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:26.494779    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d36da1d-c48a-4391-b2eb-5ffc687f1164-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-4wnbl\" (UID: \"9d36da1d-c48a-4391-b2eb-5ffc687f1164\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-4wnbl"
Sep 11 04:49:26.502566 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:26.502519    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbc7r\" (UniqueName: \"kubernetes.io/projected/9d36da1d-c48a-4391-b2eb-5ffc687f1164-kube-api-access-mbc7r\") pod \"payload-processing-5b54b7fbb7-4wnbl\" (UID: \"9d36da1d-c48a-4391-b2eb-5ffc687f1164\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-4wnbl"
Sep 11 04:49:26.605671 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:26.605635    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-4wnbl"
Sep 11 04:49:26.877948 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:26.877914    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-4wnbl"]
Sep 11 04:49:26.880898 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:49:26.880866    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d36da1d_c48a_4391_b2eb_5ffc687f1164.slice/crio-8229abbad88bca68366893606abc7aeccfca7669c092214448c411228918309d WatchSource:0}: Error finding container 8229abbad88bca68366893606abc7aeccfca7669c092214448c411228918309d: Status 404 returned error can't find the container with id 8229abbad88bca68366893606abc7aeccfca7669c092214448c411228918309d
Sep 11 04:49:27.692441 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:27.692403    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-4wnbl" event={"ID":"9d36da1d-c48a-4391-b2eb-5ffc687f1164","Type":"ContainerStarted","Data":"8229abbad88bca68366893606abc7aeccfca7669c092214448c411228918309d"}
Sep 11 04:49:28.699188 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:28.699096    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-4wnbl" event={"ID":"9d36da1d-c48a-4391-b2eb-5ffc687f1164","Type":"ContainerStarted","Data":"f9853b610d6b4ebcb3f0af830d69c893c80a3ec3c779ae037f4fe5c375a17c12"}
Sep 11 04:49:28.699539 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:28.699272    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-4wnbl"
Sep 11 04:49:28.717104 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:28.717046    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-4wnbl" podStartSLOduration=1.190214395 podStartE2EDuration="2.717031271s" totalImagesPullingTime="1.526816876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:49:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:49:28.715143811 +0000 UTC m=+583.186607689" watchObservedRunningTime="2026-09-11 04:49:28.717031271 +0000 UTC m=+583.188495147"
Sep 11 04:49:36.696506 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:36.696475    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:49:37.365833 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:37.365777    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:49:37.441560 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:37.441518    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:49:39.705880 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:39.705851    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-4wnbl"
Sep 11 04:49:44.489251 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:44.489216    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:49:52.978359 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:52.978309    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:49:56.443149 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.443116    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"]
Sep 11 04:49:56.446730 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.446707    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:49:56.449336 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.449302    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tzcgp\""
Sep 11 04:49:56.449465 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.449344    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 04:49:56.450356 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.450330    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 04:49:56.450661 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.450630    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 11 04:49:56.463337 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.461577    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"]
Sep 11 04:49:56.481280 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.481239    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:49:56.544491 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.544461    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a91f2a51-ac21-46c4-ae8d-b4be35ff9c62-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w\" (UID: \"a91f2a51-ac21-46c4-ae8d-b4be35ff9c62\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:49:56.544491 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.544493    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a91f2a51-ac21-46c4-ae8d-b4be35ff9c62-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w\" (UID: \"a91f2a51-ac21-46c4-ae8d-b4be35ff9c62\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:49:56.544676 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.544527    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a91f2a51-ac21-46c4-ae8d-b4be35ff9c62-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w\" (UID: \"a91f2a51-ac21-46c4-ae8d-b4be35ff9c62\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:49:56.544676 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.544639    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a91f2a51-ac21-46c4-ae8d-b4be35ff9c62-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w\" (UID: \"a91f2a51-ac21-46c4-ae8d-b4be35ff9c62\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:49:56.544782 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.544676    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t28c8\" (UniqueName: \"kubernetes.io/projected/a91f2a51-ac21-46c4-ae8d-b4be35ff9c62-kube-api-access-t28c8\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w\" (UID: \"a91f2a51-ac21-46c4-ae8d-b4be35ff9c62\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:49:56.544782 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.544708    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a91f2a51-ac21-46c4-ae8d-b4be35ff9c62-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w\" (UID: \"a91f2a51-ac21-46c4-ae8d-b4be35ff9c62\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:49:56.645425 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.645394    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a91f2a51-ac21-46c4-ae8d-b4be35ff9c62-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w\" (UID: \"a91f2a51-ac21-46c4-ae8d-b4be35ff9c62\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:49:56.645425 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.645429    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t28c8\" (UniqueName: \"kubernetes.io/projected/a91f2a51-ac21-46c4-ae8d-b4be35ff9c62-kube-api-access-t28c8\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w\" (UID: \"a91f2a51-ac21-46c4-ae8d-b4be35ff9c62\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:49:56.645652 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.645451    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a91f2a51-ac21-46c4-ae8d-b4be35ff9c62-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w\" (UID: \"a91f2a51-ac21-46c4-ae8d-b4be35ff9c62\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:49:56.645652 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.645513    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a91f2a51-ac21-46c4-ae8d-b4be35ff9c62-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w\" (UID: \"a91f2a51-ac21-46c4-ae8d-b4be35ff9c62\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:49:56.645652 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.645527    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a91f2a51-ac21-46c4-ae8d-b4be35ff9c62-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w\" (UID: \"a91f2a51-ac21-46c4-ae8d-b4be35ff9c62\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:49:56.645652 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.645570    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a91f2a51-ac21-46c4-ae8d-b4be35ff9c62-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w\" (UID: \"a91f2a51-ac21-46c4-ae8d-b4be35ff9c62\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:49:56.645887 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.645863    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a91f2a51-ac21-46c4-ae8d-b4be35ff9c62-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w\" (UID: \"a91f2a51-ac21-46c4-ae8d-b4be35ff9c62\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:49:56.645925 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.645873    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a91f2a51-ac21-46c4-ae8d-b4be35ff9c62-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w\" (UID: \"a91f2a51-ac21-46c4-ae8d-b4be35ff9c62\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:49:56.645965 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.645922    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a91f2a51-ac21-46c4-ae8d-b4be35ff9c62-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w\" (UID: \"a91f2a51-ac21-46c4-ae8d-b4be35ff9c62\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:49:56.648083 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.648047    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a91f2a51-ac21-46c4-ae8d-b4be35ff9c62-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w\" (UID: \"a91f2a51-ac21-46c4-ae8d-b4be35ff9c62\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:49:56.648308 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.648287    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a91f2a51-ac21-46c4-ae8d-b4be35ff9c62-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w\" (UID: \"a91f2a51-ac21-46c4-ae8d-b4be35ff9c62\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:49:56.653625 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.653594    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t28c8\" (UniqueName: \"kubernetes.io/projected/a91f2a51-ac21-46c4-ae8d-b4be35ff9c62-kube-api-access-t28c8\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w\" (UID: \"a91f2a51-ac21-46c4-ae8d-b4be35ff9c62\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:49:56.758510 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:56.758489    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:49:57.029028 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:57.029003    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"]
Sep 11 04:49:57.031372 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:49:57.031342    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda91f2a51_ac21_46c4_ae8d_b4be35ff9c62.slice/crio-80c19b2a8306a3710553eaffc619c878ac6a0571a0ebc991c428bfa889cae8c4 WatchSource:0}: Error finding container 80c19b2a8306a3710553eaffc619c878ac6a0571a0ebc991c428bfa889cae8c4: Status 404 returned error can't find the container with id 80c19b2a8306a3710553eaffc619c878ac6a0571a0ebc991c428bfa889cae8c4
Sep 11 04:49:57.839757 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:57.839719    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w" event={"ID":"a91f2a51-ac21-46c4-ae8d-b4be35ff9c62","Type":"ContainerStarted","Data":"80c19b2a8306a3710553eaffc619c878ac6a0571a0ebc991c428bfa889cae8c4"}
Sep 11 04:49:58.583686 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:58.583643    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:49:58.846056 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:58.845961    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w" event={"ID":"a91f2a51-ac21-46c4-ae8d-b4be35ff9c62","Type":"ContainerStarted","Data":"29229f8c74624dc97b0889aa63b1317aa2796784351e76e90369a21764b3c0e5"}
Sep 11 04:49:58.865209 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:49:58.865164    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w" podStartSLOduration=1.890040159 podStartE2EDuration="2.865148006s" totalImagesPullingTime="975.107847ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:49:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:49:58.8629199 +0000 UTC m=+613.334383812" watchObservedRunningTime="2026-09-11 04:49:58.865148006 +0000 UTC m=+613.336611881"
Sep 11 04:50:01.038004 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:50:01.037975    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:50:01.136460 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:50:01.136424    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:50:02.577491 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:50:02.577459    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:50:06.758915 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:50:06.758882    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:50:06.759348 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:50:06.758941    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:50:06.773312 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:50:06.773287    2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:50:06.896138 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:50:06.896051    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w"
Sep 11 04:50:16.379729 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:50:16.379683    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:50:29.490301 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:50:29.490260    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:50:40.084954 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:50:40.084920    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:51:48.450870 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:51:48.450786    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:51:48.501794 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:51:48.501763    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:09.716515 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:09.716479    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:09.816169 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:09.816126    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:10.810691 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:10.810654    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:10.911167 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:10.911125    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:25.711707 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:25.711668    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:25.823575 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:25.823542    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:26.038110 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:26.038050    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:26.146734 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:26.146694    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:27.018144 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:27.018109    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:27.118461 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:27.118429    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:27.454624 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:27.454537    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:29.190940 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:29.190899    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:29.292537 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:29.292509    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:30.474583 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:30.474519    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:30.575337 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:30.575299    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:30.797693 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:30.797663    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:30.932928 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:30.932898    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:31.022876 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.022841    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:31.023213 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.023194    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2"]
Sep 11 04:52:31.026705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.026688    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2"
Sep 11 04:52:31.029341 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.029320    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-831a1c\""
Sep 11 04:52:31.029553 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.029538    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-831a1c-dockercfg-n2wtb\""
Sep 11 04:52:31.034842 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.034821    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2"]
Sep 11 04:52:31.065754 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.065663    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh"]
Sep 11 04:52:31.069134 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.069114    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh"
Sep 11 04:52:31.081879 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.081858    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh"]
Sep 11 04:52:31.120361 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.120315    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6b49\" (UniqueName: \"kubernetes.io/projected/1b68a707-52e2-4f0e-9397-beb687c48d53-kube-api-access-b6b49\") pod \"payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh\" (UID: \"1b68a707-52e2-4f0e-9397-beb687c48d53\") " pod="openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh"
Sep 11 04:52:31.120562 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.120400    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/753bd98a-575b-4db6-ac01-25023748e452-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2\" (UID: \"753bd98a-575b-4db6-ac01-25023748e452\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2"
Sep 11 04:52:31.120562 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.120507    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d45j9\" (UniqueName: \"kubernetes.io/projected/753bd98a-575b-4db6-ac01-25023748e452-kube-api-access-d45j9\") pod \"payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2\" (UID: \"753bd98a-575b-4db6-ac01-25023748e452\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2"
Sep 11 04:52:31.120751 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.120661    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b68a707-52e2-4f0e-9397-beb687c48d53-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh\" (UID: \"1b68a707-52e2-4f0e-9397-beb687c48d53\") " pod="openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh"
Sep 11 04:52:31.221471 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.221435    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6b49\" (UniqueName: \"kubernetes.io/projected/1b68a707-52e2-4f0e-9397-beb687c48d53-kube-api-access-b6b49\") pod \"payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh\" (UID: \"1b68a707-52e2-4f0e-9397-beb687c48d53\") " pod="openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh"
Sep 11 04:52:31.221471 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.221475    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/753bd98a-575b-4db6-ac01-25023748e452-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2\" (UID: \"753bd98a-575b-4db6-ac01-25023748e452\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2"
Sep 11 04:52:31.221731 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.221520    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d45j9\" (UniqueName: \"kubernetes.io/projected/753bd98a-575b-4db6-ac01-25023748e452-kube-api-access-d45j9\") pod \"payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2\" (UID: \"753bd98a-575b-4db6-ac01-25023748e452\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2"
Sep 11 04:52:31.221731 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.221596    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b68a707-52e2-4f0e-9397-beb687c48d53-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh\" (UID: \"1b68a707-52e2-4f0e-9397-beb687c48d53\") " pod="openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh"
Sep 11 04:52:31.222518 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.222495    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b68a707-52e2-4f0e-9397-beb687c48d53-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh\" (UID: \"1b68a707-52e2-4f0e-9397-beb687c48d53\") " pod="openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh"
Sep 11 04:52:31.222593 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.222493    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/753bd98a-575b-4db6-ac01-25023748e452-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2\" (UID: \"753bd98a-575b-4db6-ac01-25023748e452\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2"
Sep 11 04:52:31.229891 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.229860    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6b49\" (UniqueName: \"kubernetes.io/projected/1b68a707-52e2-4f0e-9397-beb687c48d53-kube-api-access-b6b49\") pod \"payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh\" (UID: \"1b68a707-52e2-4f0e-9397-beb687c48d53\") " pod="openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh"
Sep 11 04:52:31.230093 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.230047    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d45j9\" (UniqueName: \"kubernetes.io/projected/753bd98a-575b-4db6-ac01-25023748e452-kube-api-access-d45j9\") pod \"payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2\" (UID: \"753bd98a-575b-4db6-ac01-25023748e452\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2"
Sep 11 04:52:31.338455 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.338367    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2"
Sep 11 04:52:31.379313 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.379283    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh"
Sep 11 04:52:31.615001 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.614973    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2"]
Sep 11 04:52:31.616990 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:52:31.616969    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod753bd98a_575b_4db6_ac01_25023748e452.slice/crio-cbf616de30e037b8b9068641f88357ad914a33dac8ae5739413fef3cbe3dcc35 WatchSource:0}: Error finding container cbf616de30e037b8b9068641f88357ad914a33dac8ae5739413fef3cbe3dcc35: Status 404 returned error can't find the container with id cbf616de30e037b8b9068641f88357ad914a33dac8ae5739413fef3cbe3dcc35
Sep 11 04:52:31.663606 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:31.663576    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh"]
Sep 11 04:52:31.665844 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:52:31.665807    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b68a707_52e2_4f0e_9397_beb687c48d53.slice/crio-27bf40e28dfecf91900936acaa0a5bcaa5444a6d6952eb9bd52973b6e4b6c0a4 WatchSource:0}: Error finding container 27bf40e28dfecf91900936acaa0a5bcaa5444a6d6952eb9bd52973b6e4b6c0a4: Status 404 returned error can't find the container with id 27bf40e28dfecf91900936acaa0a5bcaa5444a6d6952eb9bd52973b6e4b6c0a4
Sep 11 04:52:32.592821 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:32.592781    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh" event={"ID":"1b68a707-52e2-4f0e-9397-beb687c48d53","Type":"ContainerStarted","Data":"c19754a06097df750add785e9d9ee6016da6ffdb14a60d3ebfa97a049e0ccf8c"}
Sep 11 04:52:32.593020 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:32.592843    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh" event={"ID":"1b68a707-52e2-4f0e-9397-beb687c48d53","Type":"ContainerStarted","Data":"27bf40e28dfecf91900936acaa0a5bcaa5444a6d6952eb9bd52973b6e4b6c0a4"}
Sep 11 04:52:32.593020 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:32.592946    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh"
Sep 11 04:52:32.594453 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:32.594427    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2" event={"ID":"753bd98a-575b-4db6-ac01-25023748e452","Type":"ContainerStarted","Data":"d53fe0b446ed30585592a1ede9bdbf6abef6bb2d0c32172f161edcf885fcb1bc"}
Sep 11 04:52:32.594586 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:32.594470    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2" event={"ID":"753bd98a-575b-4db6-ac01-25023748e452","Type":"ContainerStarted","Data":"cbf616de30e037b8b9068641f88357ad914a33dac8ae5739413fef3cbe3dcc35"}
Sep 11 04:52:32.594586 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:32.594493    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2"
Sep 11 04:52:32.611270 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:32.611229    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh" podStartSLOduration=1.6112157759999999 podStartE2EDuration="1.611215776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:52:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:52:32.609517504 +0000 UTC m=+767.080981371" watchObservedRunningTime="2026-09-11 04:52:32.611215776 +0000 UTC m=+767.082679652"
Sep 11 04:52:32.624249 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:32.624211    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2" podStartSLOduration=1.624198386 podStartE2EDuration="1.624198386s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:52:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:52:32.623498991 +0000 UTC m=+767.094962860" watchObservedRunningTime="2026-09-11 04:52:32.624198386 +0000 UTC m=+767.095662261"
Sep 11 04:52:33.289879 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:33.289845    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:33.390668 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:33.390636    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:33.501175 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:33.501142    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:33.794902 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:33.794869    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:34.035668 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:34.035632    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:34.801992 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:34.801958    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:35.117746 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:35.117651    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:35.715636 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:35.715602    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:36.678568 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:36.678536    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:36.772784 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:36.772748    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:37.926660 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:37.926625    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:38.024544 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.024508    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:38.123568 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.123532    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:38.416427 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.416368    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:38.510536 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.510505    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:38.545374 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.545338    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59"]
Sep 11 04:52:38.549062 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.549046    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59"
Sep 11 04:52:38.551458 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.551431    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-5fa7070e\""
Sep 11 04:52:38.551543 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.551498    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-5fa7070e-dockercfg-bmfc6\""
Sep 11 04:52:38.559493 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.559473    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59"]
Sep 11 04:52:38.604745 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.604714    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f"]
Sep 11 04:52:38.608293 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.608278    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f"
Sep 11 04:52:38.618773 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.618750    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f"]
Sep 11 04:52:38.694053 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.693972    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f\" (UID: \"6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f"
Sep 11 04:52:38.694053 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.694042    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdd4z\" (UniqueName: \"kubernetes.io/projected/f25b1e0b-6f41-41dd-9daf-404a1dc3073c-kube-api-access-xdd4z\") pod \"payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59\" (UID: \"f25b1e0b-6f41-41dd-9daf-404a1dc3073c\") " pod="openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59"
Sep 11 04:52:38.694268 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.694134    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k72n5\" (UniqueName: \"kubernetes.io/projected/6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2-kube-api-access-k72n5\") pod \"payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f\" (UID: \"6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f"
Sep 11 04:52:38.694268 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.694158    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f25b1e0b-6f41-41dd-9daf-404a1dc3073c-plugins-config-volume\") pod \"payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59\" (UID: \"f25b1e0b-6f41-41dd-9daf-404a1dc3073c\") " pod="openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59"
Sep 11 04:52:38.795453 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.795417    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdd4z\" (UniqueName: \"kubernetes.io/projected/f25b1e0b-6f41-41dd-9daf-404a1dc3073c-kube-api-access-xdd4z\") pod \"payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59\" (UID: \"f25b1e0b-6f41-41dd-9daf-404a1dc3073c\") " pod="openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59"
Sep 11 04:52:38.795627 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.795476    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k72n5\" (UniqueName: \"kubernetes.io/projected/6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2-kube-api-access-k72n5\") pod \"payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f\" (UID: \"6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f"
Sep 11 04:52:38.795627 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.795500    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f25b1e0b-6f41-41dd-9daf-404a1dc3073c-plugins-config-volume\") pod \"payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59\" (UID: \"f25b1e0b-6f41-41dd-9daf-404a1dc3073c\") " pod="openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59"
Sep 11 04:52:38.795627 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.795548    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f\" (UID: \"6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f"
Sep 11 04:52:38.796447 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.796423    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f\" (UID: \"6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f"
Sep 11 04:52:38.796492 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.796423    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f25b1e0b-6f41-41dd-9daf-404a1dc3073c-plugins-config-volume\") pod \"payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59\" (UID: \"f25b1e0b-6f41-41dd-9daf-404a1dc3073c\") " pod="openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59"
Sep 11 04:52:38.803701 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.803676    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdd4z\" (UniqueName: \"kubernetes.io/projected/f25b1e0b-6f41-41dd-9daf-404a1dc3073c-kube-api-access-xdd4z\") pod \"payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59\" (UID: \"f25b1e0b-6f41-41dd-9daf-404a1dc3073c\") " pod="openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59"
Sep 11 04:52:38.803834 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.803797    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k72n5\" (UniqueName: \"kubernetes.io/projected/6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2-kube-api-access-k72n5\") pod \"payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f\" (UID: \"6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f"
Sep 11 04:52:38.861982 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.861952    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59"
Sep 11 04:52:38.918572 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.918544    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f"
Sep 11 04:52:38.988259 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:38.988231    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:39.065540 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:39.065512    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:39.141625 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:39.141601    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59"]
Sep 11 04:52:39.143086 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:52:39.143041    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf25b1e0b_6f41_41dd_9daf_404a1dc3073c.slice/crio-ba8daa61e890d2a4628c908fb8605fba948e067e3ff14a3617536ebd5a8caaf4 WatchSource:0}: Error finding container ba8daa61e890d2a4628c908fb8605fba948e067e3ff14a3617536ebd5a8caaf4: Status 404 returned error can't find the container with id ba8daa61e890d2a4628c908fb8605fba948e067e3ff14a3617536ebd5a8caaf4
Sep 11 04:52:39.197464 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:39.197337    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f"]
Sep 11 04:52:39.199514 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:52:39.199492    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ee10272_cd1a_4593_ae9a_b1ca67c6a1c2.slice/crio-7d6c96056f61ab3d730af9ffa51b196233a8f7567e9f147681f356c1c5b0e44d WatchSource:0}: Error finding container 7d6c96056f61ab3d730af9ffa51b196233a8f7567e9f147681f356c1c5b0e44d: Status 404 returned error can't find the container with id 7d6c96056f61ab3d730af9ffa51b196233a8f7567e9f147681f356c1c5b0e44d
Sep 11 04:52:39.272136 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:39.272090    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:39.630799 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:39.630741    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59" event={"ID":"f25b1e0b-6f41-41dd-9daf-404a1dc3073c","Type":"ContainerStarted","Data":"19c4bc394e74e5bbdddb4f2ea29a6233deef597f805bc0b03b6e5a07ef932d2b"}
Sep 11 04:52:39.630799 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:39.630800    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59" event={"ID":"f25b1e0b-6f41-41dd-9daf-404a1dc3073c","Type":"ContainerStarted","Data":"ba8daa61e890d2a4628c908fb8605fba948e067e3ff14a3617536ebd5a8caaf4"}
Sep 11 04:52:39.631107 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:39.630862    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59"
Sep 11 04:52:39.632851 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:39.632828    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f" event={"ID":"6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2","Type":"ContainerStarted","Data":"03291c063ccd9a333229f7ed6d0eab03087873078c4aaa37ee7a8bc125dd3fae"}
Sep 11 04:52:39.632962 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:39.632865    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f" event={"ID":"6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2","Type":"ContainerStarted","Data":"7d6c96056f61ab3d730af9ffa51b196233a8f7567e9f147681f356c1c5b0e44d"}
Sep 11 04:52:39.632962 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:39.632900    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f"
Sep 11 04:52:39.649454 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:39.649410    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59" podStartSLOduration=1.6493964129999998 podStartE2EDuration="1.649396413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:52:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:52:39.647833436 +0000 UTC m=+774.119297313" watchObservedRunningTime="2026-09-11 04:52:39.649396413 +0000 UTC m=+774.120860289"
Sep 11 04:52:40.245693 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:40.245641    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:40.384922 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:40.384887    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:40.873734 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:40.873696    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:40.967574 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:40.967539    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:41.007973 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.007921    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f" podStartSLOduration=3.007904529 podStartE2EDuration="3.007904529s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:52:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:52:39.664363087 +0000 UTC m=+774.135826963" watchObservedRunningTime="2026-09-11 04:52:41.007904529 +0000 UTC m=+775.479368399"
Sep 11 04:52:41.008217 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.008199    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4"]
Sep 11 04:52:41.011724 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.011710    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4"
Sep 11 04:52:41.014092 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.014059    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-0a8fef\""
Sep 11 04:52:41.014136 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.014108    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-0a8fef-dockercfg-spp7n\""
Sep 11 04:52:41.024029 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.024006    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4"]
Sep 11 04:52:41.029401 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.029375    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh"]
Sep 11 04:52:41.033501 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.033483    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh"
Sep 11 04:52:41.056227 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.056195    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh"]
Sep 11 04:52:41.067448 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.067419    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:41.117517 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.117482    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad24ea67-e38e-4809-89d8-bc6115095036-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4\" (UID: \"ad24ea67-e38e-4809-89d8-bc6115095036\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4"
Sep 11 04:52:41.117701 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.117558    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2vjt\" (UniqueName: \"kubernetes.io/projected/ad24ea67-e38e-4809-89d8-bc6115095036-kube-api-access-l2vjt\") pod \"payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4\" (UID: \"ad24ea67-e38e-4809-89d8-bc6115095036\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4"
Sep 11 04:52:41.117701 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.117650    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5a32b74-703a-4480-b368-425d3afd5a18-plugins-config-volume\") pod \"payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh\" (UID: \"b5a32b74-703a-4480-b368-425d3afd5a18\") " pod="openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh"
Sep 11 04:52:41.117701 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.117678    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdq2n\" (UniqueName: \"kubernetes.io/projected/b5a32b74-703a-4480-b368-425d3afd5a18-kube-api-access-fdq2n\") pod \"payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh\" (UID: \"b5a32b74-703a-4480-b368-425d3afd5a18\") " pod="openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh"
Sep 11 04:52:41.218905 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.218815    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad24ea67-e38e-4809-89d8-bc6115095036-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4\" (UID: \"ad24ea67-e38e-4809-89d8-bc6115095036\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4"
Sep 11 04:52:41.218905 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.218862    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2vjt\" (UniqueName: \"kubernetes.io/projected/ad24ea67-e38e-4809-89d8-bc6115095036-kube-api-access-l2vjt\") pod \"payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4\" (UID: \"ad24ea67-e38e-4809-89d8-bc6115095036\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4"
Sep 11 04:52:41.219163 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.218913    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5a32b74-703a-4480-b368-425d3afd5a18-plugins-config-volume\") pod \"payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh\" (UID: \"b5a32b74-703a-4480-b368-425d3afd5a18\") " pod="openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh"
Sep 11 04:52:41.219163 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.218936    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdq2n\" (UniqueName: \"kubernetes.io/projected/b5a32b74-703a-4480-b368-425d3afd5a18-kube-api-access-fdq2n\") pod \"payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh\" (UID: \"b5a32b74-703a-4480-b368-425d3afd5a18\") " pod="openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh"
Sep 11 04:52:41.219777 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.219754    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5a32b74-703a-4480-b368-425d3afd5a18-plugins-config-volume\") pod \"payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh\" (UID: \"b5a32b74-703a-4480-b368-425d3afd5a18\") " pod="openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh"
Sep 11 04:52:41.219823 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.219754    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad24ea67-e38e-4809-89d8-bc6115095036-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4\" (UID: \"ad24ea67-e38e-4809-89d8-bc6115095036\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4"
Sep 11 04:52:41.227385 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.227354    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2vjt\" (UniqueName: \"kubernetes.io/projected/ad24ea67-e38e-4809-89d8-bc6115095036-kube-api-access-l2vjt\") pod \"payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4\" (UID: \"ad24ea67-e38e-4809-89d8-bc6115095036\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4"
Sep 11 04:52:41.227497 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.227431    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdq2n\" (UniqueName: \"kubernetes.io/projected/b5a32b74-703a-4480-b368-425d3afd5a18-kube-api-access-fdq2n\") pod \"payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh\" (UID: \"b5a32b74-703a-4480-b368-425d3afd5a18\") " pod="openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh"
Sep 11 04:52:41.322636 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.322600    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4"
Sep 11 04:52:41.343893 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.343867    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh"
Sep 11 04:52:41.603691 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.603664    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4"]
Sep 11 04:52:41.605927 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:52:41.605900    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad24ea67_e38e_4809_89d8_bc6115095036.slice/crio-da9c1d24eda82f48abaea98cdcab9d065538664d664481252b7e808b94291d08 WatchSource:0}: Error finding container da9c1d24eda82f48abaea98cdcab9d065538664d664481252b7e808b94291d08: Status 404 returned error can't find the container with id da9c1d24eda82f48abaea98cdcab9d065538664d664481252b7e808b94291d08
Sep 11 04:52:41.627750 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.627726    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh"]
Sep 11 04:52:41.640276 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:52:41.640250    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5a32b74_703a_4480_b368_425d3afd5a18.slice/crio-c0bcda7b49cd9936d2a1a0cc8f3d6f8b3dbe5ed248347f4f9f6d9202041e4f81 WatchSource:0}: Error finding container c0bcda7b49cd9936d2a1a0cc8f3d6f8b3dbe5ed248347f4f9f6d9202041e4f81: Status 404 returned error can't find the container with id c0bcda7b49cd9936d2a1a0cc8f3d6f8b3dbe5ed248347f4f9f6d9202041e4f81
Sep 11 04:52:41.646763 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:41.646733    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4" event={"ID":"ad24ea67-e38e-4809-89d8-bc6115095036","Type":"ContainerStarted","Data":"da9c1d24eda82f48abaea98cdcab9d065538664d664481252b7e808b94291d08"}
Sep 11 04:52:42.655194 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:42.655147    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4" event={"ID":"ad24ea67-e38e-4809-89d8-bc6115095036","Type":"ContainerStarted","Data":"aa00bb670a3c6f1229d4404228170ab4a40e265b1fc7102ebf9e4c5c93edfde1"}
Sep 11 04:52:42.655661 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:42.655235    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4"
Sep 11 04:52:42.656970 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:42.656942    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh" event={"ID":"b5a32b74-703a-4480-b368-425d3afd5a18","Type":"ContainerStarted","Data":"f0809f22688f2dd0db904378b280298e2b03c95d5c20b9db13d2d98670cd02ca"}
Sep 11 04:52:42.657136 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:42.656991    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh" event={"ID":"b5a32b74-703a-4480-b368-425d3afd5a18","Type":"ContainerStarted","Data":"c0bcda7b49cd9936d2a1a0cc8f3d6f8b3dbe5ed248347f4f9f6d9202041e4f81"}
Sep 11 04:52:42.657136 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:42.657083    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh"
Sep 11 04:52:42.671627 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:42.671581    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4" podStartSLOduration=2.6715680219999998 podStartE2EDuration="2.671568022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:52:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:52:42.66938596 +0000 UTC m=+777.140849836" watchObservedRunningTime="2026-09-11 04:52:42.671568022 +0000 UTC m=+777.143031943"
Sep 11 04:52:42.685907 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:42.685861    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh" podStartSLOduration=1.685847387 podStartE2EDuration="1.685847387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:52:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:52:42.683333906 +0000 UTC m=+777.154797793" watchObservedRunningTime="2026-09-11 04:52:42.685847387 +0000 UTC m=+777.157311263"
Sep 11 04:52:43.601215 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:43.601182    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2"
Sep 11 04:52:43.601418 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:43.601267    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh"
Sep 11 04:52:44.279593 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:44.279555    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:44.430547 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:44.430500    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:44.532460 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:44.532377    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:44.629428 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:44.629391    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:47.267535 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:47.267499    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:48.889746 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:48.889710    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:50.642146 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:50.642111    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f"
Sep 11 04:52:50.642538 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:50.642193    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59"
Sep 11 04:52:52.390390 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:52.390355    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:53.665900 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:53.665869    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh"
Sep 11 04:52:53.666350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:53.665961    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4"
Sep 11 04:52:54.079743 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:54.079708    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:54.404240 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:54.404158    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:54.695316 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:54.695235    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:55.028024 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:55.027985    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:57.277924 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:57.277882    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:57.377844 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:57.377810    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:57.489745 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:57.489702    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:57.842442 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:57.842406    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:57.940212 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:57.940169    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:58.870931 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:58.870900    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:52:59.967322 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:52:59.967284    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:00.070517 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.070474    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:00.404860 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.404825    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:00.503935 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.503900    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:00.554992 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.554917    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6"]
Sep 11 04:53:00.560388 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.560367    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6"
Sep 11 04:53:00.563048 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.563022    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-7bdb3c-dockercfg-knnfj\""
Sep 11 04:53:00.563190 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.563092    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-7bdb3c\""
Sep 11 04:53:00.563368 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.563349    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq"]
Sep 11 04:53:00.566906 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.566856    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq"
Sep 11 04:53:00.567943 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.567921    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6"]
Sep 11 04:53:00.580274 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.580247    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq"]
Sep 11 04:53:00.581942 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.581909    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:00.601984 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.601951    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76435f19-2fb3-48ff-ae00-b3488ea0323d-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6\" (UID: \"76435f19-2fb3-48ff-ae00-b3488ea0323d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6"
Sep 11 04:53:00.602180 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.602060    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6c6ce18-02b2-4983-81b6-c4ab71d15504-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq\" (UID: \"c6c6ce18-02b2-4983-81b6-c4ab71d15504\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq"
Sep 11 04:53:00.602180 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.602111    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qq8gs\" (UniqueName: \"kubernetes.io/projected/c6c6ce18-02b2-4983-81b6-c4ab71d15504-kube-api-access-qq8gs\") pod \"payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq\" (UID: \"c6c6ce18-02b2-4983-81b6-c4ab71d15504\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq"
Sep 11 04:53:00.602319 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.602234    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngc7t\" (UniqueName: \"kubernetes.io/projected/76435f19-2fb3-48ff-ae00-b3488ea0323d-kube-api-access-ngc7t\") pod \"payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6\" (UID: \"76435f19-2fb3-48ff-ae00-b3488ea0323d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6"
Sep 11 04:53:00.703615 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.703525    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngc7t\" (UniqueName: \"kubernetes.io/projected/76435f19-2fb3-48ff-ae00-b3488ea0323d-kube-api-access-ngc7t\") pod \"payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6\" (UID: \"76435f19-2fb3-48ff-ae00-b3488ea0323d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6"
Sep 11 04:53:00.703615 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.703592    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76435f19-2fb3-48ff-ae00-b3488ea0323d-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6\" (UID: \"76435f19-2fb3-48ff-ae00-b3488ea0323d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6"
Sep 11 04:53:00.703843 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.703686    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6c6ce18-02b2-4983-81b6-c4ab71d15504-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq\" (UID: \"c6c6ce18-02b2-4983-81b6-c4ab71d15504\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq"
Sep 11 04:53:00.703843 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.703715    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qq8gs\" (UniqueName: \"kubernetes.io/projected/c6c6ce18-02b2-4983-81b6-c4ab71d15504-kube-api-access-qq8gs\") pod \"payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq\" (UID: \"c6c6ce18-02b2-4983-81b6-c4ab71d15504\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq"
Sep 11 04:53:00.704594 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.704569    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76435f19-2fb3-48ff-ae00-b3488ea0323d-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6\" (UID: \"76435f19-2fb3-48ff-ae00-b3488ea0323d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6"
Sep 11 04:53:00.704673 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.704573    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6c6ce18-02b2-4983-81b6-c4ab71d15504-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq\" (UID: \"c6c6ce18-02b2-4983-81b6-c4ab71d15504\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq"
Sep 11 04:53:00.712421 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.712391    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngc7t\" (UniqueName: \"kubernetes.io/projected/76435f19-2fb3-48ff-ae00-b3488ea0323d-kube-api-access-ngc7t\") pod \"payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6\" (UID: \"76435f19-2fb3-48ff-ae00-b3488ea0323d\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6"
Sep 11 04:53:00.712544 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.712526    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qq8gs\" (UniqueName: \"kubernetes.io/projected/c6c6ce18-02b2-4983-81b6-c4ab71d15504-kube-api-access-qq8gs\") pod \"payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq\" (UID: \"c6c6ce18-02b2-4983-81b6-c4ab71d15504\") " pod="openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq"
Sep 11 04:53:00.874229 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.874195    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6"
Sep 11 04:53:00.881228 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.881207    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq"
Sep 11 04:53:00.897967 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:00.897937    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:01.168658 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:01.168627    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6"]
Sep 11 04:53:01.170006 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:53:01.169980    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76435f19_2fb3_48ff_ae00_b3488ea0323d.slice/crio-6c45c076d40c6470ad4a06d98dbacf4f73b907daeefdbb8c63ed48ac20498a10 WatchSource:0}: Error finding container 6c45c076d40c6470ad4a06d98dbacf4f73b907daeefdbb8c63ed48ac20498a10: Status 404 returned error can't find the container with id 6c45c076d40c6470ad4a06d98dbacf4f73b907daeefdbb8c63ed48ac20498a10
Sep 11 04:53:01.177313 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:01.177283    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq"]
Sep 11 04:53:01.178032 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:53:01.178002    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6c6ce18_02b2_4983_81b6_c4ab71d15504.slice/crio-6709194aec3e71268cdb3f4c073deb00a4fabd61848ca8e1fee7960067f4ab49 WatchSource:0}: Error finding container 6709194aec3e71268cdb3f4c073deb00a4fabd61848ca8e1fee7960067f4ab49: Status 404 returned error can't find the container with id 6709194aec3e71268cdb3f4c073deb00a4fabd61848ca8e1fee7960067f4ab49
Sep 11 04:53:01.761016 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:01.760968    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6" event={"ID":"76435f19-2fb3-48ff-ae00-b3488ea0323d","Type":"ContainerStarted","Data":"733a8b64a74211a1fa48eb4b42d2a843ff0f3fa43cc99196d1d665420eba1bf3"}
Sep 11 04:53:01.761258 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:01.761030    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6" event={"ID":"76435f19-2fb3-48ff-ae00-b3488ea0323d","Type":"ContainerStarted","Data":"6c45c076d40c6470ad4a06d98dbacf4f73b907daeefdbb8c63ed48ac20498a10"}
Sep 11 04:53:01.761258 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:01.761221    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6"
Sep 11 04:53:01.762892 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:01.762868    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq" event={"ID":"c6c6ce18-02b2-4983-81b6-c4ab71d15504","Type":"ContainerStarted","Data":"809b761ab3e8df63a182a057c6b64966c69fb63153e827bc51838b5bf6476685"}
Sep 11 04:53:01.763033 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:01.762910    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq"
Sep 11 04:53:01.763033 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:01.762925    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq" event={"ID":"c6c6ce18-02b2-4983-81b6-c4ab71d15504","Type":"ContainerStarted","Data":"6709194aec3e71268cdb3f4c073deb00a4fabd61848ca8e1fee7960067f4ab49"}
Sep 11 04:53:01.764028 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:01.764003    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:01.778333 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:01.778280    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6" podStartSLOduration=1.778262408 podStartE2EDuration="1.778262408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:53:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:53:01.776411378 +0000 UTC m=+796.247875257" watchObservedRunningTime="2026-09-11 04:53:01.778262408 +0000 UTC m=+796.249726286"
Sep 11 04:53:01.794021 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:01.793982    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq" podStartSLOduration=1.793971438 podStartE2EDuration="1.793971438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:53:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:53:01.791558665 +0000 UTC m=+796.263022542" watchObservedRunningTime="2026-09-11 04:53:01.793971438 +0000 UTC m=+796.265435313"
Sep 11 04:53:02.401871 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:02.401832    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:03.259013 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:03.258976    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:03.621247 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:03.621156    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:03.729116 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:03.729046    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:03.817167 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:03.817136    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:03.955644 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:03.955554    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:04.059427 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:04.059390    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:09.872667 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:09.872580    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:09.973182 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:09.973147    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:12.775555 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:12.775520    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq"
Sep 11 04:53:12.776148 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:12.775702    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6"
Sep 11 04:53:17.978787 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:17.978748    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:18.623628 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:18.623592    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:18.645203 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:18.645171    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59"]
Sep 11 04:53:18.645547 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:18.645519    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59" podUID="f25b1e0b-6f41-41dd-9daf-404a1dc3073c" containerName="payload-processing" containerID="cri-o://19c4bc394e74e5bbdddb4f2ea29a6233deef597f805bc0b03b6e5a07ef932d2b" gracePeriod=30
Sep 11 04:53:18.656412 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:18.656109    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f"]
Sep 11 04:53:18.656412 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:18.656305    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f" podUID="6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2" containerName="payload-pre-processing" containerID="cri-o://03291c063ccd9a333229f7ed6d0eab03087873078c4aaa37ee7a8bc125dd3fae" gracePeriod=30
Sep 11 04:53:18.723745 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:18.723712    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:18.803925 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:18.803806    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:18.860552 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:18.860501    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:18.867406 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:18.867373    2567 generic.go:356] "Generic (PLEG): container finished" podID="f25b1e0b-6f41-41dd-9daf-404a1dc3073c" containerID="19c4bc394e74e5bbdddb4f2ea29a6233deef597f805bc0b03b6e5a07ef932d2b" exitCode=0
Sep 11 04:53:18.867552 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:18.867468    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59" event={"ID":"f25b1e0b-6f41-41dd-9daf-404a1dc3073c","Type":"ContainerDied","Data":"19c4bc394e74e5bbdddb4f2ea29a6233deef597f805bc0b03b6e5a07ef932d2b"}
Sep 11 04:53:18.869686 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:18.869657    2567 generic.go:356] "Generic (PLEG): container finished" podID="6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2" containerID="03291c063ccd9a333229f7ed6d0eab03087873078c4aaa37ee7a8bc125dd3fae" exitCode=0
Sep 11 04:53:18.869686 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:18.869684    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f" event={"ID":"6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2","Type":"ContainerDied","Data":"03291c063ccd9a333229f7ed6d0eab03087873078c4aaa37ee7a8bc125dd3fae"}
Sep 11 04:53:18.953367 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:18.953092    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59"
Sep 11 04:53:18.955222 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:18.955192    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:18.956537 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:18.956517    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f"
Sep 11 04:53:19.072872 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.072836    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k72n5\" (UniqueName: \"kubernetes.io/projected/6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2-kube-api-access-k72n5\") pod \"6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2\" (UID: \"6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2\") "
Sep 11 04:53:19.073306 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.072936    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2-plugins-config-volume\") pod \"6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2\" (UID: \"6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2\") "
Sep 11 04:53:19.073306 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.072970    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f25b1e0b-6f41-41dd-9daf-404a1dc3073c-plugins-config-volume\") pod \"f25b1e0b-6f41-41dd-9daf-404a1dc3073c\" (UID: \"f25b1e0b-6f41-41dd-9daf-404a1dc3073c\") "
Sep 11 04:53:19.073306 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.072992    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdd4z\" (UniqueName: \"kubernetes.io/projected/f25b1e0b-6f41-41dd-9daf-404a1dc3073c-kube-api-access-xdd4z\") pod \"f25b1e0b-6f41-41dd-9daf-404a1dc3073c\" (UID: \"f25b1e0b-6f41-41dd-9daf-404a1dc3073c\") "
Sep 11 04:53:19.073440 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.073382    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f25b1e0b-6f41-41dd-9daf-404a1dc3073c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f25b1e0b-6f41-41dd-9daf-404a1dc3073c" (UID: "f25b1e0b-6f41-41dd-9daf-404a1dc3073c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:53:19.073440 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.073413    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2" (UID: "6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:53:19.075408 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.075376    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f25b1e0b-6f41-41dd-9daf-404a1dc3073c-kube-api-access-xdd4z" (OuterVolumeSpecName: "kube-api-access-xdd4z") pod "f25b1e0b-6f41-41dd-9daf-404a1dc3073c" (UID: "f25b1e0b-6f41-41dd-9daf-404a1dc3073c"). InnerVolumeSpecName "kube-api-access-xdd4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:53:19.075548 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.075434    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2-kube-api-access-k72n5" (OuterVolumeSpecName: "kube-api-access-k72n5") pod "6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2" (UID: "6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2"). InnerVolumeSpecName "kube-api-access-k72n5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:53:19.174157 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.174055    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:53:19.174157 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.174107    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f25b1e0b-6f41-41dd-9daf-404a1dc3073c-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:53:19.174157 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.174117    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdd4z\" (UniqueName: \"kubernetes.io/projected/f25b1e0b-6f41-41dd-9daf-404a1dc3073c-kube-api-access-xdd4z\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:53:19.174157 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.174126    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k72n5\" (UniqueName: \"kubernetes.io/projected/6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2-kube-api-access-k72n5\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:53:19.382010 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.381884    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:19.482089 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.481989    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:19.581932 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.581896    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:19.866824 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.866788    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:19.877974 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.877873    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59" event={"ID":"f25b1e0b-6f41-41dd-9daf-404a1dc3073c","Type":"ContainerDied","Data":"ba8daa61e890d2a4628c908fb8605fba948e067e3ff14a3617536ebd5a8caaf4"}
Sep 11 04:53:19.877974 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.877894    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59"
Sep 11 04:53:19.877974 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.877941    2567 scope.go:117] "RemoveContainer" containerID="19c4bc394e74e5bbdddb4f2ea29a6233deef597f805bc0b03b6e5a07ef932d2b"
Sep 11 04:53:19.879462 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.879432    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f" event={"ID":"6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2","Type":"ContainerDied","Data":"7d6c96056f61ab3d730af9ffa51b196233a8f7567e9f147681f356c1c5b0e44d"}
Sep 11 04:53:19.879462 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.879459    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f"
Sep 11 04:53:19.889663 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.889645    2567 scope.go:117] "RemoveContainer" containerID="03291c063ccd9a333229f7ed6d0eab03087873078c4aaa37ee7a8bc125dd3fae"
Sep 11 04:53:19.905734 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.905710    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f"]
Sep 11 04:53:19.911126 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.911099    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5fa7070e-7b6cdb8ffd-zpk6f"]
Sep 11 04:53:19.925324 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.925293    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59"]
Sep 11 04:53:19.928991 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:19.928967    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5fa7070e-667c667f6f-4kk59"]
Sep 11 04:53:20.018170 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:20.018137    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2" path="/var/lib/kubelet/pods/6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2/volumes"
Sep 11 04:53:20.018524 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:20.018510    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f25b1e0b-6f41-41dd-9daf-404a1dc3073c" path="/var/lib/kubelet/pods/f25b1e0b-6f41-41dd-9daf-404a1dc3073c/volumes"
Sep 11 04:53:20.042255 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:20.042225    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:21.042685 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:21.042621    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:22.467708 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:22.467671    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:22.562897 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:22.562866    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:22.662564 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:22.662531    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:22.801451 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:22.801421    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:22.901737 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:22.901703    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:23.239555 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.239465    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:23.407377 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.407343    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:23.481955 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.481921    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:23.520577 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.520552    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs"]
Sep 11 04:53:23.521029 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.521014    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f25b1e0b-6f41-41dd-9daf-404a1dc3073c" containerName="payload-processing"
Sep 11 04:53:23.521094 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.521034    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f25b1e0b-6f41-41dd-9daf-404a1dc3073c" containerName="payload-processing"
Sep 11 04:53:23.521094 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.521046    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2" containerName="payload-pre-processing"
Sep 11 04:53:23.521094 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.521054    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2" containerName="payload-pre-processing"
Sep 11 04:53:23.521242 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.521181    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ee10272-cd1a-4593-ae9a-b1ca67c6a1c2" containerName="payload-pre-processing"
Sep 11 04:53:23.521242 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.521194    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f25b1e0b-6f41-41dd-9daf-404a1dc3073c" containerName="payload-processing"
Sep 11 04:53:23.530044 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.530020    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs"
Sep 11 04:53:23.531340 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.531317    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc"]
Sep 11 04:53:23.533577 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.533557    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-1ecf5332-dockercfg-smdz8\""
Sep 11 04:53:23.533755 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.533733    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-1ecf5332\""
Sep 11 04:53:23.535851 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.535807    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs"]
Sep 11 04:53:23.535962 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.535916    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc"
Sep 11 04:53:23.546999 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.546973    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc"]
Sep 11 04:53:23.614925 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.614897    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fz6h\" (UniqueName: \"kubernetes.io/projected/8ff0cfe3-5ef1-402c-b668-ce2ac7a51195-kube-api-access-5fz6h\") pod \"payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs\" (UID: \"8ff0cfe3-5ef1-402c-b668-ce2ac7a51195\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs"
Sep 11 04:53:23.615117 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.614957    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c8ca059-a884-4a3a-9c35-a5c2134c6035-plugins-config-volume\") pod \"payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc\" (UID: \"8c8ca059-a884-4a3a-9c35-a5c2134c6035\") " pod="openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc"
Sep 11 04:53:23.615117 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.615032    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ff0cfe3-5ef1-402c-b668-ce2ac7a51195-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs\" (UID: \"8ff0cfe3-5ef1-402c-b668-ce2ac7a51195\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs"
Sep 11 04:53:23.615117 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.615056    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qgc7\" (UniqueName: \"kubernetes.io/projected/8c8ca059-a884-4a3a-9c35-a5c2134c6035-kube-api-access-9qgc7\") pod \"payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc\" (UID: \"8c8ca059-a884-4a3a-9c35-a5c2134c6035\") " pod="openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc"
Sep 11 04:53:23.716162 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.716127    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ff0cfe3-5ef1-402c-b668-ce2ac7a51195-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs\" (UID: \"8ff0cfe3-5ef1-402c-b668-ce2ac7a51195\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs"
Sep 11 04:53:23.716162 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.716166    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qgc7\" (UniqueName: \"kubernetes.io/projected/8c8ca059-a884-4a3a-9c35-a5c2134c6035-kube-api-access-9qgc7\") pod \"payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc\" (UID: \"8c8ca059-a884-4a3a-9c35-a5c2134c6035\") " pod="openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc"
Sep 11 04:53:23.716361 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.716224    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fz6h\" (UniqueName: \"kubernetes.io/projected/8ff0cfe3-5ef1-402c-b668-ce2ac7a51195-kube-api-access-5fz6h\") pod \"payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs\" (UID: \"8ff0cfe3-5ef1-402c-b668-ce2ac7a51195\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs"
Sep 11 04:53:23.716361 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.716260    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c8ca059-a884-4a3a-9c35-a5c2134c6035-plugins-config-volume\") pod \"payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc\" (UID: \"8c8ca059-a884-4a3a-9c35-a5c2134c6035\") " pod="openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc"
Sep 11 04:53:23.716885 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.716865    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ff0cfe3-5ef1-402c-b668-ce2ac7a51195-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs\" (UID: \"8ff0cfe3-5ef1-402c-b668-ce2ac7a51195\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs"
Sep 11 04:53:23.716971 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.716954    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c8ca059-a884-4a3a-9c35-a5c2134c6035-plugins-config-volume\") pod \"payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc\" (UID: \"8c8ca059-a884-4a3a-9c35-a5c2134c6035\") " pod="openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc"
Sep 11 04:53:23.724322 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.724303    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fz6h\" (UniqueName: \"kubernetes.io/projected/8ff0cfe3-5ef1-402c-b668-ce2ac7a51195-kube-api-access-5fz6h\") pod \"payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs\" (UID: \"8ff0cfe3-5ef1-402c-b668-ce2ac7a51195\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs"
Sep 11 04:53:23.724430 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.724405    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qgc7\" (UniqueName: \"kubernetes.io/projected/8c8ca059-a884-4a3a-9c35-a5c2134c6035-kube-api-access-9qgc7\") pod \"payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc\" (UID: \"8c8ca059-a884-4a3a-9c35-a5c2134c6035\") " pod="openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc"
Sep 11 04:53:23.845014 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.844936    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs"
Sep 11 04:53:23.852837 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:23.852807    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc"
Sep 11 04:53:24.123555 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:24.123444    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs"]
Sep 11 04:53:24.126412 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:53:24.126384    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ff0cfe3_5ef1_402c_b668_ce2ac7a51195.slice/crio-86dd0087b76cd1e4b60c2c9e75b3f8a8a98d7fd66b10ebdd106c1e3d1291a0f3 WatchSource:0}: Error finding container 86dd0087b76cd1e4b60c2c9e75b3f8a8a98d7fd66b10ebdd106c1e3d1291a0f3: Status 404 returned error can't find the container with id 86dd0087b76cd1e4b60c2c9e75b3f8a8a98d7fd66b10ebdd106c1e3d1291a0f3
Sep 11 04:53:24.135460 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:24.135439    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc"]
Sep 11 04:53:24.139955 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:53:24.139936    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c8ca059_a884_4a3a_9c35_a5c2134c6035.slice/crio-9c439603279883a7035f8d48e0d7f10d641d1fe09ca0d51f602a7fdaf8d08bd3 WatchSource:0}: Error finding container 9c439603279883a7035f8d48e0d7f10d641d1fe09ca0d51f602a7fdaf8d08bd3: Status 404 returned error can't find the container with id 9c439603279883a7035f8d48e0d7f10d641d1fe09ca0d51f602a7fdaf8d08bd3
Sep 11 04:53:24.911631 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:24.911591    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs" event={"ID":"8ff0cfe3-5ef1-402c-b668-ce2ac7a51195","Type":"ContainerStarted","Data":"5f99a643b7d677f9e79f14e28a017d25a710e023b1f0d7e37b0b183d87c1041b"}
Sep 11 04:53:24.912124 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:24.911657    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs"
Sep 11 04:53:24.912124 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:24.911682    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs" event={"ID":"8ff0cfe3-5ef1-402c-b668-ce2ac7a51195","Type":"ContainerStarted","Data":"86dd0087b76cd1e4b60c2c9e75b3f8a8a98d7fd66b10ebdd106c1e3d1291a0f3"}
Sep 11 04:53:24.913194 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:24.913169    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc" event={"ID":"8c8ca059-a884-4a3a-9c35-a5c2134c6035","Type":"ContainerStarted","Data":"c7fd09b3e829eb8c9eca7029b700e5fe07a4adea69569b8b55f566c1e4903e73"}
Sep 11 04:53:24.913295 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:24.913227    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc" event={"ID":"8c8ca059-a884-4a3a-9c35-a5c2134c6035","Type":"ContainerStarted","Data":"9c439603279883a7035f8d48e0d7f10d641d1fe09ca0d51f602a7fdaf8d08bd3"}
Sep 11 04:53:24.913295 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:24.913285    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc"
Sep 11 04:53:24.928246 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:24.928197    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs" podStartSLOduration=1.928182242 podStartE2EDuration="1.928182242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:53:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:53:24.928150004 +0000 UTC m=+819.399613879" watchObservedRunningTime="2026-09-11 04:53:24.928182242 +0000 UTC m=+819.399646118"
Sep 11 04:53:24.945759 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:24.945713    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc" podStartSLOduration=1.945694505 podStartE2EDuration="1.945694505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:53:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:53:24.943633298 +0000 UTC m=+819.415097176" watchObservedRunningTime="2026-09-11 04:53:24.945694505 +0000 UTC m=+819.417158381"
Sep 11 04:53:31.280546 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:31.280496    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:35.920303 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:35.920275    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs"
Sep 11 04:53:35.920675 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:35.920442    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc"
Sep 11 04:53:48.981530 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:48.981495    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:54.005655 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.005617    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:54.033123 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.033090    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc"]
Sep 11 04:53:54.033346 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.033318    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc" podUID="8c8ca059-a884-4a3a-9c35-a5c2134c6035" containerName="payload-processing" containerID="cri-o://c7fd09b3e829eb8c9eca7029b700e5fe07a4adea69569b8b55f566c1e4903e73" gracePeriod=30
Sep 11 04:53:54.044889 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.044860    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs"]
Sep 11 04:53:54.045248 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.045217    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs" podUID="8ff0cfe3-5ef1-402c-b668-ce2ac7a51195" containerName="payload-pre-processing" containerID="cri-o://5f99a643b7d677f9e79f14e28a017d25a710e023b1f0d7e37b0b183d87c1041b" gracePeriod=30
Sep 11 04:53:54.075060 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.075027    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:54.138860 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.138821    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:54.197758 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.197684    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:54.262130 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.262094    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:54.318430 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.318401    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc"
Sep 11 04:53:54.322454 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.322383    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs"
Sep 11 04:53:54.346354 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.346307    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:54.401943 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.401916    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qgc7\" (UniqueName: \"kubernetes.io/projected/8c8ca059-a884-4a3a-9c35-a5c2134c6035-kube-api-access-9qgc7\") pod \"8c8ca059-a884-4a3a-9c35-a5c2134c6035\" (UID: \"8c8ca059-a884-4a3a-9c35-a5c2134c6035\") "
Sep 11 04:53:54.402138 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.401972    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c8ca059-a884-4a3a-9c35-a5c2134c6035-plugins-config-volume\") pod \"8c8ca059-a884-4a3a-9c35-a5c2134c6035\" (UID: \"8c8ca059-a884-4a3a-9c35-a5c2134c6035\") "
Sep 11 04:53:54.402138 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.402016    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ff0cfe3-5ef1-402c-b668-ce2ac7a51195-plugins-config-volume\") pod \"8ff0cfe3-5ef1-402c-b668-ce2ac7a51195\" (UID: \"8ff0cfe3-5ef1-402c-b668-ce2ac7a51195\") "
Sep 11 04:53:54.402138 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.402050    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fz6h\" (UniqueName: \"kubernetes.io/projected/8ff0cfe3-5ef1-402c-b668-ce2ac7a51195-kube-api-access-5fz6h\") pod \"8ff0cfe3-5ef1-402c-b668-ce2ac7a51195\" (UID: \"8ff0cfe3-5ef1-402c-b668-ce2ac7a51195\") "
Sep 11 04:53:54.402464 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.402436    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ff0cfe3-5ef1-402c-b668-ce2ac7a51195-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8ff0cfe3-5ef1-402c-b668-ce2ac7a51195" (UID: "8ff0cfe3-5ef1-402c-b668-ce2ac7a51195"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:53:54.402531 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.402436    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c8ca059-a884-4a3a-9c35-a5c2134c6035-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8c8ca059-a884-4a3a-9c35-a5c2134c6035" (UID: "8c8ca059-a884-4a3a-9c35-a5c2134c6035"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:53:54.404012 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.403989    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c8ca059-a884-4a3a-9c35-a5c2134c6035-kube-api-access-9qgc7" (OuterVolumeSpecName: "kube-api-access-9qgc7") pod "8c8ca059-a884-4a3a-9c35-a5c2134c6035" (UID: "8c8ca059-a884-4a3a-9c35-a5c2134c6035"). InnerVolumeSpecName "kube-api-access-9qgc7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:53:54.404285 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.404268    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ff0cfe3-5ef1-402c-b668-ce2ac7a51195-kube-api-access-5fz6h" (OuterVolumeSpecName: "kube-api-access-5fz6h") pod "8ff0cfe3-5ef1-402c-b668-ce2ac7a51195" (UID: "8ff0cfe3-5ef1-402c-b668-ce2ac7a51195"). InnerVolumeSpecName "kube-api-access-5fz6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:53:54.503030 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.502992    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ff0cfe3-5ef1-402c-b668-ce2ac7a51195-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:53:54.503030 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.503024    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fz6h\" (UniqueName: \"kubernetes.io/projected/8ff0cfe3-5ef1-402c-b668-ce2ac7a51195-kube-api-access-5fz6h\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:53:54.503030 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.503036    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qgc7\" (UniqueName: \"kubernetes.io/projected/8c8ca059-a884-4a3a-9c35-a5c2134c6035-kube-api-access-9qgc7\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:53:54.503294 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.503049    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8c8ca059-a884-4a3a-9c35-a5c2134c6035-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:53:54.751232 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:54.751119    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:55.053391 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:55.053332    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:55.078671 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:55.078641    2567 generic.go:356] "Generic (PLEG): container finished" podID="8ff0cfe3-5ef1-402c-b668-ce2ac7a51195" containerID="5f99a643b7d677f9e79f14e28a017d25a710e023b1f0d7e37b0b183d87c1041b" exitCode=0
Sep 11 04:53:55.078822 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:55.078687    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs"
Sep 11 04:53:55.078822 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:55.078723    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs" event={"ID":"8ff0cfe3-5ef1-402c-b668-ce2ac7a51195","Type":"ContainerDied","Data":"5f99a643b7d677f9e79f14e28a017d25a710e023b1f0d7e37b0b183d87c1041b"}
Sep 11 04:53:55.078822 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:55.078778    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs" event={"ID":"8ff0cfe3-5ef1-402c-b668-ce2ac7a51195","Type":"ContainerDied","Data":"86dd0087b76cd1e4b60c2c9e75b3f8a8a98d7fd66b10ebdd106c1e3d1291a0f3"}
Sep 11 04:53:55.078822 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:55.078810    2567 scope.go:117] "RemoveContainer" containerID="5f99a643b7d677f9e79f14e28a017d25a710e023b1f0d7e37b0b183d87c1041b"
Sep 11 04:53:55.080350 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:55.080325    2567 generic.go:356] "Generic (PLEG): container finished" podID="8c8ca059-a884-4a3a-9c35-a5c2134c6035" containerID="c7fd09b3e829eb8c9eca7029b700e5fe07a4adea69569b8b55f566c1e4903e73" exitCode=0
Sep 11 04:53:55.080441 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:55.080364    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc" event={"ID":"8c8ca059-a884-4a3a-9c35-a5c2134c6035","Type":"ContainerDied","Data":"c7fd09b3e829eb8c9eca7029b700e5fe07a4adea69569b8b55f566c1e4903e73"}
Sep 11 04:53:55.080441 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:55.080371    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc"
Sep 11 04:53:55.080441 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:55.080406    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc" event={"ID":"8c8ca059-a884-4a3a-9c35-a5c2134c6035","Type":"ContainerDied","Data":"9c439603279883a7035f8d48e0d7f10d641d1fe09ca0d51f602a7fdaf8d08bd3"}
Sep 11 04:53:55.089340 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:55.089325    2567 scope.go:117] "RemoveContainer" containerID="5f99a643b7d677f9e79f14e28a017d25a710e023b1f0d7e37b0b183d87c1041b"
Sep 11 04:53:55.089626 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:53:55.089605    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f99a643b7d677f9e79f14e28a017d25a710e023b1f0d7e37b0b183d87c1041b\": container with ID starting with 5f99a643b7d677f9e79f14e28a017d25a710e023b1f0d7e37b0b183d87c1041b not found: ID does not exist" containerID="5f99a643b7d677f9e79f14e28a017d25a710e023b1f0d7e37b0b183d87c1041b"
Sep 11 04:53:55.089686 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:55.089638    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f99a643b7d677f9e79f14e28a017d25a710e023b1f0d7e37b0b183d87c1041b"} err="failed to get container status \"5f99a643b7d677f9e79f14e28a017d25a710e023b1f0d7e37b0b183d87c1041b\": rpc error: code = NotFound desc = could not find container \"5f99a643b7d677f9e79f14e28a017d25a710e023b1f0d7e37b0b183d87c1041b\": container with ID starting with 5f99a643b7d677f9e79f14e28a017d25a710e023b1f0d7e37b0b183d87c1041b not found: ID does not exist"
Sep 11 04:53:55.089686 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:55.089660    2567 scope.go:117] "RemoveContainer" containerID="c7fd09b3e829eb8c9eca7029b700e5fe07a4adea69569b8b55f566c1e4903e73"
Sep 11 04:53:55.097915 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:55.097900    2567 scope.go:117] "RemoveContainer" containerID="c7fd09b3e829eb8c9eca7029b700e5fe07a4adea69569b8b55f566c1e4903e73"
Sep 11 04:53:55.098217 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:53:55.098198    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7fd09b3e829eb8c9eca7029b700e5fe07a4adea69569b8b55f566c1e4903e73\": container with ID starting with c7fd09b3e829eb8c9eca7029b700e5fe07a4adea69569b8b55f566c1e4903e73 not found: ID does not exist" containerID="c7fd09b3e829eb8c9eca7029b700e5fe07a4adea69569b8b55f566c1e4903e73"
Sep 11 04:53:55.098289 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:55.098222    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7fd09b3e829eb8c9eca7029b700e5fe07a4adea69569b8b55f566c1e4903e73"} err="failed to get container status \"c7fd09b3e829eb8c9eca7029b700e5fe07a4adea69569b8b55f566c1e4903e73\": rpc error: code = NotFound desc = could not find container \"c7fd09b3e829eb8c9eca7029b700e5fe07a4adea69569b8b55f566c1e4903e73\": container with ID starting with c7fd09b3e829eb8c9eca7029b700e5fe07a4adea69569b8b55f566c1e4903e73 not found: ID does not exist"
Sep 11 04:53:55.101600 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:55.101579    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs"]
Sep 11 04:53:55.105162 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:55.105132    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-1ecf5332-68d6fcc65d-fmmjs"]
Sep 11 04:53:55.115118 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:55.115095    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc"]
Sep 11 04:53:55.121129 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:55.121105    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-1ecf5332-7db646cd74-rdrvc"]
Sep 11 04:53:55.313307 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:55.313208    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:56.018121 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:56.018088    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c8ca059-a884-4a3a-9c35-a5c2134c6035" path="/var/lib/kubelet/pods/8c8ca059-a884-4a3a-9c35-a5c2134c6035/volumes"
Sep 11 04:53:56.018440 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:56.018426    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ff0cfe3-5ef1-402c-b668-ce2ac7a51195" path="/var/lib/kubelet/pods/8ff0cfe3-5ef1-402c-b668-ce2ac7a51195/volumes"
Sep 11 04:53:56.547832 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:56.547795    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:57.189576 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:57.189526    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:57.284922 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:57.284888    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:58.333929 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:58.333894    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:58.458585 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:58.458543    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:58.594407 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:58.594294    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:58.941572 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:58.941491    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:59.042540 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.042504    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:53:59.046720 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.046696    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7"]
Sep 11 04:53:59.047177 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.047162    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ff0cfe3-5ef1-402c-b668-ce2ac7a51195" containerName="payload-pre-processing"
Sep 11 04:53:59.047177 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.047177    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ff0cfe3-5ef1-402c-b668-ce2ac7a51195" containerName="payload-pre-processing"
Sep 11 04:53:59.047326 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.047191    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c8ca059-a884-4a3a-9c35-a5c2134c6035" containerName="payload-processing"
Sep 11 04:53:59.047326 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.047198    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c8ca059-a884-4a3a-9c35-a5c2134c6035" containerName="payload-processing"
Sep 11 04:53:59.047326 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.047281    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c8ca059-a884-4a3a-9c35-a5c2134c6035" containerName="payload-processing"
Sep 11 04:53:59.047326 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.047290    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ff0cfe3-5ef1-402c-b668-ce2ac7a51195" containerName="payload-pre-processing"
Sep 11 04:53:59.052132 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.052112    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7"
Sep 11 04:53:59.055216 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.055197    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-1d2554f2-dockercfg-5h6q8\""
Sep 11 04:53:59.055383 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.055366    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-1d2554f2\""
Sep 11 04:53:59.060978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.060955    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7"]
Sep 11 04:53:59.088567 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.088539    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs"]
Sep 11 04:53:59.091940 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.091925    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs"
Sep 11 04:53:59.101992 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.101971    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs"]
Sep 11 04:53:59.142367 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.142320    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkfjg\" (UniqueName: \"kubernetes.io/projected/5c68321c-ce33-4a9c-8f50-5dcfb041a40e-kube-api-access-dkfjg\") pod \"payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs\" (UID: \"5c68321c-ce33-4a9c-8f50-5dcfb041a40e\") " pod="openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs"
Sep 11 04:53:59.142553 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.142415    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c68321c-ce33-4a9c-8f50-5dcfb041a40e-plugins-config-volume\") pod \"payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs\" (UID: \"5c68321c-ce33-4a9c-8f50-5dcfb041a40e\") " pod="openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs"
Sep 11 04:53:59.142553 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.142486    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa030df6-d06f-45b8-b150-836a017deb17-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7\" (UID: \"fa030df6-d06f-45b8-b150-836a017deb17\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7"
Sep 11 04:53:59.142553 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.142509    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7kvk\" (UniqueName: \"kubernetes.io/projected/fa030df6-d06f-45b8-b150-836a017deb17-kube-api-access-p7kvk\") pod \"payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7\" (UID: \"fa030df6-d06f-45b8-b150-836a017deb17\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7"
Sep 11 04:53:59.243831 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.243752    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c68321c-ce33-4a9c-8f50-5dcfb041a40e-plugins-config-volume\") pod \"payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs\" (UID: \"5c68321c-ce33-4a9c-8f50-5dcfb041a40e\") " pod="openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs"
Sep 11 04:53:59.243831 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.243808    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa030df6-d06f-45b8-b150-836a017deb17-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7\" (UID: \"fa030df6-d06f-45b8-b150-836a017deb17\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7"
Sep 11 04:53:59.243831 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.243830    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7kvk\" (UniqueName: \"kubernetes.io/projected/fa030df6-d06f-45b8-b150-836a017deb17-kube-api-access-p7kvk\") pod \"payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7\" (UID: \"fa030df6-d06f-45b8-b150-836a017deb17\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7"
Sep 11 04:53:59.244088 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.243880    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkfjg\" (UniqueName: \"kubernetes.io/projected/5c68321c-ce33-4a9c-8f50-5dcfb041a40e-kube-api-access-dkfjg\") pod \"payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs\" (UID: \"5c68321c-ce33-4a9c-8f50-5dcfb041a40e\") " pod="openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs"
Sep 11 04:53:59.244525 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.244507    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c68321c-ce33-4a9c-8f50-5dcfb041a40e-plugins-config-volume\") pod \"payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs\" (UID: \"5c68321c-ce33-4a9c-8f50-5dcfb041a40e\") " pod="openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs"
Sep 11 04:53:59.244682 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.244657    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa030df6-d06f-45b8-b150-836a017deb17-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7\" (UID: \"fa030df6-d06f-45b8-b150-836a017deb17\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7"
Sep 11 04:53:59.251909 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.251885    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkfjg\" (UniqueName: \"kubernetes.io/projected/5c68321c-ce33-4a9c-8f50-5dcfb041a40e-kube-api-access-dkfjg\") pod \"payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs\" (UID: \"5c68321c-ce33-4a9c-8f50-5dcfb041a40e\") " pod="openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs"
Sep 11 04:53:59.252032 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.251952    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7kvk\" (UniqueName: \"kubernetes.io/projected/fa030df6-d06f-45b8-b150-836a017deb17-kube-api-access-p7kvk\") pod \"payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7\" (UID: \"fa030df6-d06f-45b8-b150-836a017deb17\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7"
Sep 11 04:53:59.362698 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.362666    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7"
Sep 11 04:53:59.403045 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.403010    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs"
Sep 11 04:53:59.637492 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.637467    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7"]
Sep 11 04:53:59.640213 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:53:59.640187    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa030df6_d06f_45b8_b150_836a017deb17.slice/crio-a0ed2eb9990e6d69d764f724c07961194e39a429a5ad8fc4705d3027f514db84 WatchSource:0}: Error finding container a0ed2eb9990e6d69d764f724c07961194e39a429a5ad8fc4705d3027f514db84: Status 404 returned error can't find the container with id a0ed2eb9990e6d69d764f724c07961194e39a429a5ad8fc4705d3027f514db84
Sep 11 04:53:59.691913 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.691890    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs"]
Sep 11 04:53:59.693649 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:53:59.693621    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c68321c_ce33_4a9c_8f50_5dcfb041a40e.slice/crio-e8b5bf4dc110e2508f3128644fc828b5717dbca45c75de0257014812d3b9041b WatchSource:0}: Error finding container e8b5bf4dc110e2508f3128644fc828b5717dbca45c75de0257014812d3b9041b: Status 404 returned error can't find the container with id e8b5bf4dc110e2508f3128644fc828b5717dbca45c75de0257014812d3b9041b
Sep 11 04:53:59.756477 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:53:59.756449    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:00.110874 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:00.110836    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7" event={"ID":"fa030df6-d06f-45b8-b150-836a017deb17","Type":"ContainerStarted","Data":"705d19ab539f384400d7894971303af58b05c27681fd312fcb36108878c65636"}
Sep 11 04:54:00.111103 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:00.110899    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7" event={"ID":"fa030df6-d06f-45b8-b150-836a017deb17","Type":"ContainerStarted","Data":"a0ed2eb9990e6d69d764f724c07961194e39a429a5ad8fc4705d3027f514db84"}
Sep 11 04:54:00.111103 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:00.110928    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7"
Sep 11 04:54:00.112424 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:00.112401    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs" event={"ID":"5c68321c-ce33-4a9c-8f50-5dcfb041a40e","Type":"ContainerStarted","Data":"5e13e325babc2463d7aaba62da3bddf35300072097460bdb6a3bac7ed9a0458e"}
Sep 11 04:54:00.112520 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:00.112440    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs" event={"ID":"5c68321c-ce33-4a9c-8f50-5dcfb041a40e","Type":"ContainerStarted","Data":"e8b5bf4dc110e2508f3128644fc828b5717dbca45c75de0257014812d3b9041b"}
Sep 11 04:54:00.112520 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:00.112462    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs"
Sep 11 04:54:00.128282 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:00.128226    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7" podStartSLOduration=1.12820776 podStartE2EDuration="1.12820776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:53:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:54:00.125459539 +0000 UTC m=+854.596923418" watchObservedRunningTime="2026-09-11 04:54:00.12820776 +0000 UTC m=+854.599671636"
Sep 11 04:54:00.141952 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:00.141913    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs" podStartSLOduration=1.141895164 podStartE2EDuration="1.141895164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:53:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:54:00.141438581 +0000 UTC m=+854.612902457" watchObservedRunningTime="2026-09-11 04:54:00.141895164 +0000 UTC m=+854.613359041"
Sep 11 04:54:02.448662 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:02.448627    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:02.677842 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:02.677806    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:11.120254 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:11.120224    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs"
Sep 11 04:54:11.120675 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:11.120304    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7"
Sep 11 04:54:12.971888 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:12.971846    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:12.997925 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:12.997886    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh"]
Sep 11 04:54:12.998220 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:12.998190    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh" podUID="b5a32b74-703a-4480-b368-425d3afd5a18" containerName="payload-processing" containerID="cri-o://f0809f22688f2dd0db904378b280298e2b03c95d5c20b9db13d2d98670cd02ca" gracePeriod=30
Sep 11 04:54:13.005626 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.005474    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4"]
Sep 11 04:54:13.005722 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.005696    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4" podUID="ad24ea67-e38e-4809-89d8-bc6115095036" containerName="payload-pre-processing" containerID="cri-o://aa00bb670a3c6f1229d4404228170ab4a40e265b1fc7102ebf9e4c5c93edfde1" gracePeriod=30
Sep 11 04:54:13.098572 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.098538    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:13.191130 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.191098    2567 generic.go:356] "Generic (PLEG): container finished" podID="b5a32b74-703a-4480-b368-425d3afd5a18" containerID="f0809f22688f2dd0db904378b280298e2b03c95d5c20b9db13d2d98670cd02ca" exitCode=0
Sep 11 04:54:13.191288 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.191159    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh" event={"ID":"b5a32b74-703a-4480-b368-425d3afd5a18","Type":"ContainerDied","Data":"f0809f22688f2dd0db904378b280298e2b03c95d5c20b9db13d2d98670cd02ca"}
Sep 11 04:54:13.192855 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.192833    2567 generic.go:356] "Generic (PLEG): container finished" podID="ad24ea67-e38e-4809-89d8-bc6115095036" containerID="aa00bb670a3c6f1229d4404228170ab4a40e265b1fc7102ebf9e4c5c93edfde1" exitCode=0
Sep 11 04:54:13.192966 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.192909    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4" event={"ID":"ad24ea67-e38e-4809-89d8-bc6115095036","Type":"ContainerDied","Data":"aa00bb670a3c6f1229d4404228170ab4a40e265b1fc7102ebf9e4c5c93edfde1"}
Sep 11 04:54:13.198704 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.198674    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:13.295892 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.295869    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh"
Sep 11 04:54:13.297877 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.297861    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4"
Sep 11 04:54:13.299585 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.299559    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:13.382944 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.382917    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5a32b74-703a-4480-b368-425d3afd5a18-plugins-config-volume\") pod \"b5a32b74-703a-4480-b368-425d3afd5a18\" (UID: \"b5a32b74-703a-4480-b368-425d3afd5a18\") "
Sep 11 04:54:13.383146 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.382952    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2vjt\" (UniqueName: \"kubernetes.io/projected/ad24ea67-e38e-4809-89d8-bc6115095036-kube-api-access-l2vjt\") pod \"ad24ea67-e38e-4809-89d8-bc6115095036\" (UID: \"ad24ea67-e38e-4809-89d8-bc6115095036\") "
Sep 11 04:54:13.383146 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.382977    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdq2n\" (UniqueName: \"kubernetes.io/projected/b5a32b74-703a-4480-b368-425d3afd5a18-kube-api-access-fdq2n\") pod \"b5a32b74-703a-4480-b368-425d3afd5a18\" (UID: \"b5a32b74-703a-4480-b368-425d3afd5a18\") "
Sep 11 04:54:13.383146 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.383125    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad24ea67-e38e-4809-89d8-bc6115095036-plugins-config-volume\") pod \"ad24ea67-e38e-4809-89d8-bc6115095036\" (UID: \"ad24ea67-e38e-4809-89d8-bc6115095036\") "
Sep 11 04:54:13.383474 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.383449    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5a32b74-703a-4480-b368-425d3afd5a18-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b5a32b74-703a-4480-b368-425d3afd5a18" (UID: "b5a32b74-703a-4480-b368-425d3afd5a18"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:54:13.383583 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.383478    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad24ea67-e38e-4809-89d8-bc6115095036-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ad24ea67-e38e-4809-89d8-bc6115095036" (UID: "ad24ea67-e38e-4809-89d8-bc6115095036"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:54:13.385286 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.385263    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5a32b74-703a-4480-b368-425d3afd5a18-kube-api-access-fdq2n" (OuterVolumeSpecName: "kube-api-access-fdq2n") pod "b5a32b74-703a-4480-b368-425d3afd5a18" (UID: "b5a32b74-703a-4480-b368-425d3afd5a18"). InnerVolumeSpecName "kube-api-access-fdq2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:54:13.385383 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.385338    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad24ea67-e38e-4809-89d8-bc6115095036-kube-api-access-l2vjt" (OuterVolumeSpecName: "kube-api-access-l2vjt") pod "ad24ea67-e38e-4809-89d8-bc6115095036" (UID: "ad24ea67-e38e-4809-89d8-bc6115095036"). InnerVolumeSpecName "kube-api-access-l2vjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:54:13.484754 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.484719    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad24ea67-e38e-4809-89d8-bc6115095036-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:54:13.484754 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.484748    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b5a32b74-703a-4480-b368-425d3afd5a18-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:54:13.484754 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.484761    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2vjt\" (UniqueName: \"kubernetes.io/projected/ad24ea67-e38e-4809-89d8-bc6115095036-kube-api-access-l2vjt\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:54:13.484967 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.484774    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdq2n\" (UniqueName: \"kubernetes.io/projected/b5a32b74-703a-4480-b368-425d3afd5a18-kube-api-access-fdq2n\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:54:13.507371 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:13.507342    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:14.199430 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:14.199406    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh"
Sep 11 04:54:14.199922 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:14.199461    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh" event={"ID":"b5a32b74-703a-4480-b368-425d3afd5a18","Type":"ContainerDied","Data":"c0bcda7b49cd9936d2a1a0cc8f3d6f8b3dbe5ed248347f4f9f6d9202041e4f81"}
Sep 11 04:54:14.199922 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:14.199529    2567 scope.go:117] "RemoveContainer" containerID="f0809f22688f2dd0db904378b280298e2b03c95d5c20b9db13d2d98670cd02ca"
Sep 11 04:54:14.201182 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:14.201156    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4" event={"ID":"ad24ea67-e38e-4809-89d8-bc6115095036","Type":"ContainerDied","Data":"da9c1d24eda82f48abaea98cdcab9d065538664d664481252b7e808b94291d08"}
Sep 11 04:54:14.201296 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:14.201191    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4"
Sep 11 04:54:14.212040 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:14.210524    2567 scope.go:117] "RemoveContainer" containerID="aa00bb670a3c6f1229d4404228170ab4a40e265b1fc7102ebf9e4c5c93edfde1"
Sep 11 04:54:14.219914 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:14.219790    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4"]
Sep 11 04:54:14.223666 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:14.223646    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-0a8fef-57b59fdb6-b9sh4"]
Sep 11 04:54:14.234390 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:14.234364    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh"]
Sep 11 04:54:14.238353 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:14.238326    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-0a8fef-6cf4bf45b5-684hh"]
Sep 11 04:54:16.017906 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:16.017873    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad24ea67-e38e-4809-89d8-bc6115095036" path="/var/lib/kubelet/pods/ad24ea67-e38e-4809-89d8-bc6115095036/volumes"
Sep 11 04:54:16.018293 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:16.018246    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5a32b74-703a-4480-b368-425d3afd5a18" path="/var/lib/kubelet/pods/b5a32b74-703a-4480-b368-425d3afd5a18/volumes"
Sep 11 04:54:16.172363 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:16.172329    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:19.408481 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:19.408446    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:19.479665 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:19.479628    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:19.639358 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:19.639324    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:20.067703 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:20.067667    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:20.339223 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:20.339141    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:20.417227 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:20.417185    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:20.609712 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:20.609633    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:23.212894 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:23.212854    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:24.004688 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:24.004638    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:24.100699 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:24.100656    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:25.192468 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.192433    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"]
Sep 11 04:54:25.193668 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.193642    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad24ea67-e38e-4809-89d8-bc6115095036" containerName="payload-pre-processing"
Sep 11 04:54:25.193837 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.193823    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad24ea67-e38e-4809-89d8-bc6115095036" containerName="payload-pre-processing"
Sep 11 04:54:25.193953 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.193941    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5a32b74-703a-4480-b368-425d3afd5a18" containerName="payload-processing"
Sep 11 04:54:25.194031 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.194022    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5a32b74-703a-4480-b368-425d3afd5a18" containerName="payload-processing"
Sep 11 04:54:25.194252 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.194235    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5a32b74-703a-4480-b368-425d3afd5a18" containerName="payload-processing"
Sep 11 04:54:25.194390 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.194376    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad24ea67-e38e-4809-89d8-bc6115095036" containerName="payload-pre-processing"
Sep 11 04:54:25.201321 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.201289    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"
Sep 11 04:54:25.204275 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.204233    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-74gzc\""
Sep 11 04:54:25.204588 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.204469    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-eefe49-serving-cert\""
Sep 11 04:54:25.204705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.204486    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 04:54:25.207426 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.207407    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"]
Sep 11 04:54:25.233394 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.233367    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:25.292982 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.292950    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33c0779f-35e3-4035-8175-830dba060634-openshift-service-ca\") pod \"maas-api-e2e-explicit-eefe49-666496c5bc-2459h\" (UID: \"33c0779f-35e3-4035-8175-830dba060634\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"
Sep 11 04:54:25.293176 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.293004    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33c0779f-35e3-4035-8175-830dba060634-metrics-certs\") pod \"maas-api-e2e-explicit-eefe49-666496c5bc-2459h\" (UID: \"33c0779f-35e3-4035-8175-830dba060634\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"
Sep 11 04:54:25.293176 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.293076    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33c0779f-35e3-4035-8175-830dba060634-maas-api-tls\") pod \"maas-api-e2e-explicit-eefe49-666496c5bc-2459h\" (UID: \"33c0779f-35e3-4035-8175-830dba060634\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"
Sep 11 04:54:25.293176 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.293135    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2l8d\" (UniqueName: \"kubernetes.io/projected/33c0779f-35e3-4035-8175-830dba060634-kube-api-access-r2l8d\") pod \"maas-api-e2e-explicit-eefe49-666496c5bc-2459h\" (UID: \"33c0779f-35e3-4035-8175-830dba060634\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"
Sep 11 04:54:25.335098 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.335049    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:25.394581 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.394554    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33c0779f-35e3-4035-8175-830dba060634-openshift-service-ca\") pod \"maas-api-e2e-explicit-eefe49-666496c5bc-2459h\" (UID: \"33c0779f-35e3-4035-8175-830dba060634\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"
Sep 11 04:54:25.394770 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.394594    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33c0779f-35e3-4035-8175-830dba060634-metrics-certs\") pod \"maas-api-e2e-explicit-eefe49-666496c5bc-2459h\" (UID: \"33c0779f-35e3-4035-8175-830dba060634\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"
Sep 11 04:54:25.394770 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.394614    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33c0779f-35e3-4035-8175-830dba060634-maas-api-tls\") pod \"maas-api-e2e-explicit-eefe49-666496c5bc-2459h\" (UID: \"33c0779f-35e3-4035-8175-830dba060634\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"
Sep 11 04:54:25.394770 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.394661    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2l8d\" (UniqueName: \"kubernetes.io/projected/33c0779f-35e3-4035-8175-830dba060634-kube-api-access-r2l8d\") pod \"maas-api-e2e-explicit-eefe49-666496c5bc-2459h\" (UID: \"33c0779f-35e3-4035-8175-830dba060634\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"
Sep 11 04:54:25.395305 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.395280    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33c0779f-35e3-4035-8175-830dba060634-openshift-service-ca\") pod \"maas-api-e2e-explicit-eefe49-666496c5bc-2459h\" (UID: \"33c0779f-35e3-4035-8175-830dba060634\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"
Sep 11 04:54:25.397282 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.397253    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33c0779f-35e3-4035-8175-830dba060634-metrics-certs\") pod \"maas-api-e2e-explicit-eefe49-666496c5bc-2459h\" (UID: \"33c0779f-35e3-4035-8175-830dba060634\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"
Sep 11 04:54:25.397384 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.397309    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33c0779f-35e3-4035-8175-830dba060634-maas-api-tls\") pod \"maas-api-e2e-explicit-eefe49-666496c5bc-2459h\" (UID: \"33c0779f-35e3-4035-8175-830dba060634\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"
Sep 11 04:54:25.402399 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.402374    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2l8d\" (UniqueName: \"kubernetes.io/projected/33c0779f-35e3-4035-8175-830dba060634-kube-api-access-r2l8d\") pod \"maas-api-e2e-explicit-eefe49-666496c5bc-2459h\" (UID: \"33c0779f-35e3-4035-8175-830dba060634\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"
Sep 11 04:54:25.419189 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.419150    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:25.517974 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.517942    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"
Sep 11 04:54:25.795018 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.794993    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"]
Sep 11 04:54:25.798017 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:54:25.797985    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33c0779f_35e3_4035_8175_830dba060634.slice/crio-9ef2253ccfc36ff545d2f503d1e16affe4dceb4d792650617e413eb4535e732d WatchSource:0}: Error finding container 9ef2253ccfc36ff545d2f503d1e16affe4dceb4d792650617e413eb4535e732d: Status 404 returned error can't find the container with id 9ef2253ccfc36ff545d2f503d1e16affe4dceb4d792650617e413eb4535e732d
Sep 11 04:54:25.799559 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.799539    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 04:54:25.803880 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.803854    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:25.897119 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.897085    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:25.933110 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.933080    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv"]
Sep 11 04:54:25.937856 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.937840    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv"
Sep 11 04:54:25.940428 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.940406    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-eefe49\""
Sep 11 04:54:25.940544 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.940410    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-eefe49-dockercfg-p8c28\""
Sep 11 04:54:25.947199 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.947170    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv"]
Sep 11 04:54:25.981596 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.981568    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk"]
Sep 11 04:54:25.985568 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.985551    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk"
Sep 11 04:54:25.993846 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:25.993822    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk"]
Sep 11 04:54:26.000651 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:26.000622    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgrv6\" (UniqueName: \"kubernetes.io/projected/d4f54345-cca3-45eb-86ea-ea9beb0900c9-kube-api-access-bgrv6\") pod \"payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv\" (UID: \"d4f54345-cca3-45eb-86ea-ea9beb0900c9\") " pod="openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv"
Sep 11 04:54:26.000779 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:26.000691    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4f54345-cca3-45eb-86ea-ea9beb0900c9-plugins-config-volume\") pod \"payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv\" (UID: \"d4f54345-cca3-45eb-86ea-ea9beb0900c9\") " pod="openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv"
Sep 11 04:54:26.000779 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:26.000725    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6783df6b-9e6d-4994-8c2b-f0ac628bd7f9-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk\" (UID: \"6783df6b-9e6d-4994-8c2b-f0ac628bd7f9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk"
Sep 11 04:54:26.000856 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:26.000780    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgm6q\" (UniqueName: \"kubernetes.io/projected/6783df6b-9e6d-4994-8c2b-f0ac628bd7f9-kube-api-access-qgm6q\") pod \"payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk\" (UID: \"6783df6b-9e6d-4994-8c2b-f0ac628bd7f9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk"
Sep 11 04:54:26.102160 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:26.102059    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4f54345-cca3-45eb-86ea-ea9beb0900c9-plugins-config-volume\") pod \"payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv\" (UID: \"d4f54345-cca3-45eb-86ea-ea9beb0900c9\") " pod="openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv"
Sep 11 04:54:26.102160 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:26.102117    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6783df6b-9e6d-4994-8c2b-f0ac628bd7f9-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk\" (UID: \"6783df6b-9e6d-4994-8c2b-f0ac628bd7f9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk"
Sep 11 04:54:26.102160 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:26.102157    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgm6q\" (UniqueName: \"kubernetes.io/projected/6783df6b-9e6d-4994-8c2b-f0ac628bd7f9-kube-api-access-qgm6q\") pod \"payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk\" (UID: \"6783df6b-9e6d-4994-8c2b-f0ac628bd7f9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk"
Sep 11 04:54:26.102452 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:26.102234    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgrv6\" (UniqueName: \"kubernetes.io/projected/d4f54345-cca3-45eb-86ea-ea9beb0900c9-kube-api-access-bgrv6\") pod \"payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv\" (UID: \"d4f54345-cca3-45eb-86ea-ea9beb0900c9\") " pod="openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv"
Sep 11 04:54:26.102867 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:26.102847    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4f54345-cca3-45eb-86ea-ea9beb0900c9-plugins-config-volume\") pod \"payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv\" (UID: \"d4f54345-cca3-45eb-86ea-ea9beb0900c9\") " pod="openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv"
Sep 11 04:54:26.102914 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:26.102899    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6783df6b-9e6d-4994-8c2b-f0ac628bd7f9-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk\" (UID: \"6783df6b-9e6d-4994-8c2b-f0ac628bd7f9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk"
Sep 11 04:54:26.110530 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:26.110501    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgm6q\" (UniqueName: \"kubernetes.io/projected/6783df6b-9e6d-4994-8c2b-f0ac628bd7f9-kube-api-access-qgm6q\") pod \"payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk\" (UID: \"6783df6b-9e6d-4994-8c2b-f0ac628bd7f9\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk"
Sep 11 04:54:26.110634 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:26.110538    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgrv6\" (UniqueName: \"kubernetes.io/projected/d4f54345-cca3-45eb-86ea-ea9beb0900c9-kube-api-access-bgrv6\") pod \"payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv\" (UID: \"d4f54345-cca3-45eb-86ea-ea9beb0900c9\") " pod="openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv"
Sep 11 04:54:26.253258 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:26.253227    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv"
Sep 11 04:54:26.269527 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:26.269499    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h" event={"ID":"33c0779f-35e3-4035-8175-830dba060634","Type":"ContainerStarted","Data":"9ef2253ccfc36ff545d2f503d1e16affe4dceb4d792650617e413eb4535e732d"}
Sep 11 04:54:26.296653 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:26.296616    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk"
Sep 11 04:54:26.558667 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:26.558641    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv"]
Sep 11 04:54:26.561338 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:54:26.561304    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4f54345_cca3_45eb_86ea_ea9beb0900c9.slice/crio-bd9ec0993da51761bec231a7ccfa08ea4df320836ff39d7488ee642d9123d149 WatchSource:0}: Error finding container bd9ec0993da51761bec231a7ccfa08ea4df320836ff39d7488ee642d9123d149: Status 404 returned error can't find the container with id bd9ec0993da51761bec231a7ccfa08ea4df320836ff39d7488ee642d9123d149
Sep 11 04:54:26.603131 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:26.602265    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk"]
Sep 11 04:54:26.605223 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:54:26.605117    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6783df6b_9e6d_4994_8c2b_f0ac628bd7f9.slice/crio-eb1804a14b37a948d719a4bd6fd6f57ec6d818fe314ac256dedfe162509f1585 WatchSource:0}: Error finding container eb1804a14b37a948d719a4bd6fd6f57ec6d818fe314ac256dedfe162509f1585: Status 404 returned error can't find the container with id eb1804a14b37a948d719a4bd6fd6f57ec6d818fe314ac256dedfe162509f1585
Sep 11 04:54:27.279499 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:27.279458    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv" event={"ID":"d4f54345-cca3-45eb-86ea-ea9beb0900c9","Type":"ContainerStarted","Data":"966ab8632ff73c024f4957d69a9da68ea466d620f8ee00cee5013930a7c12143"}
Sep 11 04:54:27.280119 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:27.279559    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv" event={"ID":"d4f54345-cca3-45eb-86ea-ea9beb0900c9","Type":"ContainerStarted","Data":"bd9ec0993da51761bec231a7ccfa08ea4df320836ff39d7488ee642d9123d149"}
Sep 11 04:54:27.280119 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:27.279744    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv"
Sep 11 04:54:27.282043 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:27.282012    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk" event={"ID":"6783df6b-9e6d-4994-8c2b-f0ac628bd7f9","Type":"ContainerStarted","Data":"b812d2c6cf2b8322156daff650e701d3f43587b5fd25603a4eae9ea66f80cf2b"}
Sep 11 04:54:27.282197 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:27.282091    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk" event={"ID":"6783df6b-9e6d-4994-8c2b-f0ac628bd7f9","Type":"ContainerStarted","Data":"eb1804a14b37a948d719a4bd6fd6f57ec6d818fe314ac256dedfe162509f1585"}
Sep 11 04:54:27.282197 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:27.282128    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk"
Sep 11 04:54:27.298005 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:27.297965    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv" podStartSLOduration=2.2979415850000002 podStartE2EDuration="2.297941585s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:54:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:54:27.295328144 +0000 UTC m=+881.766792021" watchObservedRunningTime="2026-09-11 04:54:27.297941585 +0000 UTC m=+881.769405468"
Sep 11 04:54:27.311149 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:27.311103    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk" podStartSLOduration=2.311090401 podStartE2EDuration="2.311090401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:54:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:54:27.309735831 +0000 UTC m=+881.781199709" watchObservedRunningTime="2026-09-11 04:54:27.311090401 +0000 UTC m=+881.782554274"
Sep 11 04:54:28.683750 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:28.683652    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:29.262318 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.262285    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:29.293463 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.292162    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs"]
Sep 11 04:54:29.293463 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.292545    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs" podUID="5c68321c-ce33-4a9c-8f50-5dcfb041a40e" containerName="payload-processing" containerID="cri-o://5e13e325babc2463d7aaba62da3bddf35300072097460bdb6a3bac7ed9a0458e" gracePeriod=30
Sep 11 04:54:29.295180 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.295151    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7"]
Sep 11 04:54:29.295368 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.295342    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7" podUID="fa030df6-d06f-45b8-b150-836a017deb17" containerName="payload-pre-processing" containerID="cri-o://705d19ab539f384400d7894971303af58b05c27681fd312fcb36108878c65636" gracePeriod=30
Sep 11 04:54:29.300748 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.298977    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h" event={"ID":"33c0779f-35e3-4035-8175-830dba060634","Type":"ContainerStarted","Data":"87f03c7eb8611b3e388ce1ca103a618ed11a0a5dd7b8a8c64d0a6b3a9d4e27fe"}
Sep 11 04:54:29.300748 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.299267    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"
Sep 11 04:54:29.349785 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.349730    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h" podStartSLOduration=1.774843248 podStartE2EDuration="4.349709518s" totalImagesPullingTime="2.57486627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:54:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:54:29.349168068 +0000 UTC m=+883.820631945" watchObservedRunningTime="2026-09-11 04:54:29.349709518 +0000 UTC m=+883.821173395"
Sep 11 04:54:29.353977 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.353941    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:29.411349 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.411314    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:29.470826 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.470788    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:29.525858 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.525775    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:29.591958 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.591928    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7"
Sep 11 04:54:29.594771 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.594753    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs"
Sep 11 04:54:29.643322 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.643296    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7kvk\" (UniqueName: \"kubernetes.io/projected/fa030df6-d06f-45b8-b150-836a017deb17-kube-api-access-p7kvk\") pod \"fa030df6-d06f-45b8-b150-836a017deb17\" (UID: \"fa030df6-d06f-45b8-b150-836a017deb17\") "
Sep 11 04:54:29.643489 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.643398    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa030df6-d06f-45b8-b150-836a017deb17-plugins-config-volume\") pod \"fa030df6-d06f-45b8-b150-836a017deb17\" (UID: \"fa030df6-d06f-45b8-b150-836a017deb17\") "
Sep 11 04:54:29.643566 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.643510    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkfjg\" (UniqueName: \"kubernetes.io/projected/5c68321c-ce33-4a9c-8f50-5dcfb041a40e-kube-api-access-dkfjg\") pod \"5c68321c-ce33-4a9c-8f50-5dcfb041a40e\" (UID: \"5c68321c-ce33-4a9c-8f50-5dcfb041a40e\") "
Sep 11 04:54:29.643662 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.643543    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c68321c-ce33-4a9c-8f50-5dcfb041a40e-plugins-config-volume\") pod \"5c68321c-ce33-4a9c-8f50-5dcfb041a40e\" (UID: \"5c68321c-ce33-4a9c-8f50-5dcfb041a40e\") "
Sep 11 04:54:29.643854 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.643824    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa030df6-d06f-45b8-b150-836a017deb17-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fa030df6-d06f-45b8-b150-836a017deb17" (UID: "fa030df6-d06f-45b8-b150-836a017deb17"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:54:29.644044 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.643969    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c68321c-ce33-4a9c-8f50-5dcfb041a40e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5c68321c-ce33-4a9c-8f50-5dcfb041a40e" (UID: "5c68321c-ce33-4a9c-8f50-5dcfb041a40e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:54:29.644176 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.644143    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c68321c-ce33-4a9c-8f50-5dcfb041a40e-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:54:29.644176 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.644162    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fa030df6-d06f-45b8-b150-836a017deb17-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:54:29.646372 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.646333    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c68321c-ce33-4a9c-8f50-5dcfb041a40e-kube-api-access-dkfjg" (OuterVolumeSpecName: "kube-api-access-dkfjg") pod "5c68321c-ce33-4a9c-8f50-5dcfb041a40e" (UID: "5c68321c-ce33-4a9c-8f50-5dcfb041a40e"). InnerVolumeSpecName "kube-api-access-dkfjg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:54:29.646372 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.646343    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa030df6-d06f-45b8-b150-836a017deb17-kube-api-access-p7kvk" (OuterVolumeSpecName: "kube-api-access-p7kvk") pod "fa030df6-d06f-45b8-b150-836a017deb17" (UID: "fa030df6-d06f-45b8-b150-836a017deb17"). InnerVolumeSpecName "kube-api-access-p7kvk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:54:29.745377 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.745343    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkfjg\" (UniqueName: \"kubernetes.io/projected/5c68321c-ce33-4a9c-8f50-5dcfb041a40e-kube-api-access-dkfjg\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:54:29.745377 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:29.745375    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7kvk\" (UniqueName: \"kubernetes.io/projected/fa030df6-d06f-45b8-b150-836a017deb17-kube-api-access-p7kvk\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:54:30.312873 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:30.312781    2567 generic.go:356] "Generic (PLEG): container finished" podID="fa030df6-d06f-45b8-b150-836a017deb17" containerID="705d19ab539f384400d7894971303af58b05c27681fd312fcb36108878c65636" exitCode=0
Sep 11 04:54:30.312873 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:30.312831    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7"
Sep 11 04:54:30.313139 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:30.312864    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7" event={"ID":"fa030df6-d06f-45b8-b150-836a017deb17","Type":"ContainerDied","Data":"705d19ab539f384400d7894971303af58b05c27681fd312fcb36108878c65636"}
Sep 11 04:54:30.313139 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:30.312964    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7" event={"ID":"fa030df6-d06f-45b8-b150-836a017deb17","Type":"ContainerDied","Data":"a0ed2eb9990e6d69d764f724c07961194e39a429a5ad8fc4705d3027f514db84"}
Sep 11 04:54:30.313139 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:30.313001    2567 scope.go:117] "RemoveContainer" containerID="705d19ab539f384400d7894971303af58b05c27681fd312fcb36108878c65636"
Sep 11 04:54:30.314392 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:30.314370    2567 generic.go:356] "Generic (PLEG): container finished" podID="5c68321c-ce33-4a9c-8f50-5dcfb041a40e" containerID="5e13e325babc2463d7aaba62da3bddf35300072097460bdb6a3bac7ed9a0458e" exitCode=0
Sep 11 04:54:30.314515 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:30.314423    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs"
Sep 11 04:54:30.314515 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:30.314457    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs" event={"ID":"5c68321c-ce33-4a9c-8f50-5dcfb041a40e","Type":"ContainerDied","Data":"5e13e325babc2463d7aaba62da3bddf35300072097460bdb6a3bac7ed9a0458e"}
Sep 11 04:54:30.314515 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:30.314508    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs" event={"ID":"5c68321c-ce33-4a9c-8f50-5dcfb041a40e","Type":"ContainerDied","Data":"e8b5bf4dc110e2508f3128644fc828b5717dbca45c75de0257014812d3b9041b"}
Sep 11 04:54:30.322399 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:30.322383    2567 scope.go:117] "RemoveContainer" containerID="705d19ab539f384400d7894971303af58b05c27681fd312fcb36108878c65636"
Sep 11 04:54:30.322683 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:54:30.322658    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"705d19ab539f384400d7894971303af58b05c27681fd312fcb36108878c65636\": container with ID starting with 705d19ab539f384400d7894971303af58b05c27681fd312fcb36108878c65636 not found: ID does not exist" containerID="705d19ab539f384400d7894971303af58b05c27681fd312fcb36108878c65636"
Sep 11 04:54:30.322785 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:30.322688    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"705d19ab539f384400d7894971303af58b05c27681fd312fcb36108878c65636"} err="failed to get container status \"705d19ab539f384400d7894971303af58b05c27681fd312fcb36108878c65636\": rpc error: code = NotFound desc = could not find container \"705d19ab539f384400d7894971303af58b05c27681fd312fcb36108878c65636\": container with ID starting with 705d19ab539f384400d7894971303af58b05c27681fd312fcb36108878c65636 not found: ID does not exist"
Sep 11 04:54:30.322785 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:30.322704    2567 scope.go:117] "RemoveContainer" containerID="5e13e325babc2463d7aaba62da3bddf35300072097460bdb6a3bac7ed9a0458e"
Sep 11 04:54:30.331818 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:30.331796    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7"]
Sep 11 04:54:30.332000 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:30.331974    2567 scope.go:117] "RemoveContainer" containerID="5e13e325babc2463d7aaba62da3bddf35300072097460bdb6a3bac7ed9a0458e"
Sep 11 04:54:30.332295 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:54:30.332276    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e13e325babc2463d7aaba62da3bddf35300072097460bdb6a3bac7ed9a0458e\": container with ID starting with 5e13e325babc2463d7aaba62da3bddf35300072097460bdb6a3bac7ed9a0458e not found: ID does not exist" containerID="5e13e325babc2463d7aaba62da3bddf35300072097460bdb6a3bac7ed9a0458e"
Sep 11 04:54:30.332370 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:30.332300    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e13e325babc2463d7aaba62da3bddf35300072097460bdb6a3bac7ed9a0458e"} err="failed to get container status \"5e13e325babc2463d7aaba62da3bddf35300072097460bdb6a3bac7ed9a0458e\": rpc error: code = NotFound desc = could not find container \"5e13e325babc2463d7aaba62da3bddf35300072097460bdb6a3bac7ed9a0458e\": container with ID starting with 5e13e325babc2463d7aaba62da3bddf35300072097460bdb6a3bac7ed9a0458e not found: ID does not exist"
Sep 11 04:54:30.333873 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:30.333851    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-1d2554f2-58cb4f76db-mk6l7"]
Sep 11 04:54:30.341976 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:30.341954    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs"]
Sep 11 04:54:30.349331 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:30.349309    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-1d2554f2-6599c5f989-m2vfs"]
Sep 11 04:54:30.556354 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:30.556315    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:30.983021 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:30.982980    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:31.283403 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:31.283338    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:31.490352 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:31.490316    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:32.017939 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:32.017903    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c68321c-ce33-4a9c-8f50-5dcfb041a40e" path="/var/lib/kubelet/pods/5c68321c-ce33-4a9c-8f50-5dcfb041a40e/volumes"
Sep 11 04:54:32.018336 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:32.018300    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa030df6-d06f-45b8-b150-836a017deb17" path="/var/lib/kubelet/pods/fa030df6-d06f-45b8-b150-836a017deb17/volumes"
Sep 11 04:54:33.800735 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:33.800700    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:33.917540 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:33.917504    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:35.085085 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.085035    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:35.185658 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.185629    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:35.286305 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.286266    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:35.320004 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.319974    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"
Sep 11 04:54:35.627467 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.627435    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:35.726284 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.726250    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:35.771903 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.771873    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7"]
Sep 11 04:54:35.772364 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.772350    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa030df6-d06f-45b8-b150-836a017deb17" containerName="payload-pre-processing"
Sep 11 04:54:35.772414 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.772368    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa030df6-d06f-45b8-b150-836a017deb17" containerName="payload-pre-processing"
Sep 11 04:54:35.772414 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.772379    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c68321c-ce33-4a9c-8f50-5dcfb041a40e" containerName="payload-processing"
Sep 11 04:54:35.772414 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.772384    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c68321c-ce33-4a9c-8f50-5dcfb041a40e" containerName="payload-processing"
Sep 11 04:54:35.772513 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.772468    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c68321c-ce33-4a9c-8f50-5dcfb041a40e" containerName="payload-processing"
Sep 11 04:54:35.772513 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.772485    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa030df6-d06f-45b8-b150-836a017deb17" containerName="payload-pre-processing"
Sep 11 04:54:35.777119 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.777101    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7"
Sep 11 04:54:35.779601 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.779582    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-fbfc153d\""
Sep 11 04:54:35.779716 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.779584    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-fbfc153d-dockercfg-66267\""
Sep 11 04:54:35.786212 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.786190    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7"]
Sep 11 04:54:35.805175 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.805146    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vdqm\" (UniqueName: \"kubernetes.io/projected/21d86497-52b1-40eb-9b19-07a80ce889d7-kube-api-access-5vdqm\") pod \"payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7\" (UID: \"21d86497-52b1-40eb-9b19-07a80ce889d7\") " pod="openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7"
Sep 11 04:54:35.805345 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.805325    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21d86497-52b1-40eb-9b19-07a80ce889d7-plugins-config-volume\") pod \"payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7\" (UID: \"21d86497-52b1-40eb-9b19-07a80ce889d7\") " pod="openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7"
Sep 11 04:54:35.807012 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.806987    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb"]
Sep 11 04:54:35.810531 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.810511    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb"
Sep 11 04:54:35.820106 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.820084    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb"]
Sep 11 04:54:35.906513 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.906431    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vdqm\" (UniqueName: \"kubernetes.io/projected/21d86497-52b1-40eb-9b19-07a80ce889d7-kube-api-access-5vdqm\") pod \"payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7\" (UID: \"21d86497-52b1-40eb-9b19-07a80ce889d7\") " pod="openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7"
Sep 11 04:54:35.906513 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.906491    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2brd\" (UniqueName: \"kubernetes.io/projected/e787d922-c064-4155-a4ad-72609c715937-kube-api-access-h2brd\") pod \"payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb\" (UID: \"e787d922-c064-4155-a4ad-72609c715937\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb"
Sep 11 04:54:35.906714 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.906524    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e787d922-c064-4155-a4ad-72609c715937-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb\" (UID: \"e787d922-c064-4155-a4ad-72609c715937\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb"
Sep 11 04:54:35.906714 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.906577    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21d86497-52b1-40eb-9b19-07a80ce889d7-plugins-config-volume\") pod \"payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7\" (UID: \"21d86497-52b1-40eb-9b19-07a80ce889d7\") " pod="openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7"
Sep 11 04:54:35.907198 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.907180    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21d86497-52b1-40eb-9b19-07a80ce889d7-plugins-config-volume\") pod \"payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7\" (UID: \"21d86497-52b1-40eb-9b19-07a80ce889d7\") " pod="openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7"
Sep 11 04:54:35.914555 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:35.914528    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vdqm\" (UniqueName: \"kubernetes.io/projected/21d86497-52b1-40eb-9b19-07a80ce889d7-kube-api-access-5vdqm\") pod \"payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7\" (UID: \"21d86497-52b1-40eb-9b19-07a80ce889d7\") " pod="openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7"
Sep 11 04:54:36.008055 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:36.008017    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2brd\" (UniqueName: \"kubernetes.io/projected/e787d922-c064-4155-a4ad-72609c715937-kube-api-access-h2brd\") pod \"payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb\" (UID: \"e787d922-c064-4155-a4ad-72609c715937\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb"
Sep 11 04:54:36.008229 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:36.008098    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e787d922-c064-4155-a4ad-72609c715937-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb\" (UID: \"e787d922-c064-4155-a4ad-72609c715937\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb"
Sep 11 04:54:36.008722 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:36.008701    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e787d922-c064-4155-a4ad-72609c715937-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb\" (UID: \"e787d922-c064-4155-a4ad-72609c715937\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb"
Sep 11 04:54:36.016389 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:36.016361    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2brd\" (UniqueName: \"kubernetes.io/projected/e787d922-c064-4155-a4ad-72609c715937-kube-api-access-h2brd\") pod \"payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb\" (UID: \"e787d922-c064-4155-a4ad-72609c715937\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb"
Sep 11 04:54:36.089131 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:36.089099    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7"
Sep 11 04:54:36.122287 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:36.122260    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb"
Sep 11 04:54:36.363351 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:36.363328    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7"]
Sep 11 04:54:36.365139 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:54:36.365101    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21d86497_52b1_40eb_9b19_07a80ce889d7.slice/crio-49c546805b16838ee80e2161c5f51e64cff625baae7606c52fff2b056d3d79e8 WatchSource:0}: Error finding container 49c546805b16838ee80e2161c5f51e64cff625baae7606c52fff2b056d3d79e8: Status 404 returned error can't find the container with id 49c546805b16838ee80e2161c5f51e64cff625baae7606c52fff2b056d3d79e8
Sep 11 04:54:36.409614 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:36.409531    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb"]
Sep 11 04:54:36.412130 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:54:36.412104    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode787d922_c064_4155_a4ad_72609c715937.slice/crio-b31a73354f3e328f49772954a25651b4bed2b67f3ee699a55fa15eed1b2838f3 WatchSource:0}: Error finding container b31a73354f3e328f49772954a25651b4bed2b67f3ee699a55fa15eed1b2838f3: Status 404 returned error can't find the container with id b31a73354f3e328f49772954a25651b4bed2b67f3ee699a55fa15eed1b2838f3
Sep 11 04:54:37.357307 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:37.357229    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7" event={"ID":"21d86497-52b1-40eb-9b19-07a80ce889d7","Type":"ContainerStarted","Data":"801b7f96deb26de8882dc730b9c0308d3dec7d3ca1e7674da8b5ec174008c7e4"}
Sep 11 04:54:37.357307 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:37.357288    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7" event={"ID":"21d86497-52b1-40eb-9b19-07a80ce889d7","Type":"ContainerStarted","Data":"49c546805b16838ee80e2161c5f51e64cff625baae7606c52fff2b056d3d79e8"}
Sep 11 04:54:37.357830 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:37.357493    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7"
Sep 11 04:54:37.359092 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:37.359042    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb" event={"ID":"e787d922-c064-4155-a4ad-72609c715937","Type":"ContainerStarted","Data":"675525218a2e98f496ac6338a3913fdc6465a625810b62e563e2808a3bce346f"}
Sep 11 04:54:37.359210 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:37.359116    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb" event={"ID":"e787d922-c064-4155-a4ad-72609c715937","Type":"ContainerStarted","Data":"b31a73354f3e328f49772954a25651b4bed2b67f3ee699a55fa15eed1b2838f3"}
Sep 11 04:54:37.359210 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:37.359143    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb"
Sep 11 04:54:37.374411 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:37.374372    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7" podStartSLOduration=2.374351549 podStartE2EDuration="2.374351549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:54:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:54:37.37321638 +0000 UTC m=+891.844680260" watchObservedRunningTime="2026-09-11 04:54:37.374351549 +0000 UTC m=+891.845815425"
Sep 11 04:54:37.388192 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:37.388146    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb" podStartSLOduration=2.388129026 podStartE2EDuration="2.388129026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:54:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:54:37.386752971 +0000 UTC m=+891.858216858" watchObservedRunningTime="2026-09-11 04:54:37.388129026 +0000 UTC m=+891.859592906"
Sep 11 04:54:38.290901 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:38.290871    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv"
Sep 11 04:54:38.291121 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:38.290938    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk"
Sep 11 04:54:46.218507 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:46.218471    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:47.390577 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:47.390527    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:48.255623 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:48.255579    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:48.368032 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:48.368003    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb"
Sep 11 04:54:48.368216 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:48.368105    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7"
Sep 11 04:54:48.800598 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:48.800560    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:54:56.008260 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:54:56.008220    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:00.138821 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:00.138770    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:05.478158 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:05.478039    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:05.504217 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:05.504184    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7"]
Sep 11 04:55:05.504495 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:05.504439    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7" podUID="21d86497-52b1-40eb-9b19-07a80ce889d7" containerName="payload-processing" containerID="cri-o://801b7f96deb26de8882dc730b9c0308d3dec7d3ca1e7674da8b5ec174008c7e4" gracePeriod=30
Sep 11 04:55:05.513862 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:05.513835    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb"]
Sep 11 04:55:05.514351 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:05.514031    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb" podUID="e787d922-c064-4155-a4ad-72609c715937" containerName="payload-pre-processing" containerID="cri-o://675525218a2e98f496ac6338a3913fdc6465a625810b62e563e2808a3bce346f" gracePeriod=30
Sep 11 04:55:05.547836 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:05.547787    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:05.620862 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:05.620832    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:05.720001 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:05.719968    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:05.764401 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:05.764376    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7"
Sep 11 04:55:05.799500 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:05.799471    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb"
Sep 11 04:55:05.799911 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:05.799888    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21d86497-52b1-40eb-9b19-07a80ce889d7-plugins-config-volume\") pod \"21d86497-52b1-40eb-9b19-07a80ce889d7\" (UID: \"21d86497-52b1-40eb-9b19-07a80ce889d7\") "
Sep 11 04:55:05.800006 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:05.799954    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vdqm\" (UniqueName: \"kubernetes.io/projected/21d86497-52b1-40eb-9b19-07a80ce889d7-kube-api-access-5vdqm\") pod \"21d86497-52b1-40eb-9b19-07a80ce889d7\" (UID: \"21d86497-52b1-40eb-9b19-07a80ce889d7\") "
Sep 11 04:55:05.800366 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:05.800339    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21d86497-52b1-40eb-9b19-07a80ce889d7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "21d86497-52b1-40eb-9b19-07a80ce889d7" (UID: "21d86497-52b1-40eb-9b19-07a80ce889d7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:55:05.802788 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:05.802762    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21d86497-52b1-40eb-9b19-07a80ce889d7-kube-api-access-5vdqm" (OuterVolumeSpecName: "kube-api-access-5vdqm") pod "21d86497-52b1-40eb-9b19-07a80ce889d7" (UID: "21d86497-52b1-40eb-9b19-07a80ce889d7"). InnerVolumeSpecName "kube-api-access-5vdqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:55:05.822237 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:05.822105    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:05.901575 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:05.901544    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2brd\" (UniqueName: \"kubernetes.io/projected/e787d922-c064-4155-a4ad-72609c715937-kube-api-access-h2brd\") pod \"e787d922-c064-4155-a4ad-72609c715937\" (UID: \"e787d922-c064-4155-a4ad-72609c715937\") "
Sep 11 04:55:05.901746 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:05.901615    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e787d922-c064-4155-a4ad-72609c715937-plugins-config-volume\") pod \"e787d922-c064-4155-a4ad-72609c715937\" (UID: \"e787d922-c064-4155-a4ad-72609c715937\") "
Sep 11 04:55:05.901842 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:05.901829    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/21d86497-52b1-40eb-9b19-07a80ce889d7-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:55:05.901884 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:05.901845    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vdqm\" (UniqueName: \"kubernetes.io/projected/21d86497-52b1-40eb-9b19-07a80ce889d7-kube-api-access-5vdqm\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:55:05.901953 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:05.901930    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e787d922-c064-4155-a4ad-72609c715937-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e787d922-c064-4155-a4ad-72609c715937" (UID: "e787d922-c064-4155-a4ad-72609c715937"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:55:05.903794 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:05.903776    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e787d922-c064-4155-a4ad-72609c715937-kube-api-access-h2brd" (OuterVolumeSpecName: "kube-api-access-h2brd") pod "e787d922-c064-4155-a4ad-72609c715937" (UID: "e787d922-c064-4155-a4ad-72609c715937"). InnerVolumeSpecName "kube-api-access-h2brd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:55:06.002559 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:06.002475    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e787d922-c064-4155-a4ad-72609c715937-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:55:06.002559 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:06.002500    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2brd\" (UniqueName: \"kubernetes.io/projected/e787d922-c064-4155-a4ad-72609c715937-kube-api-access-h2brd\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:55:06.448702 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:06.448649    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:06.523753 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:06.523719    2567 generic.go:356] "Generic (PLEG): container finished" podID="e787d922-c064-4155-a4ad-72609c715937" containerID="675525218a2e98f496ac6338a3913fdc6465a625810b62e563e2808a3bce346f" exitCode=0
Sep 11 04:55:06.524217 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:06.523776    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb"
Sep 11 04:55:06.524217 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:06.523807    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb" event={"ID":"e787d922-c064-4155-a4ad-72609c715937","Type":"ContainerDied","Data":"675525218a2e98f496ac6338a3913fdc6465a625810b62e563e2808a3bce346f"}
Sep 11 04:55:06.524217 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:06.523910    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb" event={"ID":"e787d922-c064-4155-a4ad-72609c715937","Type":"ContainerDied","Data":"b31a73354f3e328f49772954a25651b4bed2b67f3ee699a55fa15eed1b2838f3"}
Sep 11 04:55:06.524217 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:06.523944    2567 scope.go:117] "RemoveContainer" containerID="675525218a2e98f496ac6338a3913fdc6465a625810b62e563e2808a3bce346f"
Sep 11 04:55:06.525365 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:06.525337    2567 generic.go:356] "Generic (PLEG): container finished" podID="21d86497-52b1-40eb-9b19-07a80ce889d7" containerID="801b7f96deb26de8882dc730b9c0308d3dec7d3ca1e7674da8b5ec174008c7e4" exitCode=0
Sep 11 04:55:06.525424 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:06.525381    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7" event={"ID":"21d86497-52b1-40eb-9b19-07a80ce889d7","Type":"ContainerDied","Data":"801b7f96deb26de8882dc730b9c0308d3dec7d3ca1e7674da8b5ec174008c7e4"}
Sep 11 04:55:06.525424 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:06.525394    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7"
Sep 11 04:55:06.525491 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:06.525419    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7" event={"ID":"21d86497-52b1-40eb-9b19-07a80ce889d7","Type":"ContainerDied","Data":"49c546805b16838ee80e2161c5f51e64cff625baae7606c52fff2b056d3d79e8"}
Sep 11 04:55:06.534886 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:06.534864    2567 scope.go:117] "RemoveContainer" containerID="675525218a2e98f496ac6338a3913fdc6465a625810b62e563e2808a3bce346f"
Sep 11 04:55:06.535218 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:55:06.535201    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"675525218a2e98f496ac6338a3913fdc6465a625810b62e563e2808a3bce346f\": container with ID starting with 675525218a2e98f496ac6338a3913fdc6465a625810b62e563e2808a3bce346f not found: ID does not exist" containerID="675525218a2e98f496ac6338a3913fdc6465a625810b62e563e2808a3bce346f"
Sep 11 04:55:06.535286 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:06.535227    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"675525218a2e98f496ac6338a3913fdc6465a625810b62e563e2808a3bce346f"} err="failed to get container status \"675525218a2e98f496ac6338a3913fdc6465a625810b62e563e2808a3bce346f\": rpc error: code = NotFound desc = could not find container \"675525218a2e98f496ac6338a3913fdc6465a625810b62e563e2808a3bce346f\": container with ID starting with 675525218a2e98f496ac6338a3913fdc6465a625810b62e563e2808a3bce346f not found: ID does not exist"
Sep 11 04:55:06.535286 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:06.535244    2567 scope.go:117] "RemoveContainer" containerID="801b7f96deb26de8882dc730b9c0308d3dec7d3ca1e7674da8b5ec174008c7e4"
Sep 11 04:55:06.544363 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:06.544344    2567 scope.go:117] "RemoveContainer" containerID="801b7f96deb26de8882dc730b9c0308d3dec7d3ca1e7674da8b5ec174008c7e4"
Sep 11 04:55:06.544673 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:55:06.544654    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"801b7f96deb26de8882dc730b9c0308d3dec7d3ca1e7674da8b5ec174008c7e4\": container with ID starting with 801b7f96deb26de8882dc730b9c0308d3dec7d3ca1e7674da8b5ec174008c7e4 not found: ID does not exist" containerID="801b7f96deb26de8882dc730b9c0308d3dec7d3ca1e7674da8b5ec174008c7e4"
Sep 11 04:55:06.544746 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:06.544681    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"801b7f96deb26de8882dc730b9c0308d3dec7d3ca1e7674da8b5ec174008c7e4"} err="failed to get container status \"801b7f96deb26de8882dc730b9c0308d3dec7d3ca1e7674da8b5ec174008c7e4\": rpc error: code = NotFound desc = could not find container \"801b7f96deb26de8882dc730b9c0308d3dec7d3ca1e7674da8b5ec174008c7e4\": container with ID starting with 801b7f96deb26de8882dc730b9c0308d3dec7d3ca1e7674da8b5ec174008c7e4 not found: ID does not exist"
Sep 11 04:55:06.546241 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:06.546217    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb"]
Sep 11 04:55:06.550435 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:06.550413    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-fbfc153d-545b5f5df4-tbxtb"]
Sep 11 04:55:06.553288 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:06.553266    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:06.567230 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:06.567207    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7"]
Sep 11 04:55:06.573999 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:06.573981    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-fbfc153d-759d6dbbc7-8xdm7"]
Sep 11 04:55:06.628825 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:06.628802    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:07.968552 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:07.968515    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:08.017688 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:08.017656    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21d86497-52b1-40eb-9b19-07a80ce889d7" path="/var/lib/kubelet/pods/21d86497-52b1-40eb-9b19-07a80ce889d7/volumes"
Sep 11 04:55:08.018037 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:08.018025    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e787d922-c064-4155-a4ad-72609c715937" path="/var/lib/kubelet/pods/e787d922-c064-4155-a4ad-72609c715937/volumes"
Sep 11 04:55:13.646784 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:13.646749    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:13.717503 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:13.717459    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:13.799766 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:13.799730    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:13.900785 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:13.900706    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:14.002013 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:14.001967    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:14.337085 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:14.337031    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:17.787203 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:17.787163    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:24.483437 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:24.483342    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:24.795994 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:24.795953    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:34.989328 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:34.989291    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"]
Sep 11 04:55:34.989739 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:34.989506    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h" podUID="33c0779f-35e3-4035-8175-830dba060634" containerName="maas-api" containerID="cri-o://87f03c7eb8611b3e388ce1ca103a618ed11a0a5dd7b8a8c64d0a6b3a9d4e27fe" gracePeriod=30
Sep 11 04:55:35.014984 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.014950    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:35.040261 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.040230    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv"]
Sep 11 04:55:35.040457 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.040433    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv" podUID="d4f54345-cca3-45eb-86ea-ea9beb0900c9" containerName="payload-processing" containerID="cri-o://966ab8632ff73c024f4957d69a9da68ea466d620f8ee00cee5013930a7c12143" gracePeriod=30
Sep 11 04:55:35.051653 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.051565    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk"]
Sep 11 04:55:35.052592 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.051795    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk" podUID="6783df6b-9e6d-4994-8c2b-f0ac628bd7f9" containerName="payload-pre-processing" containerID="cri-o://b812d2c6cf2b8322156daff650e701d3f43587b5fd25603a4eae9ea66f80cf2b" gracePeriod=30
Sep 11 04:55:35.316231 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.316205    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"
Sep 11 04:55:35.342423 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.342392    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv"
Sep 11 04:55:35.379782 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.379758    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk"
Sep 11 04:55:35.481336 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.481308    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6783df6b-9e6d-4994-8c2b-f0ac628bd7f9-plugins-config-volume\") pod \"6783df6b-9e6d-4994-8c2b-f0ac628bd7f9\" (UID: \"6783df6b-9e6d-4994-8c2b-f0ac628bd7f9\") "
Sep 11 04:55:35.481501 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.481359    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgm6q\" (UniqueName: \"kubernetes.io/projected/6783df6b-9e6d-4994-8c2b-f0ac628bd7f9-kube-api-access-qgm6q\") pod \"6783df6b-9e6d-4994-8c2b-f0ac628bd7f9\" (UID: \"6783df6b-9e6d-4994-8c2b-f0ac628bd7f9\") "
Sep 11 04:55:35.481501 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.481382    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33c0779f-35e3-4035-8175-830dba060634-openshift-service-ca\") pod \"33c0779f-35e3-4035-8175-830dba060634\" (UID: \"33c0779f-35e3-4035-8175-830dba060634\") "
Sep 11 04:55:35.481501 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.481417    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2l8d\" (UniqueName: \"kubernetes.io/projected/33c0779f-35e3-4035-8175-830dba060634-kube-api-access-r2l8d\") pod \"33c0779f-35e3-4035-8175-830dba060634\" (UID: \"33c0779f-35e3-4035-8175-830dba060634\") "
Sep 11 04:55:35.481501 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.481448    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33c0779f-35e3-4035-8175-830dba060634-maas-api-tls\") pod \"33c0779f-35e3-4035-8175-830dba060634\" (UID: \"33c0779f-35e3-4035-8175-830dba060634\") "
Sep 11 04:55:35.481501 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.481472    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33c0779f-35e3-4035-8175-830dba060634-metrics-certs\") pod \"33c0779f-35e3-4035-8175-830dba060634\" (UID: \"33c0779f-35e3-4035-8175-830dba060634\") "
Sep 11 04:55:35.481765 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.481513    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bgrv6\" (UniqueName: \"kubernetes.io/projected/d4f54345-cca3-45eb-86ea-ea9beb0900c9-kube-api-access-bgrv6\") pod \"d4f54345-cca3-45eb-86ea-ea9beb0900c9\" (UID: \"d4f54345-cca3-45eb-86ea-ea9beb0900c9\") "
Sep 11 04:55:35.481765 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.481586    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4f54345-cca3-45eb-86ea-ea9beb0900c9-plugins-config-volume\") pod \"d4f54345-cca3-45eb-86ea-ea9beb0900c9\" (UID: \"d4f54345-cca3-45eb-86ea-ea9beb0900c9\") "
Sep 11 04:55:35.481765 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.481737    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6783df6b-9e6d-4994-8c2b-f0ac628bd7f9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6783df6b-9e6d-4994-8c2b-f0ac628bd7f9" (UID: "6783df6b-9e6d-4994-8c2b-f0ac628bd7f9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:55:35.481765 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.481747    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33c0779f-35e3-4035-8175-830dba060634-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "33c0779f-35e3-4035-8175-830dba060634" (UID: "33c0779f-35e3-4035-8175-830dba060634"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:55:35.481976 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.481899    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6783df6b-9e6d-4994-8c2b-f0ac628bd7f9-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:55:35.481976 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.481918    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/33c0779f-35e3-4035-8175-830dba060634-openshift-service-ca\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:55:35.482198 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.482151    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4f54345-cca3-45eb-86ea-ea9beb0900c9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d4f54345-cca3-45eb-86ea-ea9beb0900c9" (UID: "d4f54345-cca3-45eb-86ea-ea9beb0900c9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:55:35.483666 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.483639    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4f54345-cca3-45eb-86ea-ea9beb0900c9-kube-api-access-bgrv6" (OuterVolumeSpecName: "kube-api-access-bgrv6") pod "d4f54345-cca3-45eb-86ea-ea9beb0900c9" (UID: "d4f54345-cca3-45eb-86ea-ea9beb0900c9"). InnerVolumeSpecName "kube-api-access-bgrv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:55:35.483852 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.483833    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33c0779f-35e3-4035-8175-830dba060634-kube-api-access-r2l8d" (OuterVolumeSpecName: "kube-api-access-r2l8d") pod "33c0779f-35e3-4035-8175-830dba060634" (UID: "33c0779f-35e3-4035-8175-830dba060634"). InnerVolumeSpecName "kube-api-access-r2l8d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:55:35.483963 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.483944    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33c0779f-35e3-4035-8175-830dba060634-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "33c0779f-35e3-4035-8175-830dba060634" (UID: "33c0779f-35e3-4035-8175-830dba060634"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:55:35.484048 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.484027    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33c0779f-35e3-4035-8175-830dba060634-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "33c0779f-35e3-4035-8175-830dba060634" (UID: "33c0779f-35e3-4035-8175-830dba060634"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 04:55:35.484144 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.484115    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6783df6b-9e6d-4994-8c2b-f0ac628bd7f9-kube-api-access-qgm6q" (OuterVolumeSpecName: "kube-api-access-qgm6q") pod "6783df6b-9e6d-4994-8c2b-f0ac628bd7f9" (UID: "6783df6b-9e6d-4994-8c2b-f0ac628bd7f9"). InnerVolumeSpecName "kube-api-access-qgm6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:55:35.583121 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.583019    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/33c0779f-35e3-4035-8175-830dba060634-maas-api-tls\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:55:35.583121 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.583057    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/33c0779f-35e3-4035-8175-830dba060634-metrics-certs\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:55:35.583121 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.583096    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgrv6\" (UniqueName: \"kubernetes.io/projected/d4f54345-cca3-45eb-86ea-ea9beb0900c9-kube-api-access-bgrv6\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:55:35.583121 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.583113    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4f54345-cca3-45eb-86ea-ea9beb0900c9-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:55:35.583121 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.583126    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgm6q\" (UniqueName: \"kubernetes.io/projected/6783df6b-9e6d-4994-8c2b-f0ac628bd7f9-kube-api-access-qgm6q\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:55:35.583372 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.583139    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2l8d\" (UniqueName: \"kubernetes.io/projected/33c0779f-35e3-4035-8175-830dba060634-kube-api-access-r2l8d\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:55:35.677138 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.677104    2567 generic.go:356] "Generic (PLEG): container finished" podID="d4f54345-cca3-45eb-86ea-ea9beb0900c9" containerID="966ab8632ff73c024f4957d69a9da68ea466d620f8ee00cee5013930a7c12143" exitCode=0
Sep 11 04:55:35.677284 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.677215    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv"
Sep 11 04:55:35.677284 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.677221    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv" event={"ID":"d4f54345-cca3-45eb-86ea-ea9beb0900c9","Type":"ContainerDied","Data":"966ab8632ff73c024f4957d69a9da68ea466d620f8ee00cee5013930a7c12143"}
Sep 11 04:55:35.677284 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.677277    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv" event={"ID":"d4f54345-cca3-45eb-86ea-ea9beb0900c9","Type":"ContainerDied","Data":"bd9ec0993da51761bec231a7ccfa08ea4df320836ff39d7488ee642d9123d149"}
Sep 11 04:55:35.677447 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.677307    2567 scope.go:117] "RemoveContainer" containerID="966ab8632ff73c024f4957d69a9da68ea466d620f8ee00cee5013930a7c12143"
Sep 11 04:55:35.678570 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.678548    2567 generic.go:356] "Generic (PLEG): container finished" podID="33c0779f-35e3-4035-8175-830dba060634" containerID="87f03c7eb8611b3e388ce1ca103a618ed11a0a5dd7b8a8c64d0a6b3a9d4e27fe" exitCode=0
Sep 11 04:55:35.678675 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.678658    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h" event={"ID":"33c0779f-35e3-4035-8175-830dba060634","Type":"ContainerDied","Data":"87f03c7eb8611b3e388ce1ca103a618ed11a0a5dd7b8a8c64d0a6b3a9d4e27fe"}
Sep 11 04:55:35.678737 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.678702    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"
Sep 11 04:55:35.678806 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.678704    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h" event={"ID":"33c0779f-35e3-4035-8175-830dba060634","Type":"ContainerDied","Data":"9ef2253ccfc36ff545d2f503d1e16affe4dceb4d792650617e413eb4535e732d"}
Sep 11 04:55:35.680343 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.680320    2567 generic.go:356] "Generic (PLEG): container finished" podID="6783df6b-9e6d-4994-8c2b-f0ac628bd7f9" containerID="b812d2c6cf2b8322156daff650e701d3f43587b5fd25603a4eae9ea66f80cf2b" exitCode=0
Sep 11 04:55:35.680490 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.680361    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk"
Sep 11 04:55:35.680490 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.680393    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk" event={"ID":"6783df6b-9e6d-4994-8c2b-f0ac628bd7f9","Type":"ContainerDied","Data":"b812d2c6cf2b8322156daff650e701d3f43587b5fd25603a4eae9ea66f80cf2b"}
Sep 11 04:55:35.680490 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.680431    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk" event={"ID":"6783df6b-9e6d-4994-8c2b-f0ac628bd7f9","Type":"ContainerDied","Data":"eb1804a14b37a948d719a4bd6fd6f57ec6d818fe314ac256dedfe162509f1585"}
Sep 11 04:55:35.687681 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.687652    2567 scope.go:117] "RemoveContainer" containerID="966ab8632ff73c024f4957d69a9da68ea466d620f8ee00cee5013930a7c12143"
Sep 11 04:55:35.687996 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:55:35.687971    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"966ab8632ff73c024f4957d69a9da68ea466d620f8ee00cee5013930a7c12143\": container with ID starting with 966ab8632ff73c024f4957d69a9da68ea466d620f8ee00cee5013930a7c12143 not found: ID does not exist" containerID="966ab8632ff73c024f4957d69a9da68ea466d620f8ee00cee5013930a7c12143"
Sep 11 04:55:35.688118 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.688004    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"966ab8632ff73c024f4957d69a9da68ea466d620f8ee00cee5013930a7c12143"} err="failed to get container status \"966ab8632ff73c024f4957d69a9da68ea466d620f8ee00cee5013930a7c12143\": rpc error: code = NotFound desc = could not find container \"966ab8632ff73c024f4957d69a9da68ea466d620f8ee00cee5013930a7c12143\": container with ID starting with 966ab8632ff73c024f4957d69a9da68ea466d620f8ee00cee5013930a7c12143 not found: ID does not exist"
Sep 11 04:55:35.688118 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.688020    2567 scope.go:117] "RemoveContainer" containerID="87f03c7eb8611b3e388ce1ca103a618ed11a0a5dd7b8a8c64d0a6b3a9d4e27fe"
Sep 11 04:55:35.697224 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.697209    2567 scope.go:117] "RemoveContainer" containerID="87f03c7eb8611b3e388ce1ca103a618ed11a0a5dd7b8a8c64d0a6b3a9d4e27fe"
Sep 11 04:55:35.697477 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:55:35.697460    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87f03c7eb8611b3e388ce1ca103a618ed11a0a5dd7b8a8c64d0a6b3a9d4e27fe\": container with ID starting with 87f03c7eb8611b3e388ce1ca103a618ed11a0a5dd7b8a8c64d0a6b3a9d4e27fe not found: ID does not exist" containerID="87f03c7eb8611b3e388ce1ca103a618ed11a0a5dd7b8a8c64d0a6b3a9d4e27fe"
Sep 11 04:55:35.697552 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.697487    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87f03c7eb8611b3e388ce1ca103a618ed11a0a5dd7b8a8c64d0a6b3a9d4e27fe"} err="failed to get container status \"87f03c7eb8611b3e388ce1ca103a618ed11a0a5dd7b8a8c64d0a6b3a9d4e27fe\": rpc error: code = NotFound desc = could not find container \"87f03c7eb8611b3e388ce1ca103a618ed11a0a5dd7b8a8c64d0a6b3a9d4e27fe\": container with ID starting with 87f03c7eb8611b3e388ce1ca103a618ed11a0a5dd7b8a8c64d0a6b3a9d4e27fe not found: ID does not exist"
Sep 11 04:55:35.697552 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.697514    2567 scope.go:117] "RemoveContainer" containerID="b812d2c6cf2b8322156daff650e701d3f43587b5fd25603a4eae9ea66f80cf2b"
Sep 11 04:55:35.704683 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.704657    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"]
Sep 11 04:55:35.705978 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.705961    2567 scope.go:117] "RemoveContainer" containerID="b812d2c6cf2b8322156daff650e701d3f43587b5fd25603a4eae9ea66f80cf2b"
Sep 11 04:55:35.706292 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:55:35.706263    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b812d2c6cf2b8322156daff650e701d3f43587b5fd25603a4eae9ea66f80cf2b\": container with ID starting with b812d2c6cf2b8322156daff650e701d3f43587b5fd25603a4eae9ea66f80cf2b not found: ID does not exist" containerID="b812d2c6cf2b8322156daff650e701d3f43587b5fd25603a4eae9ea66f80cf2b"
Sep 11 04:55:35.706398 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.706298    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b812d2c6cf2b8322156daff650e701d3f43587b5fd25603a4eae9ea66f80cf2b"} err="failed to get container status \"b812d2c6cf2b8322156daff650e701d3f43587b5fd25603a4eae9ea66f80cf2b\": rpc error: code = NotFound desc = could not find container \"b812d2c6cf2b8322156daff650e701d3f43587b5fd25603a4eae9ea66f80cf2b\": container with ID starting with b812d2c6cf2b8322156daff650e701d3f43587b5fd25603a4eae9ea66f80cf2b not found: ID does not exist"
Sep 11 04:55:35.706580 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.706561    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-eefe49-666496c5bc-2459h"]
Sep 11 04:55:35.715504 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.715486    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv"]
Sep 11 04:55:35.719258 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.719238    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-eefe49-6f9565f8f7-z8vjv"]
Sep 11 04:55:35.729233 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.729213    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk"]
Sep 11 04:55:35.732486 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:35.732465    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-eefe49-76fdbd898b-pcgqk"]
Sep 11 04:55:36.018016 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:36.017981    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33c0779f-35e3-4035-8175-830dba060634" path="/var/lib/kubelet/pods/33c0779f-35e3-4035-8175-830dba060634/volumes"
Sep 11 04:55:36.018540 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:36.018520    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6783df6b-9e6d-4994-8c2b-f0ac628bd7f9" path="/var/lib/kubelet/pods/6783df6b-9e6d-4994-8c2b-f0ac628bd7f9/volumes"
Sep 11 04:55:36.018905 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:36.018887    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4f54345-cca3-45eb-86ea-ea9beb0900c9" path="/var/lib/kubelet/pods/d4f54345-cca3-45eb-86ea-ea9beb0900c9/volumes"
Sep 11 04:55:36.579910 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:36.579872    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:39.670058 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:39.670012    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:39.759660 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:39.759619    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:40.209027 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:40.208989    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:41.195626 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:41.195593    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:41.278056 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:41.278024    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:41.451851 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:41.451768    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:41.851381 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:41.851335    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:42.143917 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:42.143741    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:42.584912 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:42.584877    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:43.145711 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:43.145677    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:43.243942 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:43.243871    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:43.343917 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:43.343868    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:45.782704 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:45.782674    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:45.851919 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:45.851886    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:45.930678 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:45.930643    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:55:55.477579 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:55:55.477545    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:01.791436 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:01.791403    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:04.508684 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:04.508645    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:04.596716 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:04.596685    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:04.796297 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:04.796208    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:04.915451 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:04.915394    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:07.599827 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:07.599728    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:07.655388 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:07.655359    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:07.720260 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:07.720226    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:07.820949 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:07.820915    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:09.985571 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:09.985535    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:18.254509 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:18.254455    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:18.578056 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:18.578011    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:18.676472 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:18.676436    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:18.744137 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:18.744106    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:19.013730 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:19.013695    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:19.764753 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:19.764722    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:19.875674 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:19.875638    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-831a1c-86t4q"]
Sep 11 04:56:19.876051 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:19.876038    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21d86497-52b1-40eb-9b19-07a80ce889d7" containerName="payload-processing"
Sep 11 04:56:19.876126 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:19.876052    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="21d86497-52b1-40eb-9b19-07a80ce889d7" containerName="payload-processing"
Sep 11 04:56:19.876126 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:19.876062    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33c0779f-35e3-4035-8175-830dba060634" containerName="maas-api"
Sep 11 04:56:19.876126 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:19.876082    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="33c0779f-35e3-4035-8175-830dba060634" containerName="maas-api"
Sep 11 04:56:19.876126 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:19.876090    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4f54345-cca3-45eb-86ea-ea9beb0900c9" containerName="payload-processing"
Sep 11 04:56:19.876126 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:19.876099    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4f54345-cca3-45eb-86ea-ea9beb0900c9" containerName="payload-processing"
Sep 11 04:56:19.876126 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:19.876118    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e787d922-c064-4155-a4ad-72609c715937" containerName="payload-pre-processing"
Sep 11 04:56:19.876126 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:19.876124    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e787d922-c064-4155-a4ad-72609c715937" containerName="payload-pre-processing"
Sep 11 04:56:19.876340 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:19.876133    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6783df6b-9e6d-4994-8c2b-f0ac628bd7f9" containerName="payload-pre-processing"
Sep 11 04:56:19.876340 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:19.876139    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6783df6b-9e6d-4994-8c2b-f0ac628bd7f9" containerName="payload-pre-processing"
Sep 11 04:56:19.876340 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:19.876196    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="21d86497-52b1-40eb-9b19-07a80ce889d7" containerName="payload-processing"
Sep 11 04:56:19.876340 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:19.876203    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6783df6b-9e6d-4994-8c2b-f0ac628bd7f9" containerName="payload-pre-processing"
Sep 11 04:56:19.876340 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:19.876209    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="33c0779f-35e3-4035-8175-830dba060634" containerName="maas-api"
Sep 11 04:56:19.876340 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:19.876215    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4f54345-cca3-45eb-86ea-ea9beb0900c9" containerName="payload-processing"
Sep 11 04:56:19.876340 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:19.876222    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e787d922-c064-4155-a4ad-72609c715937" containerName="payload-pre-processing"
Sep 11 04:56:19.879317 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:19.879302    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-831a1c-86t4q"
Sep 11 04:56:19.881823 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:19.881806    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vl9kf\""
Sep 11 04:56:19.886192 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:19.886170    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-831a1c-86t4q"]
Sep 11 04:56:19.969988 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:19.969946    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bfda989-12a4-4413-84a1-f28507f361f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-831a1c-86t4q\" (UID: \"9bfda989-12a4-4413-84a1-f28507f361f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-831a1c-86t4q"
Sep 11 04:56:19.970810 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:19.970785    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:20.071325 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:20.071245    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bfda989-12a4-4413-84a1-f28507f361f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-831a1c-86t4q\" (UID: \"9bfda989-12a4-4413-84a1-f28507f361f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-831a1c-86t4q"
Sep 11 04:56:20.071861 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:20.071841    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bfda989-12a4-4413-84a1-f28507f361f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-831a1c-86t4q\" (UID: \"9bfda989-12a4-4413-84a1-f28507f361f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-831a1c-86t4q"
Sep 11 04:56:20.190988 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:20.190952    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-831a1c-86t4q"
Sep 11 04:56:20.462406 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:20.462359    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-831a1c-86t4q"]
Sep 11 04:56:20.463822 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:56:20.463795    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bfda989_12a4_4413_84a1_f28507f361f4.slice/crio-3a57c88383e746a532087a8aa194d668f6a600e4de460b7c66f5a4ca0678cdac WatchSource:0}: Error finding container 3a57c88383e746a532087a8aa194d668f6a600e4de460b7c66f5a4ca0678cdac: Status 404 returned error can't find the container with id 3a57c88383e746a532087a8aa194d668f6a600e4de460b7c66f5a4ca0678cdac
Sep 11 04:56:20.912033 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:20.911998    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-831a1c-86t4q" event={"ID":"9bfda989-12a4-4413-84a1-f28507f361f4","Type":"ContainerStarted","Data":"3a57c88383e746a532087a8aa194d668f6a600e4de460b7c66f5a4ca0678cdac"}
Sep 11 04:56:22.924379 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:22.924343    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-831a1c-86t4q" event={"ID":"9bfda989-12a4-4413-84a1-f28507f361f4","Type":"ContainerStarted","Data":"044695aec04f9c08c129eaa76c622024dc79589c9532bdaefca1d97f89d47188"}
Sep 11 04:56:22.941043 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:22.940997    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-831a1c-86t4q" podStartSLOduration=1.907815871 podStartE2EDuration="3.940982078s" totalImagesPullingTime="2.033166207s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:56:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:56:22.938796879 +0000 UTC m=+997.410260754" watchObservedRunningTime="2026-09-11 04:56:22.940982078 +0000 UTC m=+997.412445987"
Sep 11 04:56:23.930668 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:23.930631    2567 generic.go:356] "Generic (PLEG): container finished" podID="9bfda989-12a4-4413-84a1-f28507f361f4" containerID="044695aec04f9c08c129eaa76c622024dc79589c9532bdaefca1d97f89d47188" exitCode=0
Sep 11 04:56:23.931049 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:23.930718    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-831a1c-86t4q" event={"ID":"9bfda989-12a4-4413-84a1-f28507f361f4","Type":"ContainerDied","Data":"044695aec04f9c08c129eaa76c622024dc79589c9532bdaefca1d97f89d47188"}
Sep 11 04:56:24.824481 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:24.824449    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:24.930635 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:24.930596    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:25.025157 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:25.025027    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:25.075313 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:25.075260    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-831a1c-86t4q"
Sep 11 04:56:25.098001 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:25.097965    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:25.223329 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:25.223300    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bfda989-12a4-4413-84a1-f28507f361f4-maas-api-service-ca\") pod \"9bfda989-12a4-4413-84a1-f28507f361f4\" (UID: \"9bfda989-12a4-4413-84a1-f28507f361f4\") "
Sep 11 04:56:25.223641 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:25.223620    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bfda989-12a4-4413-84a1-f28507f361f4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9bfda989-12a4-4413-84a1-f28507f361f4" (UID: "9bfda989-12a4-4413-84a1-f28507f361f4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:56:25.324795 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:25.324761    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bfda989-12a4-4413-84a1-f28507f361f4-maas-api-service-ca\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:56:25.711994 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:25.711950    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:25.810791 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:25.810754    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:25.912990 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:25.912958    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:25.943148 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:25.943118    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-831a1c-86t4q" event={"ID":"9bfda989-12a4-4413-84a1-f28507f361f4","Type":"ContainerDied","Data":"3a57c88383e746a532087a8aa194d668f6a600e4de460b7c66f5a4ca0678cdac"}
Sep 11 04:56:25.943291 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:25.943170    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a57c88383e746a532087a8aa194d668f6a600e4de460b7c66f5a4ca0678cdac"
Sep 11 04:56:25.943291 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:25.943129    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-831a1c-86t4q"
Sep 11 04:56:27.302588 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:27.302528    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:30.281152 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.281088    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:30.296679 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.296651    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh"]
Sep 11 04:56:30.296986 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.296956    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh" podUID="1b68a707-52e2-4f0e-9397-beb687c48d53" containerName="payload-processing" containerID="cri-o://c19754a06097df750add785e9d9ee6016da6ffdb14a60d3ebfa97a049e0ccf8c" gracePeriod=30
Sep 11 04:56:30.305133 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.305103    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2"]
Sep 11 04:56:30.305380 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.305350    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2" podUID="753bd98a-575b-4db6-ac01-25023748e452" containerName="payload-pre-processing" containerID="cri-o://d53fe0b446ed30585592a1ede9bdbf6abef6bb2d0c32172f161edcf885fcb1bc" gracePeriod=30
Sep 11 04:56:30.339115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.339022    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:30.439672 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.439424    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:30.447361 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.447288    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-831a1c-86t4q"]
Sep 11 04:56:30.450427 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.450402    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-831a1c-86t4q"]
Sep 11 04:56:30.499668 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.499629    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:30.564801 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.564778    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh"
Sep 11 04:56:30.591330 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.591303    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2"
Sep 11 04:56:30.676695 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.676664    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b68a707-52e2-4f0e-9397-beb687c48d53-plugins-config-volume\") pod \"1b68a707-52e2-4f0e-9397-beb687c48d53\" (UID: \"1b68a707-52e2-4f0e-9397-beb687c48d53\") "
Sep 11 04:56:30.676840 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.676715    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6b49\" (UniqueName: \"kubernetes.io/projected/1b68a707-52e2-4f0e-9397-beb687c48d53-kube-api-access-b6b49\") pod \"1b68a707-52e2-4f0e-9397-beb687c48d53\" (UID: \"1b68a707-52e2-4f0e-9397-beb687c48d53\") "
Sep 11 04:56:30.677097 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.677052    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b68a707-52e2-4f0e-9397-beb687c48d53-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1b68a707-52e2-4f0e-9397-beb687c48d53" (UID: "1b68a707-52e2-4f0e-9397-beb687c48d53"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:56:30.679033 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.679015    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b68a707-52e2-4f0e-9397-beb687c48d53-kube-api-access-b6b49" (OuterVolumeSpecName: "kube-api-access-b6b49") pod "1b68a707-52e2-4f0e-9397-beb687c48d53" (UID: "1b68a707-52e2-4f0e-9397-beb687c48d53"). InnerVolumeSpecName "kube-api-access-b6b49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:56:30.777660 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.777635    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d45j9\" (UniqueName: \"kubernetes.io/projected/753bd98a-575b-4db6-ac01-25023748e452-kube-api-access-d45j9\") pod \"753bd98a-575b-4db6-ac01-25023748e452\" (UID: \"753bd98a-575b-4db6-ac01-25023748e452\") "
Sep 11 04:56:30.777814 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.777736    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/753bd98a-575b-4db6-ac01-25023748e452-plugins-config-volume\") pod \"753bd98a-575b-4db6-ac01-25023748e452\" (UID: \"753bd98a-575b-4db6-ac01-25023748e452\") "
Sep 11 04:56:30.778010 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.777993    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b68a707-52e2-4f0e-9397-beb687c48d53-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:56:30.778056 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.778017    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6b49\" (UniqueName: \"kubernetes.io/projected/1b68a707-52e2-4f0e-9397-beb687c48d53-kube-api-access-b6b49\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:56:30.778176 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.778149    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/753bd98a-575b-4db6-ac01-25023748e452-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "753bd98a-575b-4db6-ac01-25023748e452" (UID: "753bd98a-575b-4db6-ac01-25023748e452"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:56:30.779914 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.779896    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/753bd98a-575b-4db6-ac01-25023748e452-kube-api-access-d45j9" (OuterVolumeSpecName: "kube-api-access-d45j9") pod "753bd98a-575b-4db6-ac01-25023748e452" (UID: "753bd98a-575b-4db6-ac01-25023748e452"). InnerVolumeSpecName "kube-api-access-d45j9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:56:30.795419 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.795354    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:30.879189 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.879165    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d45j9\" (UniqueName: \"kubernetes.io/projected/753bd98a-575b-4db6-ac01-25023748e452-kube-api-access-d45j9\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:56:30.879189 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.879187    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/753bd98a-575b-4db6-ac01-25023748e452-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:56:30.970013 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.969977    2567 generic.go:356] "Generic (PLEG): container finished" podID="1b68a707-52e2-4f0e-9397-beb687c48d53" containerID="c19754a06097df750add785e9d9ee6016da6ffdb14a60d3ebfa97a049e0ccf8c" exitCode=0
Sep 11 04:56:30.970351 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.970027    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh"
Sep 11 04:56:30.970351 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.970080    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh" event={"ID":"1b68a707-52e2-4f0e-9397-beb687c48d53","Type":"ContainerDied","Data":"c19754a06097df750add785e9d9ee6016da6ffdb14a60d3ebfa97a049e0ccf8c"}
Sep 11 04:56:30.970351 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.970145    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh" event={"ID":"1b68a707-52e2-4f0e-9397-beb687c48d53","Type":"ContainerDied","Data":"27bf40e28dfecf91900936acaa0a5bcaa5444a6d6952eb9bd52973b6e4b6c0a4"}
Sep 11 04:56:30.970351 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.970176    2567 scope.go:117] "RemoveContainer" containerID="c19754a06097df750add785e9d9ee6016da6ffdb14a60d3ebfa97a049e0ccf8c"
Sep 11 04:56:30.971517 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.971484    2567 generic.go:356] "Generic (PLEG): container finished" podID="753bd98a-575b-4db6-ac01-25023748e452" containerID="d53fe0b446ed30585592a1ede9bdbf6abef6bb2d0c32172f161edcf885fcb1bc" exitCode=0
Sep 11 04:56:30.971636 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.971515    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2" event={"ID":"753bd98a-575b-4db6-ac01-25023748e452","Type":"ContainerDied","Data":"d53fe0b446ed30585592a1ede9bdbf6abef6bb2d0c32172f161edcf885fcb1bc"}
Sep 11 04:56:30.971636 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.971546    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2"
Sep 11 04:56:30.971636 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.971562    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2" event={"ID":"753bd98a-575b-4db6-ac01-25023748e452","Type":"ContainerDied","Data":"cbf616de30e037b8b9068641f88357ad914a33dac8ae5739413fef3cbe3dcc35"}
Sep 11 04:56:30.981729 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.981708    2567 scope.go:117] "RemoveContainer" containerID="c19754a06097df750add785e9d9ee6016da6ffdb14a60d3ebfa97a049e0ccf8c"
Sep 11 04:56:30.982051 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:56:30.982027    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c19754a06097df750add785e9d9ee6016da6ffdb14a60d3ebfa97a049e0ccf8c\": container with ID starting with c19754a06097df750add785e9d9ee6016da6ffdb14a60d3ebfa97a049e0ccf8c not found: ID does not exist" containerID="c19754a06097df750add785e9d9ee6016da6ffdb14a60d3ebfa97a049e0ccf8c"
Sep 11 04:56:30.982140 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.982081    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c19754a06097df750add785e9d9ee6016da6ffdb14a60d3ebfa97a049e0ccf8c"} err="failed to get container status \"c19754a06097df750add785e9d9ee6016da6ffdb14a60d3ebfa97a049e0ccf8c\": rpc error: code = NotFound desc = could not find container \"c19754a06097df750add785e9d9ee6016da6ffdb14a60d3ebfa97a049e0ccf8c\": container with ID starting with c19754a06097df750add785e9d9ee6016da6ffdb14a60d3ebfa97a049e0ccf8c not found: ID does not exist"
Sep 11 04:56:30.982140 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.982103    2567 scope.go:117] "RemoveContainer" containerID="d53fe0b446ed30585592a1ede9bdbf6abef6bb2d0c32172f161edcf885fcb1bc"
Sep 11 04:56:30.990315 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.990297    2567 scope.go:117] "RemoveContainer" containerID="d53fe0b446ed30585592a1ede9bdbf6abef6bb2d0c32172f161edcf885fcb1bc"
Sep 11 04:56:30.990583 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:56:30.990565    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d53fe0b446ed30585592a1ede9bdbf6abef6bb2d0c32172f161edcf885fcb1bc\": container with ID starting with d53fe0b446ed30585592a1ede9bdbf6abef6bb2d0c32172f161edcf885fcb1bc not found: ID does not exist" containerID="d53fe0b446ed30585592a1ede9bdbf6abef6bb2d0c32172f161edcf885fcb1bc"
Sep 11 04:56:30.990659 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.990590    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d53fe0b446ed30585592a1ede9bdbf6abef6bb2d0c32172f161edcf885fcb1bc"} err="failed to get container status \"d53fe0b446ed30585592a1ede9bdbf6abef6bb2d0c32172f161edcf885fcb1bc\": rpc error: code = NotFound desc = could not find container \"d53fe0b446ed30585592a1ede9bdbf6abef6bb2d0c32172f161edcf885fcb1bc\": container with ID starting with d53fe0b446ed30585592a1ede9bdbf6abef6bb2d0c32172f161edcf885fcb1bc not found: ID does not exist"
Sep 11 04:56:30.991626 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.991604    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh"]
Sep 11 04:56:30.995178 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:30.995158    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-831a1c-5cdb89ff9-f62xh"]
Sep 11 04:56:31.004912 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:31.004893    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2"]
Sep 11 04:56:31.009024 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:31.009004    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-831a1c-867676f9c8-9rzp2"]
Sep 11 04:56:32.017033 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:32.016999    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b68a707-52e2-4f0e-9397-beb687c48d53" path="/var/lib/kubelet/pods/1b68a707-52e2-4f0e-9397-beb687c48d53/volumes"
Sep 11 04:56:32.017414 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:32.017372    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="753bd98a-575b-4db6-ac01-25023748e452" path="/var/lib/kubelet/pods/753bd98a-575b-4db6-ac01-25023748e452/volumes"
Sep 11 04:56:32.017655 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:32.017642    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bfda989-12a4-4413-84a1-f28507f361f4" path="/var/lib/kubelet/pods/9bfda989-12a4-4413-84a1-f28507f361f4/volumes"
Sep 11 04:56:33.262177 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:33.262135    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:33.328793 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:33.328752    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:33.910783 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:33.910746    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:36.530206 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:36.530172    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:36.631417 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:36.631383    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:36.738456 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:36.738424    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:37.052191 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:37.052164    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7bdb3c-5twlw"]
Sep 11 04:56:37.052558 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:37.052545    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="753bd98a-575b-4db6-ac01-25023748e452" containerName="payload-pre-processing"
Sep 11 04:56:37.052558 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:37.052559    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="753bd98a-575b-4db6-ac01-25023748e452" containerName="payload-pre-processing"
Sep 11 04:56:37.052663 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:37.052582    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bfda989-12a4-4413-84a1-f28507f361f4" containerName="revoke-keys"
Sep 11 04:56:37.052663 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:37.052588    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bfda989-12a4-4413-84a1-f28507f361f4" containerName="revoke-keys"
Sep 11 04:56:37.052663 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:37.052597    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b68a707-52e2-4f0e-9397-beb687c48d53" containerName="payload-processing"
Sep 11 04:56:37.052663 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:37.052603    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b68a707-52e2-4f0e-9397-beb687c48d53" containerName="payload-processing"
Sep 11 04:56:37.052663 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:37.052664    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b68a707-52e2-4f0e-9397-beb687c48d53" containerName="payload-processing"
Sep 11 04:56:37.052849 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:37.052671    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bfda989-12a4-4413-84a1-f28507f361f4" containerName="revoke-keys"
Sep 11 04:56:37.052849 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:37.052678    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="753bd98a-575b-4db6-ac01-25023748e452" containerName="payload-pre-processing"
Sep 11 04:56:37.057195 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:37.057178    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7bdb3c-5twlw"
Sep 11 04:56:37.060550 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:37.060341    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vl9kf\""
Sep 11 04:56:37.061748 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:37.061728    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7bdb3c-5twlw"]
Sep 11 04:56:37.134590 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:37.134562    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dba40194-11fc-4214-b038-e22be2645731-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7bdb3c-5twlw\" (UID: \"dba40194-11fc-4214-b038-e22be2645731\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7bdb3c-5twlw"
Sep 11 04:56:37.161911 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:37.161881    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:37.235757 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:37.235701    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dba40194-11fc-4214-b038-e22be2645731-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7bdb3c-5twlw\" (UID: \"dba40194-11fc-4214-b038-e22be2645731\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7bdb3c-5twlw"
Sep 11 04:56:37.236379 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:37.236355    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dba40194-11fc-4214-b038-e22be2645731-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-7bdb3c-5twlw\" (UID: \"dba40194-11fc-4214-b038-e22be2645731\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7bdb3c-5twlw"
Sep 11 04:56:37.369319 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:37.369243    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7bdb3c-5twlw"
Sep 11 04:56:37.640588 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:37.640566    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7bdb3c-5twlw"]
Sep 11 04:56:37.642967 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:56:37.642936    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba40194_11fc_4214_b038_e22be2645731.slice/crio-d2e9f8a732f431604e78026567ae60dd9715e3a1f96cf018051bef63d7147990 WatchSource:0}: Error finding container d2e9f8a732f431604e78026567ae60dd9715e3a1f96cf018051bef63d7147990: Status 404 returned error can't find the container with id d2e9f8a732f431604e78026567ae60dd9715e3a1f96cf018051bef63d7147990
Sep 11 04:56:38.019025 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:38.018994    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7bdb3c-5twlw" event={"ID":"dba40194-11fc-4214-b038-e22be2645731","Type":"ContainerStarted","Data":"ca183333798b59359dc5fc815bfb9744ed385beef71566f543dd2d8a2a3f9338"}
Sep 11 04:56:38.019182 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:38.019043    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7bdb3c-5twlw" event={"ID":"dba40194-11fc-4214-b038-e22be2645731","Type":"ContainerStarted","Data":"d2e9f8a732f431604e78026567ae60dd9715e3a1f96cf018051bef63d7147990"}
Sep 11 04:56:38.032658 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:38.032606    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7bdb3c-5twlw" podStartSLOduration=1.03258784 podStartE2EDuration="1.03258784s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:56:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:56:38.032197689 +0000 UTC m=+1012.503661560" watchObservedRunningTime="2026-09-11 04:56:38.03258784 +0000 UTC m=+1012.504051716"
Sep 11 04:56:38.990753 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:38.990714    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:39.025345 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:39.025314    2567 generic.go:356] "Generic (PLEG): container finished" podID="dba40194-11fc-4214-b038-e22be2645731" containerID="ca183333798b59359dc5fc815bfb9744ed385beef71566f543dd2d8a2a3f9338" exitCode=0
Sep 11 04:56:39.025484 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:39.025353    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7bdb3c-5twlw" event={"ID":"dba40194-11fc-4214-b038-e22be2645731","Type":"ContainerDied","Data":"ca183333798b59359dc5fc815bfb9744ed385beef71566f543dd2d8a2a3f9338"}
Sep 11 04:56:39.917637 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:39.917600    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:40.020300 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:40.020248    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:40.164775 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:40.164752    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7bdb3c-5twlw"
Sep 11 04:56:40.213732 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:40.213623    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:40.261715 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:40.261688    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dba40194-11fc-4214-b038-e22be2645731-maas-api-service-ca\") pod \"dba40194-11fc-4214-b038-e22be2645731\" (UID: \"dba40194-11fc-4214-b038-e22be2645731\") "
Sep 11 04:56:40.262003 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:40.261982    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dba40194-11fc-4214-b038-e22be2645731-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dba40194-11fc-4214-b038-e22be2645731" (UID: "dba40194-11fc-4214-b038-e22be2645731"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:56:40.362799 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:40.362767    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dba40194-11fc-4214-b038-e22be2645731-maas-api-service-ca\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:56:40.455545 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:40.455513    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:40.682133 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:40.682104    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:41.036861 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:41.036834    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7bdb3c-5twlw"
Sep 11 04:56:41.037259 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:41.036828    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7bdb3c-5twlw" event={"ID":"dba40194-11fc-4214-b038-e22be2645731","Type":"ContainerDied","Data":"d2e9f8a732f431604e78026567ae60dd9715e3a1f96cf018051bef63d7147990"}
Sep 11 04:56:41.037259 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:41.036964    2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d2e9f8a732f431604e78026567ae60dd9715e3a1f96cf018051bef63d7147990"
Sep 11 04:56:43.068314 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:43.068274    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:43.168728 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:43.168696    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:44.194562 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:44.194522    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:44.294716 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:44.294678    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:44.345470 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:44.345432    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:44.724534 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:44.724495    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:44.824583 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:44.824548    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:44.829724 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:44.829699    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2"]
Sep 11 04:56:44.830102 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:44.830090    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dba40194-11fc-4214-b038-e22be2645731" containerName="revoke-keys"
Sep 11 04:56:44.830151 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:44.830104    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="dba40194-11fc-4214-b038-e22be2645731" containerName="revoke-keys"
Sep 11 04:56:44.830187 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:44.830177    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="dba40194-11fc-4214-b038-e22be2645731" containerName="revoke-keys"
Sep 11 04:56:44.833274 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:44.833259    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2"
Sep 11 04:56:44.836428 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:44.836406    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-ca9fd5b9\""
Sep 11 04:56:44.836545 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:44.836466    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-ca9fd5b9-dockercfg-85hqd\""
Sep 11 04:56:44.844162 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:44.844140    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2"]
Sep 11 04:56:44.868021 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:44.867988    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h"]
Sep 11 04:56:44.871664 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:44.871645    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h"
Sep 11 04:56:44.881948 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:44.881925    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h"]
Sep 11 04:56:44.902033 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:44.902003    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tjjf\" (UniqueName: \"kubernetes.io/projected/f3d143c4-ceed-4d30-a7ee-aab7b515885d-kube-api-access-9tjjf\") pod \"payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h\" (UID: \"f3d143c4-ceed-4d30-a7ee-aab7b515885d\") " pod="openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h"
Sep 11 04:56:44.902153 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:44.902057    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3d143c4-ceed-4d30-a7ee-aab7b515885d-plugins-config-volume\") pod \"payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h\" (UID: \"f3d143c4-ceed-4d30-a7ee-aab7b515885d\") " pod="openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h"
Sep 11 04:56:44.902204 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:44.902150    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4e673c5-8e82-4457-8ed7-c00f85cd0024-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2\" (UID: \"e4e673c5-8e82-4457-8ed7-c00f85cd0024\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2"
Sep 11 04:56:44.902240 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:44.902209    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8572\" (UniqueName: \"kubernetes.io/projected/e4e673c5-8e82-4457-8ed7-c00f85cd0024-kube-api-access-b8572\") pod \"payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2\" (UID: \"e4e673c5-8e82-4457-8ed7-c00f85cd0024\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2"
Sep 11 04:56:45.003580 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:45.003491    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3d143c4-ceed-4d30-a7ee-aab7b515885d-plugins-config-volume\") pod \"payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h\" (UID: \"f3d143c4-ceed-4d30-a7ee-aab7b515885d\") " pod="openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h"
Sep 11 04:56:45.003580 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:45.003541    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4e673c5-8e82-4457-8ed7-c00f85cd0024-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2\" (UID: \"e4e673c5-8e82-4457-8ed7-c00f85cd0024\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2"
Sep 11 04:56:45.003810 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:45.003588    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8572\" (UniqueName: \"kubernetes.io/projected/e4e673c5-8e82-4457-8ed7-c00f85cd0024-kube-api-access-b8572\") pod \"payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2\" (UID: \"e4e673c5-8e82-4457-8ed7-c00f85cd0024\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2"
Sep 11 04:56:45.003810 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:45.003654    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tjjf\" (UniqueName: \"kubernetes.io/projected/f3d143c4-ceed-4d30-a7ee-aab7b515885d-kube-api-access-9tjjf\") pod \"payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h\" (UID: \"f3d143c4-ceed-4d30-a7ee-aab7b515885d\") " pod="openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h"
Sep 11 04:56:45.004323 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:45.004299    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3d143c4-ceed-4d30-a7ee-aab7b515885d-plugins-config-volume\") pod \"payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h\" (UID: \"f3d143c4-ceed-4d30-a7ee-aab7b515885d\") " pod="openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h"
Sep 11 04:56:45.004425 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:45.004368    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4e673c5-8e82-4457-8ed7-c00f85cd0024-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2\" (UID: \"e4e673c5-8e82-4457-8ed7-c00f85cd0024\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2"
Sep 11 04:56:45.012849 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:45.012829    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tjjf\" (UniqueName: \"kubernetes.io/projected/f3d143c4-ceed-4d30-a7ee-aab7b515885d-kube-api-access-9tjjf\") pod \"payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h\" (UID: \"f3d143c4-ceed-4d30-a7ee-aab7b515885d\") " pod="openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h"
Sep 11 04:56:45.012972 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:45.012952    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8572\" (UniqueName: \"kubernetes.io/projected/e4e673c5-8e82-4457-8ed7-c00f85cd0024-kube-api-access-b8572\") pod \"payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2\" (UID: \"e4e673c5-8e82-4457-8ed7-c00f85cd0024\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2"
Sep 11 04:56:45.143960 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:45.143926    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2"
Sep 11 04:56:45.183523 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:45.183486    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h"
Sep 11 04:56:45.420894 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:45.420872    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2"]
Sep 11 04:56:45.422592 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:56:45.422560    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4e673c5_8e82_4457_8ed7_c00f85cd0024.slice/crio-79e694d21fc1b4f3cc1861ac9dd9b6e0ed89e0b75eef603773c923eb255daf21 WatchSource:0}: Error finding container 79e694d21fc1b4f3cc1861ac9dd9b6e0ed89e0b75eef603773c923eb255daf21: Status 404 returned error can't find the container with id 79e694d21fc1b4f3cc1861ac9dd9b6e0ed89e0b75eef603773c923eb255daf21
Sep 11 04:56:45.470037 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:45.470015    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h"]
Sep 11 04:56:45.472403 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:56:45.472374    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3d143c4_ceed_4d30_a7ee_aab7b515885d.slice/crio-48a3764a40d0445bf554168b7834c68ca3a8a5d021cd3b5f2c094c2645af4a10 WatchSource:0}: Error finding container 48a3764a40d0445bf554168b7834c68ca3a8a5d021cd3b5f2c094c2645af4a10: Status 404 returned error can't find the container with id 48a3764a40d0445bf554168b7834c68ca3a8a5d021cd3b5f2c094c2645af4a10
Sep 11 04:56:46.066548 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:46.066507    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h" event={"ID":"f3d143c4-ceed-4d30-a7ee-aab7b515885d","Type":"ContainerStarted","Data":"8925612361d22bced1a7b562f7af59cfa4f221e1a6c42d62ad21a6317c4cd443"}
Sep 11 04:56:46.066810 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:46.066581    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h" event={"ID":"f3d143c4-ceed-4d30-a7ee-aab7b515885d","Type":"ContainerStarted","Data":"48a3764a40d0445bf554168b7834c68ca3a8a5d021cd3b5f2c094c2645af4a10"}
Sep 11 04:56:46.066810 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:46.066707    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h"
Sep 11 04:56:46.068239 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:46.068211    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2" event={"ID":"e4e673c5-8e82-4457-8ed7-c00f85cd0024","Type":"ContainerStarted","Data":"afb33a4732b4eea448fab71ebe0d0ef9d148e3a107e470f751f0d9b833ad7c85"}
Sep 11 04:56:46.068239 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:46.068252    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2" event={"ID":"e4e673c5-8e82-4457-8ed7-c00f85cd0024","Type":"ContainerStarted","Data":"79e694d21fc1b4f3cc1861ac9dd9b6e0ed89e0b75eef603773c923eb255daf21"}
Sep 11 04:56:46.068433 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:46.068337    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2"
Sep 11 04:56:46.082604 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:46.082555    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h" podStartSLOduration=2.082538681 podStartE2EDuration="2.082538681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:56:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:56:46.081848249 +0000 UTC m=+1020.553312125" watchObservedRunningTime="2026-09-11 04:56:46.082538681 +0000 UTC m=+1020.554002558"
Sep 11 04:56:46.097307 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:46.097253    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2" podStartSLOduration=2.097236392 podStartE2EDuration="2.097236392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:56:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:56:46.096027711 +0000 UTC m=+1020.567491625" watchObservedRunningTime="2026-09-11 04:56:46.097236392 +0000 UTC m=+1020.568700270"
Sep 11 04:56:47.184747 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:47.184708    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:47.705339 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:47.705299    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:47.732451 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:47.732421    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq"]
Sep 11 04:56:47.732683 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:47.732657    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq" podUID="c6c6ce18-02b2-4983-81b6-c4ab71d15504" containerName="payload-processing" containerID="cri-o://809b761ab3e8df63a182a057c6b64966c69fb63153e827bc51838b5bf6476685" gracePeriod=30
Sep 11 04:56:47.737557 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:47.737529    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6"]
Sep 11 04:56:47.737777 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:47.737751    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6" podUID="76435f19-2fb3-48ff-ae00-b3488ea0323d" containerName="payload-pre-processing" containerID="cri-o://733a8b64a74211a1fa48eb4b42d2a843ff0f3fa43cc99196d1d665420eba1bf3" gracePeriod=30
Sep 11 04:56:47.770745 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:47.770705    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:47.827796 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:47.827763    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:47.864566 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:47.864502    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7bdb3c-5twlw"]
Sep 11 04:56:47.866974 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:47.866949    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-7bdb3c-5twlw"]
Sep 11 04:56:47.883848 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:47.883813    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:48.018331 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.018308    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dba40194-11fc-4214-b038-e22be2645731" path="/var/lib/kubelet/pods/dba40194-11fc-4214-b038-e22be2645731/volumes"
Sep 11 04:56:48.019043 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.019027    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq"
Sep 11 04:56:48.022330 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.022314    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6"
Sep 11 04:56:48.082579 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.082548    2567 generic.go:356] "Generic (PLEG): container finished" podID="76435f19-2fb3-48ff-ae00-b3488ea0323d" containerID="733a8b64a74211a1fa48eb4b42d2a843ff0f3fa43cc99196d1d665420eba1bf3" exitCode=0
Sep 11 04:56:48.082755 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.082603    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6"
Sep 11 04:56:48.082755 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.082638    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6" event={"ID":"76435f19-2fb3-48ff-ae00-b3488ea0323d","Type":"ContainerDied","Data":"733a8b64a74211a1fa48eb4b42d2a843ff0f3fa43cc99196d1d665420eba1bf3"}
Sep 11 04:56:48.082755 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.082698    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6" event={"ID":"76435f19-2fb3-48ff-ae00-b3488ea0323d","Type":"ContainerDied","Data":"6c45c076d40c6470ad4a06d98dbacf4f73b907daeefdbb8c63ed48ac20498a10"}
Sep 11 04:56:48.082755 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.082731    2567 scope.go:117] "RemoveContainer" containerID="733a8b64a74211a1fa48eb4b42d2a843ff0f3fa43cc99196d1d665420eba1bf3"
Sep 11 04:56:48.084055 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.084033    2567 generic.go:356] "Generic (PLEG): container finished" podID="c6c6ce18-02b2-4983-81b6-c4ab71d15504" containerID="809b761ab3e8df63a182a057c6b64966c69fb63153e827bc51838b5bf6476685" exitCode=0
Sep 11 04:56:48.084173 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.084110    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq"
Sep 11 04:56:48.084173 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.084099    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq" event={"ID":"c6c6ce18-02b2-4983-81b6-c4ab71d15504","Type":"ContainerDied","Data":"809b761ab3e8df63a182a057c6b64966c69fb63153e827bc51838b5bf6476685"}
Sep 11 04:56:48.084270 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.084187    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq" event={"ID":"c6c6ce18-02b2-4983-81b6-c4ab71d15504","Type":"ContainerDied","Data":"6709194aec3e71268cdb3f4c073deb00a4fabd61848ca8e1fee7960067f4ab49"}
Sep 11 04:56:48.092108 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.092086    2567 scope.go:117] "RemoveContainer" containerID="733a8b64a74211a1fa48eb4b42d2a843ff0f3fa43cc99196d1d665420eba1bf3"
Sep 11 04:56:48.092419 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:56:48.092392    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"733a8b64a74211a1fa48eb4b42d2a843ff0f3fa43cc99196d1d665420eba1bf3\": container with ID starting with 733a8b64a74211a1fa48eb4b42d2a843ff0f3fa43cc99196d1d665420eba1bf3 not found: ID does not exist" containerID="733a8b64a74211a1fa48eb4b42d2a843ff0f3fa43cc99196d1d665420eba1bf3"
Sep 11 04:56:48.092501 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.092429    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"733a8b64a74211a1fa48eb4b42d2a843ff0f3fa43cc99196d1d665420eba1bf3"} err="failed to get container status \"733a8b64a74211a1fa48eb4b42d2a843ff0f3fa43cc99196d1d665420eba1bf3\": rpc error: code = NotFound desc = could not find container \"733a8b64a74211a1fa48eb4b42d2a843ff0f3fa43cc99196d1d665420eba1bf3\": container with ID starting with 733a8b64a74211a1fa48eb4b42d2a843ff0f3fa43cc99196d1d665420eba1bf3 not found: ID does not exist"
Sep 11 04:56:48.092501 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.092444    2567 scope.go:117] "RemoveContainer" containerID="809b761ab3e8df63a182a057c6b64966c69fb63153e827bc51838b5bf6476685"
Sep 11 04:56:48.100358 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.100295    2567 scope.go:117] "RemoveContainer" containerID="809b761ab3e8df63a182a057c6b64966c69fb63153e827bc51838b5bf6476685"
Sep 11 04:56:48.100699 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:56:48.100672    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"809b761ab3e8df63a182a057c6b64966c69fb63153e827bc51838b5bf6476685\": container with ID starting with 809b761ab3e8df63a182a057c6b64966c69fb63153e827bc51838b5bf6476685 not found: ID does not exist" containerID="809b761ab3e8df63a182a057c6b64966c69fb63153e827bc51838b5bf6476685"
Sep 11 04:56:48.100787 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.100708    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"809b761ab3e8df63a182a057c6b64966c69fb63153e827bc51838b5bf6476685"} err="failed to get container status \"809b761ab3e8df63a182a057c6b64966c69fb63153e827bc51838b5bf6476685\": rpc error: code = NotFound desc = could not find container \"809b761ab3e8df63a182a057c6b64966c69fb63153e827bc51838b5bf6476685\": container with ID starting with 809b761ab3e8df63a182a057c6b64966c69fb63153e827bc51838b5bf6476685 not found: ID does not exist"
Sep 11 04:56:48.130034 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.130015    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76435f19-2fb3-48ff-ae00-b3488ea0323d-plugins-config-volume\") pod \"76435f19-2fb3-48ff-ae00-b3488ea0323d\" (UID: \"76435f19-2fb3-48ff-ae00-b3488ea0323d\") "
Sep 11 04:56:48.130148 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.130139    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngc7t\" (UniqueName: \"kubernetes.io/projected/76435f19-2fb3-48ff-ae00-b3488ea0323d-kube-api-access-ngc7t\") pod \"76435f19-2fb3-48ff-ae00-b3488ea0323d\" (UID: \"76435f19-2fb3-48ff-ae00-b3488ea0323d\") "
Sep 11 04:56:48.130192 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.130168    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6c6ce18-02b2-4983-81b6-c4ab71d15504-plugins-config-volume\") pod \"c6c6ce18-02b2-4983-81b6-c4ab71d15504\" (UID: \"c6c6ce18-02b2-4983-81b6-c4ab71d15504\") "
Sep 11 04:56:48.130225 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.130212    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qq8gs\" (UniqueName: \"kubernetes.io/projected/c6c6ce18-02b2-4983-81b6-c4ab71d15504-kube-api-access-qq8gs\") pod \"c6c6ce18-02b2-4983-81b6-c4ab71d15504\" (UID: \"c6c6ce18-02b2-4983-81b6-c4ab71d15504\") "
Sep 11 04:56:48.130443 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.130414    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76435f19-2fb3-48ff-ae00-b3488ea0323d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "76435f19-2fb3-48ff-ae00-b3488ea0323d" (UID: "76435f19-2fb3-48ff-ae00-b3488ea0323d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:56:48.130521 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.130502    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6c6ce18-02b2-4983-81b6-c4ab71d15504-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c6c6ce18-02b2-4983-81b6-c4ab71d15504" (UID: "c6c6ce18-02b2-4983-81b6-c4ab71d15504"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:56:48.132277 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.132252    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76435f19-2fb3-48ff-ae00-b3488ea0323d-kube-api-access-ngc7t" (OuterVolumeSpecName: "kube-api-access-ngc7t") pod "76435f19-2fb3-48ff-ae00-b3488ea0323d" (UID: "76435f19-2fb3-48ff-ae00-b3488ea0323d"). InnerVolumeSpecName "kube-api-access-ngc7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:56:48.132434 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.132416    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6c6ce18-02b2-4983-81b6-c4ab71d15504-kube-api-access-qq8gs" (OuterVolumeSpecName: "kube-api-access-qq8gs") pod "c6c6ce18-02b2-4983-81b6-c4ab71d15504" (UID: "c6c6ce18-02b2-4983-81b6-c4ab71d15504"). InnerVolumeSpecName "kube-api-access-qq8gs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:56:48.211886 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.211851    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:48.231040 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.231015    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6c6ce18-02b2-4983-81b6-c4ab71d15504-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:56:48.231040 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.231036    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qq8gs\" (UniqueName: \"kubernetes.io/projected/c6c6ce18-02b2-4983-81b6-c4ab71d15504-kube-api-access-qq8gs\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:56:48.231211 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.231048    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76435f19-2fb3-48ff-ae00-b3488ea0323d-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:56:48.231211 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.231058    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngc7t\" (UniqueName: \"kubernetes.io/projected/76435f19-2fb3-48ff-ae00-b3488ea0323d-kube-api-access-ngc7t\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:56:48.406651 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.406623    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6"]
Sep 11 04:56:48.411001 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.410970    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-7bdb3c-6dd998585d-2b8n6"]
Sep 11 04:56:48.420542 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.420517    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq"]
Sep 11 04:56:48.426213 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:48.426186    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-7bdb3c-6bdbbc9c4f-rdzsq"]
Sep 11 04:56:50.018027 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:50.017991    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76435f19-2fb3-48ff-ae00-b3488ea0323d" path="/var/lib/kubelet/pods/76435f19-2fb3-48ff-ae00-b3488ea0323d/volumes"
Sep 11 04:56:50.018437 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:50.018415    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6c6ce18-02b2-4983-81b6-c4ab71d15504" path="/var/lib/kubelet/pods/c6c6ce18-02b2-4983-81b6-c4ab71d15504/volumes"
Sep 11 04:56:53.899669 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:53.899633    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:54.013523 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:54.013493    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:54.178286 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:54.178189    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:54.703041 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:54.703005    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:55.008975 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:55.008942    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:55.409668 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:55.409580    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:57.076114 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:57.076085    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2"
Sep 11 04:56:57.076501 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:57.076167    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h"
Sep 11 04:56:57.777712 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:57.777675    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:57.869284 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:57.869240    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:59.043364 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.043326    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"]
Sep 11 04:56:59.043993 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.043973    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76435f19-2fb3-48ff-ae00-b3488ea0323d" containerName="payload-pre-processing"
Sep 11 04:56:59.044095 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.043998    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="76435f19-2fb3-48ff-ae00-b3488ea0323d" containerName="payload-pre-processing"
Sep 11 04:56:59.044095 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.044039    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6c6ce18-02b2-4983-81b6-c4ab71d15504" containerName="payload-processing"
Sep 11 04:56:59.044095 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.044048    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6c6ce18-02b2-4983-81b6-c4ab71d15504" containerName="payload-processing"
Sep 11 04:56:59.044224 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.044177    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6c6ce18-02b2-4983-81b6-c4ab71d15504" containerName="payload-processing"
Sep 11 04:56:59.044224 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.044193    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="76435f19-2fb3-48ff-ae00-b3488ea0323d" containerName="payload-pre-processing"
Sep 11 04:56:59.053377 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.053351    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"
Sep 11 04:56:59.057078 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.057035    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"]
Sep 11 04:56:59.057368 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.057344    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 04:56:59.058346 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.058321    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-74gzc\""
Sep 11 04:56:59.058736 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.058717    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-c1858a-serving-cert\""
Sep 11 04:56:59.079997 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.079953    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:59.133145 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.133117    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:59.133503 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.133484    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpkkp\" (UniqueName: \"kubernetes.io/projected/7dfc98bf-1183-4a35-93e7-d57204bed9d4-kube-api-access-rpkkp\") pod \"maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk\" (UID: \"7dfc98bf-1183-4a35-93e7-d57204bed9d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"
Sep 11 04:56:59.133579 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.133546    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7dfc98bf-1183-4a35-93e7-d57204bed9d4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk\" (UID: \"7dfc98bf-1183-4a35-93e7-d57204bed9d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"
Sep 11 04:56:59.133633 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.133587    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dfc98bf-1183-4a35-93e7-d57204bed9d4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk\" (UID: \"7dfc98bf-1183-4a35-93e7-d57204bed9d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"
Sep 11 04:56:59.133633 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.133624    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7dfc98bf-1183-4a35-93e7-d57204bed9d4-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk\" (UID: \"7dfc98bf-1183-4a35-93e7-d57204bed9d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"
Sep 11 04:56:59.234967 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.234930    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpkkp\" (UniqueName: \"kubernetes.io/projected/7dfc98bf-1183-4a35-93e7-d57204bed9d4-kube-api-access-rpkkp\") pod \"maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk\" (UID: \"7dfc98bf-1183-4a35-93e7-d57204bed9d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"
Sep 11 04:56:59.235155 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.234992    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7dfc98bf-1183-4a35-93e7-d57204bed9d4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk\" (UID: \"7dfc98bf-1183-4a35-93e7-d57204bed9d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"
Sep 11 04:56:59.235155 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.235032    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dfc98bf-1183-4a35-93e7-d57204bed9d4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk\" (UID: \"7dfc98bf-1183-4a35-93e7-d57204bed9d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"
Sep 11 04:56:59.235155 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.235060    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7dfc98bf-1183-4a35-93e7-d57204bed9d4-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk\" (UID: \"7dfc98bf-1183-4a35-93e7-d57204bed9d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"
Sep 11 04:56:59.235155 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:56:59.235110    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-serving-cert: secret "maas-api-e2e-shared-a-w3-c1858a-serving-cert" not found
Sep 11 04:56:59.235344 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:56:59.235175    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dfc98bf-1183-4a35-93e7-d57204bed9d4-maas-api-tls podName:7dfc98bf-1183-4a35-93e7-d57204bed9d4 nodeName:}" failed. No retries permitted until 2026-09-11 04:56:59.735155069 +0000 UTC m=+1034.206618946 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7dfc98bf-1183-4a35-93e7-d57204bed9d4-maas-api-tls") pod "maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk" (UID: "7dfc98bf-1183-4a35-93e7-d57204bed9d4") : secret "maas-api-e2e-shared-a-w3-c1858a-serving-cert" not found
Sep 11 04:56:59.235671 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.235652    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dfc98bf-1183-4a35-93e7-d57204bed9d4-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk\" (UID: \"7dfc98bf-1183-4a35-93e7-d57204bed9d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"
Sep 11 04:56:59.237726 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.237702    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7dfc98bf-1183-4a35-93e7-d57204bed9d4-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk\" (UID: \"7dfc98bf-1183-4a35-93e7-d57204bed9d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"
Sep 11 04:56:59.245891 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.245872    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpkkp\" (UniqueName: \"kubernetes.io/projected/7dfc98bf-1183-4a35-93e7-d57204bed9d4-kube-api-access-rpkkp\") pod \"maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk\" (UID: \"7dfc98bf-1183-4a35-93e7-d57204bed9d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"
Sep 11 04:56:59.387126 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.387031    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:59.735191 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.735090    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:59.740337 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.740318    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7dfc98bf-1183-4a35-93e7-d57204bed9d4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk\" (UID: \"7dfc98bf-1183-4a35-93e7-d57204bed9d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"
Sep 11 04:56:59.743231 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.743207    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7dfc98bf-1183-4a35-93e7-d57204bed9d4-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk\" (UID: \"7dfc98bf-1183-4a35-93e7-d57204bed9d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"
Sep 11 04:56:59.832839 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.832804    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:56:59.867852 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.867821    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp"]
Sep 11 04:56:59.871397 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.871382    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp"
Sep 11 04:56:59.873854 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.873822    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-c1858a\""
Sep 11 04:56:59.873971 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.873881    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-c1858a-dockercfg-wq2jl\""
Sep 11 04:56:59.880227 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.880204    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp"]
Sep 11 04:56:59.941412 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.941382    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v9l8\" (UniqueName: \"kubernetes.io/projected/12178127-9faf-4eda-b496-a8e4766592da-kube-api-access-4v9l8\") pod \"payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp\" (UID: \"12178127-9faf-4eda-b496-a8e4766592da\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp"
Sep 11 04:56:59.941412 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.941421    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12178127-9faf-4eda-b496-a8e4766592da-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp\" (UID: \"12178127-9faf-4eda-b496-a8e4766592da\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp"
Sep 11 04:56:59.969262 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:56:59.969217    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"
Sep 11 04:57:00.044224 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:00.043766    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v9l8\" (UniqueName: \"kubernetes.io/projected/12178127-9faf-4eda-b496-a8e4766592da-kube-api-access-4v9l8\") pod \"payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp\" (UID: \"12178127-9faf-4eda-b496-a8e4766592da\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp"
Sep 11 04:57:00.044224 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:00.043821    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12178127-9faf-4eda-b496-a8e4766592da-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp\" (UID: \"12178127-9faf-4eda-b496-a8e4766592da\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp"
Sep 11 04:57:00.044831 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:00.044804    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12178127-9faf-4eda-b496-a8e4766592da-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp\" (UID: \"12178127-9faf-4eda-b496-a8e4766592da\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp"
Sep 11 04:57:00.052488 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:00.052455    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v9l8\" (UniqueName: \"kubernetes.io/projected/12178127-9faf-4eda-b496-a8e4766592da-kube-api-access-4v9l8\") pod \"payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp\" (UID: \"12178127-9faf-4eda-b496-a8e4766592da\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp"
Sep 11 04:57:00.183615 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:00.183580    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp"
Sep 11 04:57:00.244611 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:00.244562    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"]
Sep 11 04:57:00.248482 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:57:00.248143    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dfc98bf_1183_4a35_93e7_d57204bed9d4.slice/crio-4c848fdb2f23a206463c83c1779a9e3f29f039816dab6e53c3550147a8f49214 WatchSource:0}: Error finding container 4c848fdb2f23a206463c83c1779a9e3f29f039816dab6e53c3550147a8f49214: Status 404 returned error can't find the container with id 4c848fdb2f23a206463c83c1779a9e3f29f039816dab6e53c3550147a8f49214
Sep 11 04:57:00.456008 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:00.455933    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp"]
Sep 11 04:57:00.458971 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:57:00.458948    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12178127_9faf_4eda_b496_a8e4766592da.slice/crio-8993757841e3e0a3852dd3490a4158ea389d2b39ec412698f1664d86e18fe1e8 WatchSource:0}: Error finding container 8993757841e3e0a3852dd3490a4158ea389d2b39ec412698f1664d86e18fe1e8: Status 404 returned error can't find the container with id 8993757841e3e0a3852dd3490a4158ea389d2b39ec412698f1664d86e18fe1e8
Sep 11 04:57:01.152912 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:01.152879    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk" event={"ID":"7dfc98bf-1183-4a35-93e7-d57204bed9d4","Type":"ContainerStarted","Data":"25a043661e4b9062f52aeac2e548c0cfdbfb9893efde89b6c88b35b6c5152a83"}
Sep 11 04:57:01.153394 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:01.152944    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"
Sep 11 04:57:01.153394 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:01.152969    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk" event={"ID":"7dfc98bf-1183-4a35-93e7-d57204bed9d4","Type":"ContainerStarted","Data":"4c848fdb2f23a206463c83c1779a9e3f29f039816dab6e53c3550147a8f49214"}
Sep 11 04:57:01.154788 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:01.154759    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp" event={"ID":"12178127-9faf-4eda-b496-a8e4766592da","Type":"ContainerStarted","Data":"a16b3a4656abb183ebac85d78f193655bde281f3a106d4e8f9bbc077f13d1690"}
Sep 11 04:57:01.154884 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:01.154814    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp" event={"ID":"12178127-9faf-4eda-b496-a8e4766592da","Type":"ContainerStarted","Data":"8993757841e3e0a3852dd3490a4158ea389d2b39ec412698f1664d86e18fe1e8"}
Sep 11 04:57:01.155351 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:01.155333    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp"
Sep 11 04:57:01.170648 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:01.170603    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk" podStartSLOduration=1.8625240280000002 podStartE2EDuration="2.170590586s" totalImagesPullingTime="308.066558ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:56:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:57:01.169546298 +0000 UTC m=+1035.641010184" watchObservedRunningTime="2026-09-11 04:57:01.170590586 +0000 UTC m=+1035.642054461"
Sep 11 04:57:01.184826 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:01.184777    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp" podStartSLOduration=2.184762427 podStartE2EDuration="2.184762427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:56:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:57:01.18402374 +0000 UTC m=+1035.655487616" watchObservedRunningTime="2026-09-11 04:57:01.184762427 +0000 UTC m=+1035.656226302"
Sep 11 04:57:01.497705 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:01.497614    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:01.598007 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:01.597962    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:05.109886 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:05.109849    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:07.166199 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:07.166171    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"
Sep 11 04:57:07.972565 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:07.972528    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:08.614563 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:08.614512    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:08.710751 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:08.710718    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:09.017953 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:09.017925    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:13.167219 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:13.167191    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp"
Sep 11 04:57:14.966115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:14.966051    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:15.064972 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:15.064927    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:20.977792 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:20.977749    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:22.724415 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:22.724344    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:23.036465 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:23.036434    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:23.336994 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:23.336866    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:23.750882 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:23.750791    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:25.377612 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:25.377570    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:25.928524 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:25.928496    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:25.953708 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:25.952736    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h"]
Sep 11 04:57:25.954179 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:25.954129    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h" podUID="f3d143c4-ceed-4d30-a7ee-aab7b515885d" containerName="payload-processing" containerID="cri-o://8925612361d22bced1a7b562f7af59cfa4f221e1a6c42d62ad21a6317c4cd443" gracePeriod=30
Sep 11 04:57:25.957579 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:25.956920    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2"]
Sep 11 04:57:25.957579 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:25.957352    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2" podUID="e4e673c5-8e82-4457-8ed7-c00f85cd0024" containerName="payload-pre-processing" containerID="cri-o://afb33a4732b4eea448fab71ebe0d0ef9d148e3a107e470f751f0d9b833ad7c85" gracePeriod=30
Sep 11 04:57:26.002090 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.002026    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:26.086611 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.086572    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:26.144383 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.144351    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:26.239960 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.239938    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h"
Sep 11 04:57:26.242895 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.242874    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2"
Sep 11 04:57:26.292948 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.292914    2567 generic.go:356] "Generic (PLEG): container finished" podID="f3d143c4-ceed-4d30-a7ee-aab7b515885d" containerID="8925612361d22bced1a7b562f7af59cfa4f221e1a6c42d62ad21a6317c4cd443" exitCode=0
Sep 11 04:57:26.293172 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.292998    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h"
Sep 11 04:57:26.293172 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.293026    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h" event={"ID":"f3d143c4-ceed-4d30-a7ee-aab7b515885d","Type":"ContainerDied","Data":"8925612361d22bced1a7b562f7af59cfa4f221e1a6c42d62ad21a6317c4cd443"}
Sep 11 04:57:26.293172 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.293102    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h" event={"ID":"f3d143c4-ceed-4d30-a7ee-aab7b515885d","Type":"ContainerDied","Data":"48a3764a40d0445bf554168b7834c68ca3a8a5d021cd3b5f2c094c2645af4a10"}
Sep 11 04:57:26.293172 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.293135    2567 scope.go:117] "RemoveContainer" containerID="8925612361d22bced1a7b562f7af59cfa4f221e1a6c42d62ad21a6317c4cd443"
Sep 11 04:57:26.294477 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.294455    2567 generic.go:356] "Generic (PLEG): container finished" podID="e4e673c5-8e82-4457-8ed7-c00f85cd0024" containerID="afb33a4732b4eea448fab71ebe0d0ef9d148e3a107e470f751f0d9b833ad7c85" exitCode=0
Sep 11 04:57:26.294602 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.294486    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2" event={"ID":"e4e673c5-8e82-4457-8ed7-c00f85cd0024","Type":"ContainerDied","Data":"afb33a4732b4eea448fab71ebe0d0ef9d148e3a107e470f751f0d9b833ad7c85"}
Sep 11 04:57:26.294602 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.294521    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2" event={"ID":"e4e673c5-8e82-4457-8ed7-c00f85cd0024","Type":"ContainerDied","Data":"79e694d21fc1b4f3cc1861ac9dd9b6e0ed89e0b75eef603773c923eb255daf21"}
Sep 11 04:57:26.294602 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.294532    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2"
Sep 11 04:57:26.299249 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.299227    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b8572\" (UniqueName: \"kubernetes.io/projected/e4e673c5-8e82-4457-8ed7-c00f85cd0024-kube-api-access-b8572\") pod \"e4e673c5-8e82-4457-8ed7-c00f85cd0024\" (UID: \"e4e673c5-8e82-4457-8ed7-c00f85cd0024\") "
Sep 11 04:57:26.299394 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.299261    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tjjf\" (UniqueName: \"kubernetes.io/projected/f3d143c4-ceed-4d30-a7ee-aab7b515885d-kube-api-access-9tjjf\") pod \"f3d143c4-ceed-4d30-a7ee-aab7b515885d\" (UID: \"f3d143c4-ceed-4d30-a7ee-aab7b515885d\") "
Sep 11 04:57:26.299394 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.299316    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3d143c4-ceed-4d30-a7ee-aab7b515885d-plugins-config-volume\") pod \"f3d143c4-ceed-4d30-a7ee-aab7b515885d\" (UID: \"f3d143c4-ceed-4d30-a7ee-aab7b515885d\") "
Sep 11 04:57:26.299514 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.299393    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4e673c5-8e82-4457-8ed7-c00f85cd0024-plugins-config-volume\") pod \"e4e673c5-8e82-4457-8ed7-c00f85cd0024\" (UID: \"e4e673c5-8e82-4457-8ed7-c00f85cd0024\") "
Sep 11 04:57:26.299719 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.299690    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3d143c4-ceed-4d30-a7ee-aab7b515885d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f3d143c4-ceed-4d30-a7ee-aab7b515885d" (UID: "f3d143c4-ceed-4d30-a7ee-aab7b515885d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:57:26.299829 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.299728    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4e673c5-8e82-4457-8ed7-c00f85cd0024-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e4e673c5-8e82-4457-8ed7-c00f85cd0024" (UID: "e4e673c5-8e82-4457-8ed7-c00f85cd0024"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 04:57:26.302373 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.302328    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3d143c4-ceed-4d30-a7ee-aab7b515885d-kube-api-access-9tjjf" (OuterVolumeSpecName: "kube-api-access-9tjjf") pod "f3d143c4-ceed-4d30-a7ee-aab7b515885d" (UID: "f3d143c4-ceed-4d30-a7ee-aab7b515885d"). InnerVolumeSpecName "kube-api-access-9tjjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:57:26.303877 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.303762    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4e673c5-8e82-4457-8ed7-c00f85cd0024-kube-api-access-b8572" (OuterVolumeSpecName: "kube-api-access-b8572") pod "e4e673c5-8e82-4457-8ed7-c00f85cd0024" (UID: "e4e673c5-8e82-4457-8ed7-c00f85cd0024"). InnerVolumeSpecName "kube-api-access-b8572". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 04:57:26.305956 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.305936    2567 scope.go:117] "RemoveContainer" containerID="8925612361d22bced1a7b562f7af59cfa4f221e1a6c42d62ad21a6317c4cd443"
Sep 11 04:57:26.306265 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:57:26.306244    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8925612361d22bced1a7b562f7af59cfa4f221e1a6c42d62ad21a6317c4cd443\": container with ID starting with 8925612361d22bced1a7b562f7af59cfa4f221e1a6c42d62ad21a6317c4cd443 not found: ID does not exist" containerID="8925612361d22bced1a7b562f7af59cfa4f221e1a6c42d62ad21a6317c4cd443"
Sep 11 04:57:26.306329 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.306275    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8925612361d22bced1a7b562f7af59cfa4f221e1a6c42d62ad21a6317c4cd443"} err="failed to get container status \"8925612361d22bced1a7b562f7af59cfa4f221e1a6c42d62ad21a6317c4cd443\": rpc error: code = NotFound desc = could not find container \"8925612361d22bced1a7b562f7af59cfa4f221e1a6c42d62ad21a6317c4cd443\": container with ID starting with 8925612361d22bced1a7b562f7af59cfa4f221e1a6c42d62ad21a6317c4cd443 not found: ID does not exist"
Sep 11 04:57:26.306329 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.306291    2567 scope.go:117] "RemoveContainer" containerID="afb33a4732b4eea448fab71ebe0d0ef9d148e3a107e470f751f0d9b833ad7c85"
Sep 11 04:57:26.314386 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.314354    2567 scope.go:117] "RemoveContainer" containerID="afb33a4732b4eea448fab71ebe0d0ef9d148e3a107e470f751f0d9b833ad7c85"
Sep 11 04:57:26.314632 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:57:26.314613    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afb33a4732b4eea448fab71ebe0d0ef9d148e3a107e470f751f0d9b833ad7c85\": container with ID starting with afb33a4732b4eea448fab71ebe0d0ef9d148e3a107e470f751f0d9b833ad7c85 not found: ID does not exist" containerID="afb33a4732b4eea448fab71ebe0d0ef9d148e3a107e470f751f0d9b833ad7c85"
Sep 11 04:57:26.314718 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.314638    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afb33a4732b4eea448fab71ebe0d0ef9d148e3a107e470f751f0d9b833ad7c85"} err="failed to get container status \"afb33a4732b4eea448fab71ebe0d0ef9d148e3a107e470f751f0d9b833ad7c85\": rpc error: code = NotFound desc = could not find container \"afb33a4732b4eea448fab71ebe0d0ef9d148e3a107e470f751f0d9b833ad7c85\": container with ID starting with afb33a4732b4eea448fab71ebe0d0ef9d148e3a107e470f751f0d9b833ad7c85 not found: ID does not exist"
Sep 11 04:57:26.400844 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.400812    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3d143c4-ceed-4d30-a7ee-aab7b515885d-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:57:26.400844 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.400837    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e4e673c5-8e82-4457-8ed7-c00f85cd0024-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:57:26.400844 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.400849    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b8572\" (UniqueName: \"kubernetes.io/projected/e4e673c5-8e82-4457-8ed7-c00f85cd0024-kube-api-access-b8572\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:57:26.401304 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.400858    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tjjf\" (UniqueName: \"kubernetes.io/projected/f3d143c4-ceed-4d30-a7ee-aab7b515885d-kube-api-access-9tjjf\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 04:57:26.424715 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.424642    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:26.498606 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.498522    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:26.621780 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.621751    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h"]
Sep 11 04:57:26.629980 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.629958    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-ca9fd5b9-676f66498f-xvl2h"]
Sep 11 04:57:26.640357 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.640335    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2"]
Sep 11 04:57:26.645114 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:26.645091    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-ca9fd5b9-5fcd9c9c94-rn8l2"]
Sep 11 04:57:27.649549 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.649512    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"]
Sep 11 04:57:27.650135 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.650119    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4e673c5-8e82-4457-8ed7-c00f85cd0024" containerName="payload-pre-processing"
Sep 11 04:57:27.650192 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.650139    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4e673c5-8e82-4457-8ed7-c00f85cd0024" containerName="payload-pre-processing"
Sep 11 04:57:27.650192 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.650173    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3d143c4-ceed-4d30-a7ee-aab7b515885d" containerName="payload-processing"
Sep 11 04:57:27.650192 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.650182    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3d143c4-ceed-4d30-a7ee-aab7b515885d" containerName="payload-processing"
Sep 11 04:57:27.650282 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.650271    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3d143c4-ceed-4d30-a7ee-aab7b515885d" containerName="payload-processing"
Sep 11 04:57:27.650316 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.650285    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4e673c5-8e82-4457-8ed7-c00f85cd0024" containerName="payload-pre-processing"
Sep 11 04:57:27.655147 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.655120    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"
Sep 11 04:57:27.657783 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.657765    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-16b799-serving-cert\""
Sep 11 04:57:27.664419 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.664390    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"]
Sep 11 04:57:27.686742 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.686713    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:27.710941 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.710909    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj\" (UID: \"e71f04c8-1bc1-463e-a6ab-111adbca0b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"
Sep 11 04:57:27.711108 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.710973    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqm8c\" (UniqueName: \"kubernetes.io/projected/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-kube-api-access-wqm8c\") pod \"maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj\" (UID: \"e71f04c8-1bc1-463e-a6ab-111adbca0b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"
Sep 11 04:57:27.711108 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.710998    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj\" (UID: \"e71f04c8-1bc1-463e-a6ab-111adbca0b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"
Sep 11 04:57:27.711108 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.711100    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj\" (UID: \"e71f04c8-1bc1-463e-a6ab-111adbca0b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"
Sep 11 04:57:27.785010 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.784972    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:27.811642 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.811612    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqm8c\" (UniqueName: \"kubernetes.io/projected/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-kube-api-access-wqm8c\") pod \"maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj\" (UID: \"e71f04c8-1bc1-463e-a6ab-111adbca0b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"
Sep 11 04:57:27.811782 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.811647    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj\" (UID: \"e71f04c8-1bc1-463e-a6ab-111adbca0b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"
Sep 11 04:57:27.811782 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.811694    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj\" (UID: \"e71f04c8-1bc1-463e-a6ab-111adbca0b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"
Sep 11 04:57:27.811782 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.811713    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj\" (UID: \"e71f04c8-1bc1-463e-a6ab-111adbca0b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"
Sep 11 04:57:27.812399 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.812374    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj\" (UID: \"e71f04c8-1bc1-463e-a6ab-111adbca0b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"
Sep 11 04:57:27.814270 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.814248    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj\" (UID: \"e71f04c8-1bc1-463e-a6ab-111adbca0b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"
Sep 11 04:57:27.814347 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.814332    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj\" (UID: \"e71f04c8-1bc1-463e-a6ab-111adbca0b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"
Sep 11 04:57:27.819320 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.819299    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqm8c\" (UniqueName: \"kubernetes.io/projected/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-kube-api-access-wqm8c\") pod \"maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj\" (UID: \"e71f04c8-1bc1-463e-a6ab-111adbca0b2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"
Sep 11 04:57:27.888655 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.888606    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:27.968606 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:27.968521    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"
Sep 11 04:57:28.018318 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:28.018284    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4e673c5-8e82-4457-8ed7-c00f85cd0024" path="/var/lib/kubelet/pods/e4e673c5-8e82-4457-8ed7-c00f85cd0024/volumes"
Sep 11 04:57:28.018735 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:28.018720    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3d143c4-ceed-4d30-a7ee-aab7b515885d" path="/var/lib/kubelet/pods/f3d143c4-ceed-4d30-a7ee-aab7b515885d/volumes"
Sep 11 04:57:28.127696 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:28.127658    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:28.249489 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:28.249408    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"]
Sep 11 04:57:28.253699 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:57:28.253665    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode71f04c8_1bc1_463e_a6ab_111adbca0b2d.slice/crio-42a3d2aa40770f164b66c73a6e26b5162c0c0a969ff3a8f5161cc9ac7cfbf836 WatchSource:0}: Error finding container 42a3d2aa40770f164b66c73a6e26b5162c0c0a969ff3a8f5161cc9ac7cfbf836: Status 404 returned error can't find the container with id 42a3d2aa40770f164b66c73a6e26b5162c0c0a969ff3a8f5161cc9ac7cfbf836
Sep 11 04:57:28.306439 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:28.306403    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj" event={"ID":"e71f04c8-1bc1-463e-a6ab-111adbca0b2d","Type":"ContainerStarted","Data":"42a3d2aa40770f164b66c73a6e26b5162c0c0a969ff3a8f5161cc9ac7cfbf836"}
Sep 11 04:57:29.222774 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:29.222743    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:29.313239 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:29.313207    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj" event={"ID":"e71f04c8-1bc1-463e-a6ab-111adbca0b2d","Type":"ContainerStarted","Data":"24019fdab57084b256da74c9d0f479cb1b3df5708b0e856aee0f2df23efcec01"}
Sep 11 04:57:29.313414 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:29.313321    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"
Sep 11 04:57:29.330962 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:29.330918    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj" podStartSLOduration=1.9853919150000001 podStartE2EDuration="2.330902779s" totalImagesPullingTime="345.510864ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:57:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:57:29.328298061 +0000 UTC m=+1063.799761933" watchObservedRunningTime="2026-09-11 04:57:29.330902779 +0000 UTC m=+1063.802366655"
Sep 11 04:57:32.431362 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:32.431321    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:32.482821 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:32.482788    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:32.682800 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:32.682720    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:33.038077 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:33.038027    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:33.138956 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:33.138919    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:33.236188 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:33.236160    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:35.324105 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:35.324053    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"
Sep 11 04:57:38.852855 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:38.852762    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:38.966628 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:38.966577    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:39.052791 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:39.052732    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:39.182510 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:39.182421    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:45.017029 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:45.016999    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:45.078184 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:45.078147    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:45.913523 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:45.913494    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:46.213921 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:46.213798    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:47.271321 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:47.271259    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:49.496431 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:49.496387    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:49.616451 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:49.616422    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:50.600961 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:50.600927    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"]
Sep 11 04:57:50.606782 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:50.606758    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"
Sep 11 04:57:50.609175 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:50.609154    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-f5e6f7-serving-cert\""
Sep 11 04:57:50.617866 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:50.617841    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"]
Sep 11 04:57:50.640562 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:50.640527    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:50.717028 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:50.717001    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:50.723717 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:50.723692    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx7kv\" (UniqueName: \"kubernetes.io/projected/75ed05fd-87c0-4c62-91da-99bebc7445d8-kube-api-access-wx7kv\") pod \"maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4\" (UID: \"75ed05fd-87c0-4c62-91da-99bebc7445d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"
Sep 11 04:57:50.723860 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:50.723730    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75ed05fd-87c0-4c62-91da-99bebc7445d8-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4\" (UID: \"75ed05fd-87c0-4c62-91da-99bebc7445d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"
Sep 11 04:57:50.723860 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:50.723774    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75ed05fd-87c0-4c62-91da-99bebc7445d8-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4\" (UID: \"75ed05fd-87c0-4c62-91da-99bebc7445d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"
Sep 11 04:57:50.723860 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:50.723802    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75ed05fd-87c0-4c62-91da-99bebc7445d8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4\" (UID: \"75ed05fd-87c0-4c62-91da-99bebc7445d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"
Sep 11 04:57:50.825024 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:50.824992    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx7kv\" (UniqueName: \"kubernetes.io/projected/75ed05fd-87c0-4c62-91da-99bebc7445d8-kube-api-access-wx7kv\") pod \"maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4\" (UID: \"75ed05fd-87c0-4c62-91da-99bebc7445d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"
Sep 11 04:57:50.825192 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:50.825030    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75ed05fd-87c0-4c62-91da-99bebc7445d8-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4\" (UID: \"75ed05fd-87c0-4c62-91da-99bebc7445d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"
Sep 11 04:57:50.825269 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:50.825244    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75ed05fd-87c0-4c62-91da-99bebc7445d8-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4\" (UID: \"75ed05fd-87c0-4c62-91da-99bebc7445d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"
Sep 11 04:57:50.825326 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:50.825309    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75ed05fd-87c0-4c62-91da-99bebc7445d8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4\" (UID: \"75ed05fd-87c0-4c62-91da-99bebc7445d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"
Sep 11 04:57:50.825392 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:57:50.825375    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-serving-cert: secret "maas-api-e2e-shared-a-w2-f5e6f7-serving-cert" not found
Sep 11 04:57:50.825457 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:57:50.825444    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75ed05fd-87c0-4c62-91da-99bebc7445d8-maas-api-tls podName:75ed05fd-87c0-4c62-91da-99bebc7445d8 nodeName:}" failed. No retries permitted until 2026-09-11 04:57:51.32542264 +0000 UTC m=+1085.796886498 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/75ed05fd-87c0-4c62-91da-99bebc7445d8-maas-api-tls") pod "maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4" (UID: "75ed05fd-87c0-4c62-91da-99bebc7445d8") : secret "maas-api-e2e-shared-a-w2-f5e6f7-serving-cert" not found
Sep 11 04:57:50.826106 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:50.826060    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75ed05fd-87c0-4c62-91da-99bebc7445d8-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4\" (UID: \"75ed05fd-87c0-4c62-91da-99bebc7445d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"
Sep 11 04:57:50.827691 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:50.827662    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75ed05fd-87c0-4c62-91da-99bebc7445d8-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4\" (UID: \"75ed05fd-87c0-4c62-91da-99bebc7445d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"
Sep 11 04:57:50.833613 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:50.833584    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx7kv\" (UniqueName: \"kubernetes.io/projected/75ed05fd-87c0-4c62-91da-99bebc7445d8-kube-api-access-wx7kv\") pod \"maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4\" (UID: \"75ed05fd-87c0-4c62-91da-99bebc7445d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"
Sep 11 04:57:51.106506 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:51.106474    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:51.244322 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:51.244282    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:51.329539 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:51.329500    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75ed05fd-87c0-4c62-91da-99bebc7445d8-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4\" (UID: \"75ed05fd-87c0-4c62-91da-99bebc7445d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"
Sep 11 04:57:51.332039 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:51.332022    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75ed05fd-87c0-4c62-91da-99bebc7445d8-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4\" (UID: \"75ed05fd-87c0-4c62-91da-99bebc7445d8\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"
Sep 11 04:57:51.336985 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:51.336962    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk"]
Sep 11 04:57:51.339456 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:51.339430    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:57:51.340742 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:51.340727    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk"
Sep 11 04:57:51.343602 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:51.343577    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-f5e6f7\""
Sep 11 04:57:51.343713 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:51.343605    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-f5e6f7-dockercfg-wdsrt\""
Sep 11 04:57:51.354920 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:51.354899    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk"]
Sep 11 04:57:51.431144 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:51.431022    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44655566-d3ce-46d4-94c3-fd2123ffbe8d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk\" (UID: \"44655566-d3ce-46d4-94c3-fd2123ffbe8d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk"
Sep 11 04:57:51.431144 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:51.431093    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg8sx\" (UniqueName: \"kubernetes.io/projected/44655566-d3ce-46d4-94c3-fd2123ffbe8d-kube-api-access-mg8sx\") pod \"payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk\" (UID: \"44655566-d3ce-46d4-94c3-fd2123ffbe8d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk"
Sep 11 04:57:51.518881 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:51.518837    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"
Sep 11 04:57:51.532025 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:51.531999    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg8sx\" (UniqueName: \"kubernetes.io/projected/44655566-d3ce-46d4-94c3-fd2123ffbe8d-kube-api-access-mg8sx\") pod \"payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk\" (UID: \"44655566-d3ce-46d4-94c3-fd2123ffbe8d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk"
Sep 11 04:57:51.532167 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:51.532150    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44655566-d3ce-46d4-94c3-fd2123ffbe8d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk\" (UID: \"44655566-d3ce-46d4-94c3-fd2123ffbe8d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk"
Sep 11 04:57:51.532771 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:51.532752    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44655566-d3ce-46d4-94c3-fd2123ffbe8d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk\" (UID: \"44655566-d3ce-46d4-94c3-fd2123ffbe8d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk"
Sep 11 04:57:51.539830 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:51.539807    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg8sx\" (UniqueName: \"kubernetes.io/projected/44655566-d3ce-46d4-94c3-fd2123ffbe8d-kube-api-access-mg8sx\") pod \"payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk\" (UID: \"44655566-d3ce-46d4-94c3-fd2123ffbe8d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk"
Sep 11 04:57:51.652024 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:51.651991    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk"
Sep 11 04:57:51.790441 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:51.790414    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"]
Sep 11 04:57:51.792482 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:57:51.792448    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75ed05fd_87c0_4c62_91da_99bebc7445d8.slice/crio-0b30880dc538e6845bad6154886a8f9cec2fbe950df0347d50f9e4cd01591d5c WatchSource:0}: Error finding container 0b30880dc538e6845bad6154886a8f9cec2fbe950df0347d50f9e4cd01591d5c: Status 404 returned error can't find the container with id 0b30880dc538e6845bad6154886a8f9cec2fbe950df0347d50f9e4cd01591d5c
Sep 11 04:57:51.923577 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:51.923435    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk"]
Sep 11 04:57:51.925851 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:57:51.925818    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44655566_d3ce_46d4_94c3_fd2123ffbe8d.slice/crio-7c45739b0940aec0ab4fed87536d4da70a6c0e92ec2946f3671c30e5f16d7389 WatchSource:0}: Error finding container 7c45739b0940aec0ab4fed87536d4da70a6c0e92ec2946f3671c30e5f16d7389: Status 404 returned error can't find the container with id 7c45739b0940aec0ab4fed87536d4da70a6c0e92ec2946f3671c30e5f16d7389
Sep 11 04:57:52.432396 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:52.432358    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk" event={"ID":"44655566-d3ce-46d4-94c3-fd2123ffbe8d","Type":"ContainerStarted","Data":"0f3301d2d70befc41af202c1213e974248f855e13210b4f4325b9c86f452f74b"}
Sep 11 04:57:52.432587 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:52.432418    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk" event={"ID":"44655566-d3ce-46d4-94c3-fd2123ffbe8d","Type":"ContainerStarted","Data":"7c45739b0940aec0ab4fed87536d4da70a6c0e92ec2946f3671c30e5f16d7389"}
Sep 11 04:57:52.432587 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:52.432445    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk"
Sep 11 04:57:52.434051 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:52.434017    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4" event={"ID":"75ed05fd-87c0-4c62-91da-99bebc7445d8","Type":"ContainerStarted","Data":"9b962132e34a713fae56562f81f6e68ace5e4d958ac49db874ef295d884c2280"}
Sep 11 04:57:52.434192 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:52.434082    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4" event={"ID":"75ed05fd-87c0-4c62-91da-99bebc7445d8","Type":"ContainerStarted","Data":"0b30880dc538e6845bad6154886a8f9cec2fbe950df0347d50f9e4cd01591d5c"}
Sep 11 04:57:52.434192 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:52.434112    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"
Sep 11 04:57:52.448374 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:52.448328    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk" podStartSLOduration=1.448309806 podStartE2EDuration="1.448309806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:57:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:57:52.446703658 +0000 UTC m=+1086.918167536" watchObservedRunningTime="2026-09-11 04:57:52.448309806 +0000 UTC m=+1086.919773682"
Sep 11 04:57:52.463420 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:52.463376    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4" podStartSLOduration=2.094096862 podStartE2EDuration="2.46336326s" totalImagesPullingTime="369.266398ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:57:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:57:52.461779251 +0000 UTC m=+1086.933243128" watchObservedRunningTime="2026-09-11 04:57:52.46336326 +0000 UTC m=+1086.934827166"
Sep 11 04:57:58.444821 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:57:58.444782    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"
Sep 11 04:58:03.441631 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:03.441599    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk"
Sep 11 04:58:14.432244 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:14.432206    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:14.757271 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:14.757240    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:15.081964 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:15.081856    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:15.725598 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:15.725562    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:17.669782 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:17.669741    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:17.772409 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:17.772371    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:18.861481 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:18.861434    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"]
Sep 11 04:58:18.865503 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:18.865476    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"
Sep 11 04:58:18.867833 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:18.867810    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-f81bf2-serving-cert\""
Sep 11 04:58:18.876530 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:18.876506    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"]
Sep 11 04:58:18.893290 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:18.893261    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:18.947764 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:18.947732    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:18.986926 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:18.986886    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24942598-c316-4136-b9f2-5dda388ae49c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5\" (UID: \"24942598-c316-4136-b9f2-5dda388ae49c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"
Sep 11 04:58:18.987115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:18.986936    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24942598-c316-4136-b9f2-5dda388ae49c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5\" (UID: \"24942598-c316-4136-b9f2-5dda388ae49c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"
Sep 11 04:58:18.987115 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:18.987048    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bbs8\" (UniqueName: \"kubernetes.io/projected/24942598-c316-4136-b9f2-5dda388ae49c-kube-api-access-5bbs8\") pod \"maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5\" (UID: \"24942598-c316-4136-b9f2-5dda388ae49c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"
Sep 11 04:58:18.987214 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:18.987182    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24942598-c316-4136-b9f2-5dda388ae49c-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5\" (UID: \"24942598-c316-4136-b9f2-5dda388ae49c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"
Sep 11 04:58:19.087781 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.087743    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24942598-c316-4136-b9f2-5dda388ae49c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5\" (UID: \"24942598-c316-4136-b9f2-5dda388ae49c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"
Sep 11 04:58:19.087950 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.087798    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24942598-c316-4136-b9f2-5dda388ae49c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5\" (UID: \"24942598-c316-4136-b9f2-5dda388ae49c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"
Sep 11 04:58:19.087950 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.087868    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bbs8\" (UniqueName: \"kubernetes.io/projected/24942598-c316-4136-b9f2-5dda388ae49c-kube-api-access-5bbs8\") pod \"maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5\" (UID: \"24942598-c316-4136-b9f2-5dda388ae49c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"
Sep 11 04:58:19.088053 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.087976    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24942598-c316-4136-b9f2-5dda388ae49c-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5\" (UID: \"24942598-c316-4136-b9f2-5dda388ae49c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"
Sep 11 04:58:19.088053 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:58:19.087987    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-serving-cert: secret "maas-api-e2e-shared-b-w2-f81bf2-serving-cert" not found
Sep 11 04:58:19.088156 ip-10-0-128-18 kubenswrapper[2567]: E0911 04:58:19.088113    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24942598-c316-4136-b9f2-5dda388ae49c-maas-api-tls podName:24942598-c316-4136-b9f2-5dda388ae49c nodeName:}" failed. No retries permitted until 2026-09-11 04:58:19.588059772 +0000 UTC m=+1114.059523646 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/24942598-c316-4136-b9f2-5dda388ae49c-maas-api-tls") pod "maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5" (UID: "24942598-c316-4136-b9f2-5dda388ae49c") : secret "maas-api-e2e-shared-b-w2-f81bf2-serving-cert" not found
Sep 11 04:58:19.088492 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.088472    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24942598-c316-4136-b9f2-5dda388ae49c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5\" (UID: \"24942598-c316-4136-b9f2-5dda388ae49c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"
Sep 11 04:58:19.091181 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.091156    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24942598-c316-4136-b9f2-5dda388ae49c-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5\" (UID: \"24942598-c316-4136-b9f2-5dda388ae49c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"
Sep 11 04:58:19.098577 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.098554    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bbs8\" (UniqueName: \"kubernetes.io/projected/24942598-c316-4136-b9f2-5dda388ae49c-kube-api-access-5bbs8\") pod \"maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5\" (UID: \"24942598-c316-4136-b9f2-5dda388ae49c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"
Sep 11 04:58:19.272759 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.272719    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:19.452544 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.452507    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:19.590997 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.590910    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx"]
Sep 11 04:58:19.592952 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.592931    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24942598-c316-4136-b9f2-5dda388ae49c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5\" (UID: \"24942598-c316-4136-b9f2-5dda388ae49c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"
Sep 11 04:58:19.594793 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.594777    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx"
Sep 11 04:58:19.595627 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.595599    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24942598-c316-4136-b9f2-5dda388ae49c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5\" (UID: \"24942598-c316-4136-b9f2-5dda388ae49c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"
Sep 11 04:58:19.597364 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.597337    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-f81bf2\""
Sep 11 04:58:19.597467 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.597390    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-f81bf2-dockercfg-trsv5\""
Sep 11 04:58:19.607905 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.607877    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx"]
Sep 11 04:58:19.613484 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.613460    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:19.693500 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.693462    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4gg4\" (UniqueName: \"kubernetes.io/projected/8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02-kube-api-access-k4gg4\") pod \"payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx\" (UID: \"8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx"
Sep 11 04:58:19.693684 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.693524    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx\" (UID: \"8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx"
Sep 11 04:58:19.780647 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.780612    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"
Sep 11 04:58:19.794854 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.794829    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4gg4\" (UniqueName: \"kubernetes.io/projected/8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02-kube-api-access-k4gg4\") pod \"payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx\" (UID: \"8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx"
Sep 11 04:58:19.794968 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.794887    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx\" (UID: \"8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx"
Sep 11 04:58:19.795503 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.795482    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx\" (UID: \"8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx"
Sep 11 04:58:19.804179 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.804162    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4gg4\" (UniqueName: \"kubernetes.io/projected/8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02-kube-api-access-k4gg4\") pod \"payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx\" (UID: \"8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx"
Sep 11 04:58:19.914656 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:19.914567    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx"
Sep 11 04:58:20.053277 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:20.053253    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"]
Sep 11 04:58:20.055139 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:58:20.055117    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24942598_c316_4136_b9f2_5dda388ae49c.slice/crio-782806987d547cb06d92d8ffe72ac1af7dc9d8c52a6037c9e655b62e81b4b610 WatchSource:0}: Error finding container 782806987d547cb06d92d8ffe72ac1af7dc9d8c52a6037c9e655b62e81b4b610: Status 404 returned error can't find the container with id 782806987d547cb06d92d8ffe72ac1af7dc9d8c52a6037c9e655b62e81b4b610
Sep 11 04:58:20.186648 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:20.186624    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx"]
Sep 11 04:58:20.188775 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:58:20.188749    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f96fb36_62c6_4f29_87a5_1d0ba0b1ed02.slice/crio-9eb947365a7f5064d13a43fd8a541e77efc2bf24748b70bbfab9bc770b38a6c5 WatchSource:0}: Error finding container 9eb947365a7f5064d13a43fd8a541e77efc2bf24748b70bbfab9bc770b38a6c5: Status 404 returned error can't find the container with id 9eb947365a7f5064d13a43fd8a541e77efc2bf24748b70bbfab9bc770b38a6c5
Sep 11 04:58:20.578423 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:20.578388    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx" event={"ID":"8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02","Type":"ContainerStarted","Data":"1eecd81873f15aab09164eb04313e745a8ec0f90ef9475d01ae65d5766ae0ec9"}
Sep 11 04:58:20.578620 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:20.578446    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx" event={"ID":"8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02","Type":"ContainerStarted","Data":"9eb947365a7f5064d13a43fd8a541e77efc2bf24748b70bbfab9bc770b38a6c5"}
Sep 11 04:58:20.578620 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:20.578501    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx"
Sep 11 04:58:20.580010 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:20.579982    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5" event={"ID":"24942598-c316-4136-b9f2-5dda388ae49c","Type":"ContainerStarted","Data":"a7d6c083ce0291cfb4489b7b59cbf2f48028b3750fe098e7330b1db1dcc4d174"}
Sep 11 04:58:20.580130 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:20.580033    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"
Sep 11 04:58:20.580130 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:20.580051    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5" event={"ID":"24942598-c316-4136-b9f2-5dda388ae49c","Type":"ContainerStarted","Data":"782806987d547cb06d92d8ffe72ac1af7dc9d8c52a6037c9e655b62e81b4b610"}
Sep 11 04:58:20.596288 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:20.596235    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx" podStartSLOduration=1.5962172479999999 podStartE2EDuration="1.596217248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:58:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:58:20.592806637 +0000 UTC m=+1115.064270515" watchObservedRunningTime="2026-09-11 04:58:20.596217248 +0000 UTC m=+1115.067681129"
Sep 11 04:58:20.610418 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:20.610377    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5" podStartSLOduration=2.291969167 podStartE2EDuration="2.610357109s" totalImagesPullingTime="318.387942ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:58:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:58:20.608539741 +0000 UTC m=+1115.080003618" watchObservedRunningTime="2026-09-11 04:58:20.610357109 +0000 UTC m=+1115.081820984"
Sep 11 04:58:23.649801 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:23.649764    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:26.592161 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:26.592133    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"
Sep 11 04:58:31.588861 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:31.588827    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx"
Sep 11 04:58:39.022846 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:39.022807    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:42.636386 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:42.636348    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:52.229186 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:52.229143    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:52.551289 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:52.551255    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:52.650977 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:52.650922    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:55.567497 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:55.567462    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:55.691551 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:55.691516    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:56.715086 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:56.715038    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:56.854059 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:56.854015    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:56.953044 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:56.953014    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:57.346640 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.346592    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:57.454919 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.454888    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:58:57.456619 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.456590    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97"]
Sep 11 04:58:57.461488 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.461472    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97"
Sep 11 04:58:57.464010 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.463990    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-89e681d0-dockercfg-lwm7k\""
Sep 11 04:58:57.464399 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.464383    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-89e681d0\""
Sep 11 04:58:57.476097 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.475712    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97"]
Sep 11 04:58:57.487103 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.487059    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv"]
Sep 11 04:58:57.490731 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.490708    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv"
Sep 11 04:58:57.505018 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.504996    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv"]
Sep 11 04:58:57.561709 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.561675    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e3816de-681f-4ed8-9a8f-e5836af172c6-plugins-config-volume\") pod \"payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv\" (UID: \"6e3816de-681f-4ed8-9a8f-e5836af172c6\") " pod="openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv"
Sep 11 04:58:57.561895 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.561722    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97\" (UID: \"d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97"
Sep 11 04:58:57.561895 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.561756    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxgfs\" (UniqueName: \"kubernetes.io/projected/d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f-kube-api-access-nxgfs\") pod \"payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97\" (UID: \"d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97"
Sep 11 04:58:57.561895 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.561778    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wn49\" (UniqueName: \"kubernetes.io/projected/6e3816de-681f-4ed8-9a8f-e5836af172c6-kube-api-access-7wn49\") pod \"payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv\" (UID: \"6e3816de-681f-4ed8-9a8f-e5836af172c6\") " pod="openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv"
Sep 11 04:58:57.662901 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.662819    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e3816de-681f-4ed8-9a8f-e5836af172c6-plugins-config-volume\") pod \"payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv\" (UID: \"6e3816de-681f-4ed8-9a8f-e5836af172c6\") " pod="openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv"
Sep 11 04:58:57.662901 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.662856    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97\" (UID: \"d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97"
Sep 11 04:58:57.662901 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.662881    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxgfs\" (UniqueName: \"kubernetes.io/projected/d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f-kube-api-access-nxgfs\") pod \"payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97\" (UID: \"d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97"
Sep 11 04:58:57.662901 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.662898    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wn49\" (UniqueName: \"kubernetes.io/projected/6e3816de-681f-4ed8-9a8f-e5836af172c6-kube-api-access-7wn49\") pod \"payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv\" (UID: \"6e3816de-681f-4ed8-9a8f-e5836af172c6\") " pod="openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv"
Sep 11 04:58:57.663664 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.663637    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97\" (UID: \"d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97"
Sep 11 04:58:57.663664 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.663663    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e3816de-681f-4ed8-9a8f-e5836af172c6-plugins-config-volume\") pod \"payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv\" (UID: \"6e3816de-681f-4ed8-9a8f-e5836af172c6\") " pod="openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv"
Sep 11 04:58:57.671612 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.671582    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxgfs\" (UniqueName: \"kubernetes.io/projected/d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f-kube-api-access-nxgfs\") pod \"payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97\" (UID: \"d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97"
Sep 11 04:58:57.671726 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.671697    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wn49\" (UniqueName: \"kubernetes.io/projected/6e3816de-681f-4ed8-9a8f-e5836af172c6-kube-api-access-7wn49\") pod \"payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv\" (UID: \"6e3816de-681f-4ed8-9a8f-e5836af172c6\") " pod="openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv"
Sep 11 04:58:57.776203 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.776168    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97"
Sep 11 04:58:57.800851 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:57.800820    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv"
Sep 11 04:58:58.051365 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:58.051345    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97"]
Sep 11 04:58:58.053364 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:58:58.053341    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9c04e0f_d4c5_4a85_9f63_2ef03a13e76f.slice/crio-612bcfabf9aa04e5befd7ad7680fb97b3eb1b527a72e189e4668c6a333181410 WatchSource:0}: Error finding container 612bcfabf9aa04e5befd7ad7680fb97b3eb1b527a72e189e4668c6a333181410: Status 404 returned error can't find the container with id 612bcfabf9aa04e5befd7ad7680fb97b3eb1b527a72e189e4668c6a333181410
Sep 11 04:58:58.088354 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:58.088331    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv"]
Sep 11 04:58:58.093523 ip-10-0-128-18 kubenswrapper[2567]: W0911 04:58:58.093479    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e3816de_681f_4ed8_9a8f_e5836af172c6.slice/crio-c9a0bcfffa730bb681e98acd3948379be2f3828959129fd86b7465c7244bdd33 WatchSource:0}: Error finding container c9a0bcfffa730bb681e98acd3948379be2f3828959129fd86b7465c7244bdd33: Status 404 returned error can't find the container with id c9a0bcfffa730bb681e98acd3948379be2f3828959129fd86b7465c7244bdd33
Sep 11 04:58:58.789138 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:58.789101    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv" event={"ID":"6e3816de-681f-4ed8-9a8f-e5836af172c6","Type":"ContainerStarted","Data":"6b9eed1717223f9a13f11f9348aac2046ad1c8200f16458e8e95c8d880019c50"}
Sep 11 04:58:58.789616 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:58.789173    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv" event={"ID":"6e3816de-681f-4ed8-9a8f-e5836af172c6","Type":"ContainerStarted","Data":"c9a0bcfffa730bb681e98acd3948379be2f3828959129fd86b7465c7244bdd33"}
Sep 11 04:58:58.789616 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:58.789206    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv"
Sep 11 04:58:58.790934 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:58.790905    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97" event={"ID":"d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f","Type":"ContainerStarted","Data":"39d7a91ecbb247ba167ac409af9b0278ce18d9a2d033bbbc9513e37a845992b6"}
Sep 11 04:58:58.791104 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:58.790956    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97"
Sep 11 04:58:58.791104 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:58.790973    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97" event={"ID":"d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f","Type":"ContainerStarted","Data":"612bcfabf9aa04e5befd7ad7680fb97b3eb1b527a72e189e4668c6a333181410"}
Sep 11 04:58:58.824012 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:58.823959    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv" podStartSLOduration=1.823933902 podStartE2EDuration="1.823933902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:58:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:58:58.821791483 +0000 UTC m=+1153.293255360" watchObservedRunningTime="2026-09-11 04:58:58.823933902 +0000 UTC m=+1153.295397784"
Sep 11 04:58:58.844543 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:58:58.844490    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97" podStartSLOduration=1.8444722919999998 podStartE2EDuration="1.844472292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 04:58:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 04:58:58.842714053 +0000 UTC m=+1153.314177930" watchObservedRunningTime="2026-09-11 04:58:58.844472292 +0000 UTC m=+1153.315936167"
Sep 11 04:59:01.958522 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:59:01.958483    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:59:08.039266 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:59:08.039175    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:59:09.798030 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:59:09.798003    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97"
Sep 11 04:59:09.798516 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:59:09.798095    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv"
Sep 11 04:59:20.481953 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:59:20.481913    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:59:46.783139 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:59:46.783099    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 04:59:48.275138 ip-10-0-128-18 kubenswrapper[2567]: I0911 04:59:48.275101    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:00:15.102402 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:15.102367    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:00:30.506633 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:30.506599    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:00:30.534017 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:30.533980    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv"]
Sep 11 05:00:30.534281 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:30.534251    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv" podUID="6e3816de-681f-4ed8-9a8f-e5836af172c6" containerName="payload-processing" containerID="cri-o://6b9eed1717223f9a13f11f9348aac2046ad1c8200f16458e8e95c8d880019c50" gracePeriod=30
Sep 11 05:00:30.539581 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:30.539557    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97"]
Sep 11 05:00:30.539749 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:30.539712    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97" podUID="d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f" containerName="payload-pre-processing" containerID="cri-o://39d7a91ecbb247ba167ac409af9b0278ce18d9a2d033bbbc9513e37a845992b6" gracePeriod=30
Sep 11 05:00:30.591654 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:30.591613    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:00:30.692027 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:30.691951    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:00:30.774173 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:30.774045    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:00:30.803266 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:30.803243    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv"
Sep 11 05:00:30.830225 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:30.830203    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97"
Sep 11 05:00:30.974848 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:30.974810    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f-plugins-config-volume\") pod \"d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f\" (UID: \"d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f\") "
Sep 11 05:00:30.975041 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:30.974925    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wn49\" (UniqueName: \"kubernetes.io/projected/6e3816de-681f-4ed8-9a8f-e5836af172c6-kube-api-access-7wn49\") pod \"6e3816de-681f-4ed8-9a8f-e5836af172c6\" (UID: \"6e3816de-681f-4ed8-9a8f-e5836af172c6\") "
Sep 11 05:00:30.975041 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:30.974972    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxgfs\" (UniqueName: \"kubernetes.io/projected/d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f-kube-api-access-nxgfs\") pod \"d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f\" (UID: \"d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f\") "
Sep 11 05:00:30.975041 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:30.975016    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e3816de-681f-4ed8-9a8f-e5836af172c6-plugins-config-volume\") pod \"6e3816de-681f-4ed8-9a8f-e5836af172c6\" (UID: \"6e3816de-681f-4ed8-9a8f-e5836af172c6\") "
Sep 11 05:00:30.975402 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:30.975367    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f" (UID: "d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:00:30.975487 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:30.975440    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e3816de-681f-4ed8-9a8f-e5836af172c6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6e3816de-681f-4ed8-9a8f-e5836af172c6" (UID: "6e3816de-681f-4ed8-9a8f-e5836af172c6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:00:30.977347 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:30.977322    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e3816de-681f-4ed8-9a8f-e5836af172c6-kube-api-access-7wn49" (OuterVolumeSpecName: "kube-api-access-7wn49") pod "6e3816de-681f-4ed8-9a8f-e5836af172c6" (UID: "6e3816de-681f-4ed8-9a8f-e5836af172c6"). InnerVolumeSpecName "kube-api-access-7wn49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:00:30.977440 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:30.977375    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f-kube-api-access-nxgfs" (OuterVolumeSpecName: "kube-api-access-nxgfs") pod "d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f" (UID: "d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f"). InnerVolumeSpecName "kube-api-access-nxgfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:00:31.083208 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:31.077054    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e3816de-681f-4ed8-9a8f-e5836af172c6-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:00:31.083208 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:31.077124    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:00:31.083208 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:31.077149    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wn49\" (UniqueName: \"kubernetes.io/projected/6e3816de-681f-4ed8-9a8f-e5836af172c6-kube-api-access-7wn49\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:00:31.083208 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:31.077162    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxgfs\" (UniqueName: \"kubernetes.io/projected/d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f-kube-api-access-nxgfs\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:00:31.277449 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:31.277411    2567 generic.go:356] "Generic (PLEG): container finished" podID="6e3816de-681f-4ed8-9a8f-e5836af172c6" containerID="6b9eed1717223f9a13f11f9348aac2046ad1c8200f16458e8e95c8d880019c50" exitCode=0
Sep 11 05:00:31.277643 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:31.277470    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv"
Sep 11 05:00:31.277643 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:31.277497    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv" event={"ID":"6e3816de-681f-4ed8-9a8f-e5836af172c6","Type":"ContainerDied","Data":"6b9eed1717223f9a13f11f9348aac2046ad1c8200f16458e8e95c8d880019c50"}
Sep 11 05:00:31.277643 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:31.277597    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv" event={"ID":"6e3816de-681f-4ed8-9a8f-e5836af172c6","Type":"ContainerDied","Data":"c9a0bcfffa730bb681e98acd3948379be2f3828959129fd86b7465c7244bdd33"}
Sep 11 05:00:31.277643 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:31.277613    2567 scope.go:117] "RemoveContainer" containerID="6b9eed1717223f9a13f11f9348aac2046ad1c8200f16458e8e95c8d880019c50"
Sep 11 05:00:31.279005 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:31.278981    2567 generic.go:356] "Generic (PLEG): container finished" podID="d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f" containerID="39d7a91ecbb247ba167ac409af9b0278ce18d9a2d033bbbc9513e37a845992b6" exitCode=0
Sep 11 05:00:31.279156 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:31.279014    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97" event={"ID":"d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f","Type":"ContainerDied","Data":"39d7a91ecbb247ba167ac409af9b0278ce18d9a2d033bbbc9513e37a845992b6"}
Sep 11 05:00:31.279156 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:31.279055    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97"
Sep 11 05:00:31.279156 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:31.279082    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97" event={"ID":"d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f","Type":"ContainerDied","Data":"612bcfabf9aa04e5befd7ad7680fb97b3eb1b527a72e189e4668c6a333181410"}
Sep 11 05:00:31.288368 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:31.288354    2567 scope.go:117] "RemoveContainer" containerID="6b9eed1717223f9a13f11f9348aac2046ad1c8200f16458e8e95c8d880019c50"
Sep 11 05:00:31.288778 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:00:31.288646    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b9eed1717223f9a13f11f9348aac2046ad1c8200f16458e8e95c8d880019c50\": container with ID starting with 6b9eed1717223f9a13f11f9348aac2046ad1c8200f16458e8e95c8d880019c50 not found: ID does not exist" containerID="6b9eed1717223f9a13f11f9348aac2046ad1c8200f16458e8e95c8d880019c50"
Sep 11 05:00:31.288778 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:31.288670    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b9eed1717223f9a13f11f9348aac2046ad1c8200f16458e8e95c8d880019c50"} err="failed to get container status \"6b9eed1717223f9a13f11f9348aac2046ad1c8200f16458e8e95c8d880019c50\": rpc error: code = NotFound desc = could not find container \"6b9eed1717223f9a13f11f9348aac2046ad1c8200f16458e8e95c8d880019c50\": container with ID starting with 6b9eed1717223f9a13f11f9348aac2046ad1c8200f16458e8e95c8d880019c50 not found: ID does not exist"
Sep 11 05:00:31.288778 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:31.288684    2567 scope.go:117] "RemoveContainer" containerID="39d7a91ecbb247ba167ac409af9b0278ce18d9a2d033bbbc9513e37a845992b6"
Sep 11 05:00:31.296667 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:31.296651    2567 scope.go:117] "RemoveContainer" containerID="39d7a91ecbb247ba167ac409af9b0278ce18d9a2d033bbbc9513e37a845992b6"
Sep 11 05:00:31.296964 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:00:31.296946    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39d7a91ecbb247ba167ac409af9b0278ce18d9a2d033bbbc9513e37a845992b6\": container with ID starting with 39d7a91ecbb247ba167ac409af9b0278ce18d9a2d033bbbc9513e37a845992b6 not found: ID does not exist" containerID="39d7a91ecbb247ba167ac409af9b0278ce18d9a2d033bbbc9513e37a845992b6"
Sep 11 05:00:31.297051 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:31.296973    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39d7a91ecbb247ba167ac409af9b0278ce18d9a2d033bbbc9513e37a845992b6"} err="failed to get container status \"39d7a91ecbb247ba167ac409af9b0278ce18d9a2d033bbbc9513e37a845992b6\": rpc error: code = NotFound desc = could not find container \"39d7a91ecbb247ba167ac409af9b0278ce18d9a2d033bbbc9513e37a845992b6\": container with ID starting with 39d7a91ecbb247ba167ac409af9b0278ce18d9a2d033bbbc9513e37a845992b6 not found: ID does not exist"
Sep 11 05:00:31.299777 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:31.299759    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv"]
Sep 11 05:00:31.303694 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:31.303674    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-89e681d0-b597bb8cc-9v2cv"]
Sep 11 05:00:31.314467 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:31.314443    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97"]
Sep 11 05:00:31.321653 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:31.321627    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-89e681d0-56d966f74b-v6q97"]
Sep 11 05:00:32.018079 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:32.018046    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e3816de-681f-4ed8-9a8f-e5836af172c6" path="/var/lib/kubelet/pods/6e3816de-681f-4ed8-9a8f-e5836af172c6/volumes"
Sep 11 05:00:32.018472 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:32.018457    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f" path="/var/lib/kubelet/pods/d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f/volumes"
Sep 11 05:00:34.096329 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:34.096291    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:00:38.861178 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:38.861049    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:00:55.315859 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:55.315817    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:00:56.386448 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:00:56.386404    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:04.344917 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:04.344874    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:04.446175 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:04.446135    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:15.742609 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:15.742569    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:16.269517 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:16.269483    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:19.298595 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:19.298552    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:21.987980 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:21.987941    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:22.086684 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:22.086639    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:22.346093 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:22.346034    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:23.538162 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:23.538125    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:23.720401 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:23.720367    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:23.909205 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:23.909107    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:24.312182 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:24.312147    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:24.411867 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:24.411838    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:24.516025 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:24.515981    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:25.934649 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:25.934605    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:31.644983 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:31.644940    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:31.867317 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:31.867278    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:32.233443 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:32.233405    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:38.024977 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:38.024936    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:38.132641 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:38.132593    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:38.226282 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:38.226252    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:46.989286 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:46.989243    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:48.727983 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:48.727939    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:01:48.943557 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:01:48.943516    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:09.916534 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:09.916440    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:15.664447 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:15.664415    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:15.765264 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:15.765230    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:16.006698 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:16.006598    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:16.315148 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:16.315114    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:16.401325 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:16.401273    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:19.727700 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:19.727667    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:19.828535 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:19.828494    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:20.912578 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:20.912542    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:21.012946 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.012904    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:21.312056 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.312017    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:21.559375 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.559321    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:21.638370 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.638282    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:21.642881 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.642855    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c"]
Sep 11 05:02:21.643440 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.643425    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f" containerName="payload-pre-processing"
Sep 11 05:02:21.643487 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.643443    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f" containerName="payload-pre-processing"
Sep 11 05:02:21.643524 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.643491    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e3816de-681f-4ed8-9a8f-e5836af172c6" containerName="payload-processing"
Sep 11 05:02:21.643524 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.643499    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e3816de-681f-4ed8-9a8f-e5836af172c6" containerName="payload-processing"
Sep 11 05:02:21.643592 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.643574    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e3816de-681f-4ed8-9a8f-e5836af172c6" containerName="payload-processing"
Sep 11 05:02:21.643592 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.643586    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9c04e0f-d4c5-4a85-9f63-2ef03a13e76f" containerName="payload-pre-processing"
Sep 11 05:02:21.646920 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.646904    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c"
Sep 11 05:02:21.649644 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.649620    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-b9198b\""
Sep 11 05:02:21.661314 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.661292    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c"]
Sep 11 05:02:21.682817 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.682784    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg"]
Sep 11 05:02:21.686395 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.686375    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg"
Sep 11 05:02:21.688799 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.688780    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-b9198b-dockercfg-qcg6q\""
Sep 11 05:02:21.693648 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.693620    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecb7e910-5c0f-4850-abc5-7bf456c2c844-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c\" (UID: \"ecb7e910-5c0f-4850-abc5-7bf456c2c844\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c"
Sep 11 05:02:21.693846 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.693819    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqrzx\" (UniqueName: \"kubernetes.io/projected/ecb7e910-5c0f-4850-abc5-7bf456c2c844-kube-api-access-kqrzx\") pod \"payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c\" (UID: \"ecb7e910-5c0f-4850-abc5-7bf456c2c844\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c"
Sep 11 05:02:21.697321 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.697298    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg"]
Sep 11 05:02:21.794809 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.794773    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecb7e910-5c0f-4850-abc5-7bf456c2c844-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c\" (UID: \"ecb7e910-5c0f-4850-abc5-7bf456c2c844\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c"
Sep 11 05:02:21.794972 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.794906    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqrzx\" (UniqueName: \"kubernetes.io/projected/ecb7e910-5c0f-4850-abc5-7bf456c2c844-kube-api-access-kqrzx\") pod \"payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c\" (UID: \"ecb7e910-5c0f-4850-abc5-7bf456c2c844\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c"
Sep 11 05:02:21.794972 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.794935    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faf657f3-5806-4df4-9e6a-d791ac76c686-plugins-config-volume\") pod \"payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg\" (UID: \"faf657f3-5806-4df4-9e6a-d791ac76c686\") " pod="openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg"
Sep 11 05:02:21.794972 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.794969    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkdkp\" (UniqueName: \"kubernetes.io/projected/faf657f3-5806-4df4-9e6a-d791ac76c686-kube-api-access-kkdkp\") pod \"payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg\" (UID: \"faf657f3-5806-4df4-9e6a-d791ac76c686\") " pod="openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg"
Sep 11 05:02:21.795724 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.795698    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecb7e910-5c0f-4850-abc5-7bf456c2c844-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c\" (UID: \"ecb7e910-5c0f-4850-abc5-7bf456c2c844\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c"
Sep 11 05:02:21.808371 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.808342    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqrzx\" (UniqueName: \"kubernetes.io/projected/ecb7e910-5c0f-4850-abc5-7bf456c2c844-kube-api-access-kqrzx\") pod \"payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c\" (UID: \"ecb7e910-5c0f-4850-abc5-7bf456c2c844\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c"
Sep 11 05:02:21.895891 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.895813    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faf657f3-5806-4df4-9e6a-d791ac76c686-plugins-config-volume\") pod \"payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg\" (UID: \"faf657f3-5806-4df4-9e6a-d791ac76c686\") " pod="openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg"
Sep 11 05:02:21.895891 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.895860    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkdkp\" (UniqueName: \"kubernetes.io/projected/faf657f3-5806-4df4-9e6a-d791ac76c686-kube-api-access-kkdkp\") pod \"payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg\" (UID: \"faf657f3-5806-4df4-9e6a-d791ac76c686\") " pod="openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg"
Sep 11 05:02:21.896561 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.896537    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faf657f3-5806-4df4-9e6a-d791ac76c686-plugins-config-volume\") pod \"payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg\" (UID: \"faf657f3-5806-4df4-9e6a-d791ac76c686\") " pod="openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg"
Sep 11 05:02:21.905279 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.905261    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkdkp\" (UniqueName: \"kubernetes.io/projected/faf657f3-5806-4df4-9e6a-d791ac76c686-kube-api-access-kkdkp\") pod \"payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg\" (UID: \"faf657f3-5806-4df4-9e6a-d791ac76c686\") " pod="openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg"
Sep 11 05:02:21.957296 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.957273    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c"
Sep 11 05:02:21.998628 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:21.998050    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg"
Sep 11 05:02:22.258844 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:22.258818    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c"]
Sep 11 05:02:22.260963 ip-10-0-128-18 kubenswrapper[2567]: W0911 05:02:22.260936    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecb7e910_5c0f_4850_abc5_7bf456c2c844.slice/crio-35b732293ae66459cdfab57dbc98458bcb812fb8d572efc66761af58004aefb4 WatchSource:0}: Error finding container 35b732293ae66459cdfab57dbc98458bcb812fb8d572efc66761af58004aefb4: Status 404 returned error can't find the container with id 35b732293ae66459cdfab57dbc98458bcb812fb8d572efc66761af58004aefb4
Sep 11 05:02:22.312523 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:22.312501    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg"]
Sep 11 05:02:22.314272 ip-10-0-128-18 kubenswrapper[2567]: W0911 05:02:22.314248    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaf657f3_5806_4df4_9e6a_d791ac76c686.slice/crio-39e23055463b3d68313786926b4f2c0f1696bba70fb19e809e9d3aeb39a5e0bc WatchSource:0}: Error finding container 39e23055463b3d68313786926b4f2c0f1696bba70fb19e809e9d3aeb39a5e0bc: Status 404 returned error can't find the container with id 39e23055463b3d68313786926b4f2c0f1696bba70fb19e809e9d3aeb39a5e0bc
Sep 11 05:02:22.321518 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:22.321492    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:22.863703 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:22.863650    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:22.866469 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:22.866438    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c" event={"ID":"ecb7e910-5c0f-4850-abc5-7bf456c2c844","Type":"ContainerStarted","Data":"b18fd829e8b591d095b3dd3be61c3f87bdf45fc1669df1de83daddda7e346e63"}
Sep 11 05:02:22.866632 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:22.866503    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c"
Sep 11 05:02:22.866632 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:22.866528    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c" event={"ID":"ecb7e910-5c0f-4850-abc5-7bf456c2c844","Type":"ContainerStarted","Data":"35b732293ae66459cdfab57dbc98458bcb812fb8d572efc66761af58004aefb4"}
Sep 11 05:02:22.868165 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:22.868141    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg" event={"ID":"faf657f3-5806-4df4-9e6a-d791ac76c686","Type":"ContainerStarted","Data":"67e73d66aa43f2d845201192273251918c2091a80fcdc5bba717fdea956c3ce9"}
Sep 11 05:02:22.868271 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:22.868186    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg" event={"ID":"faf657f3-5806-4df4-9e6a-d791ac76c686","Type":"ContainerStarted","Data":"39e23055463b3d68313786926b4f2c0f1696bba70fb19e809e9d3aeb39a5e0bc"}
Sep 11 05:02:22.868271 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:22.868219    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg"
Sep 11 05:02:22.885665 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:22.885612    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c" podStartSLOduration=1.8855984270000001 podStartE2EDuration="1.885598427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:02:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:02:22.882998729 +0000 UTC m=+1357.354462602" watchObservedRunningTime="2026-09-11 05:02:22.885598427 +0000 UTC m=+1357.357062305"
Sep 11 05:02:22.900272 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:22.900227    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg" podStartSLOduration=1.900213044 podStartE2EDuration="1.900213044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:02:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:02:22.899298067 +0000 UTC m=+1357.370761943" watchObservedRunningTime="2026-09-11 05:02:22.900213044 +0000 UTC m=+1357.371676919"
Sep 11 05:02:33.876266 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:33.876227    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c"
Sep 11 05:02:33.876652 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:33.876553    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg"
Sep 11 05:02:34.991874 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:34.991836    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:35.980698 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:35.980654    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:44.363915 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:44.363879    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:44.674847 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:44.674753    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:45.120461 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:45.120418    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:45.446229 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:45.446143    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:45.756155 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:45.756049    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:46.188588 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:46.188558    2567 scope.go:117] "RemoveContainer" containerID="ca183333798b59359dc5fc815bfb9744ed385beef71566f543dd2d8a2a3f9338"
Sep 11 05:02:46.197339 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:46.197318    2567 scope.go:117] "RemoveContainer" containerID="044695aec04f9c08c129eaa76c622024dc79589c9532bdaefca1d97f89d47188"
Sep 11 05:02:48.335874 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:48.335839    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:48.431942 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:48.431902    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:49.007217 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:49.007168    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:49.815212 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:49.815182    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:49.915886 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:49.915847    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:50.363656 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:50.363623    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:50.449132 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:50.449098    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:50.511679 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:50.511650    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9"]
Sep 11 05:02:50.516679 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:50.516665    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9"
Sep 11 05:02:50.520011 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:50.519990    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-9c1509\""
Sep 11 05:02:50.520232 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:50.520219    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-9c1509-dockercfg-wp9h5\""
Sep 11 05:02:50.528466 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:50.528408    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:02:50.529581 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:50.529560    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9"]
Sep 11 05:02:50.588661 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:50.588634    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvnzb\" (UniqueName: \"kubernetes.io/projected/2955831b-a53d-4340-9023-13a00ad2b863-kube-api-access-zvnzb\") pod \"payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9\" (UID: \"2955831b-a53d-4340-9023-13a00ad2b863\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9"
Sep 11 05:02:50.588813 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:50.588747    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2955831b-a53d-4340-9023-13a00ad2b863-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9\" (UID: \"2955831b-a53d-4340-9023-13a00ad2b863\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9"
Sep 11 05:02:50.689382 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:50.689307    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvnzb\" (UniqueName: \"kubernetes.io/projected/2955831b-a53d-4340-9023-13a00ad2b863-kube-api-access-zvnzb\") pod \"payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9\" (UID: \"2955831b-a53d-4340-9023-13a00ad2b863\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9"
Sep 11 05:02:50.689530 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:50.689423    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2955831b-a53d-4340-9023-13a00ad2b863-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9\" (UID: \"2955831b-a53d-4340-9023-13a00ad2b863\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9"
Sep 11 05:02:50.690103 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:50.690083    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2955831b-a53d-4340-9023-13a00ad2b863-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9\" (UID: \"2955831b-a53d-4340-9023-13a00ad2b863\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9"
Sep 11 05:02:50.700416 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:50.700396    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvnzb\" (UniqueName: \"kubernetes.io/projected/2955831b-a53d-4340-9023-13a00ad2b863-kube-api-access-zvnzb\") pod \"payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9\" (UID: \"2955831b-a53d-4340-9023-13a00ad2b863\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9"
Sep 11 05:02:50.827255 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:50.827228    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9"
Sep 11 05:02:51.099904 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:51.099876    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9"]
Sep 11 05:02:51.102163 ip-10-0-128-18 kubenswrapper[2567]: W0911 05:02:51.102131    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2955831b_a53d_4340_9023_13a00ad2b863.slice/crio-6769b9c38d230bd89488293a26dab6072c3c217f3926cfa42cd29d49e1972038 WatchSource:0}: Error finding container 6769b9c38d230bd89488293a26dab6072c3c217f3926cfa42cd29d49e1972038: Status 404 returned error can't find the container with id 6769b9c38d230bd89488293a26dab6072c3c217f3926cfa42cd29d49e1972038
Sep 11 05:02:52.041598 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:52.041560    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9" event={"ID":"2955831b-a53d-4340-9023-13a00ad2b863","Type":"ContainerStarted","Data":"d1b24d38ee346733a28ef39f50f4080a41a13336d1869fe8894507cdd5126ed4"}
Sep 11 05:02:52.041989 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:52.041613    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9" event={"ID":"2955831b-a53d-4340-9023-13a00ad2b863","Type":"ContainerStarted","Data":"6769b9c38d230bd89488293a26dab6072c3c217f3926cfa42cd29d49e1972038"}
Sep 11 05:02:52.041989 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:52.041682    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9"
Sep 11 05:02:52.059830 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:02:52.059774    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9" podStartSLOduration=2.059758565 podStartE2EDuration="2.059758565s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:02:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:02:52.057658573 +0000 UTC m=+1386.529122449" watchObservedRunningTime="2026-09-11 05:02:52.059758565 +0000 UTC m=+1386.531222440"
Sep 11 05:03:03.049509 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:03.049476    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9"
Sep 11 05:03:12.214875 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:12.214832    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:03:12.875692 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:12.875653    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:03:13.279628 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:13.279591    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:03:16.279608 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:16.279559    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:03:31.643334 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:31.643281    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:03:31.681472 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:31.681434    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg"]
Sep 11 05:03:31.682022 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:31.681960    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg" podUID="faf657f3-5806-4df4-9e6a-d791ac76c686" containerName="payload-processing" containerID="cri-o://67e73d66aa43f2d845201192273251918c2091a80fcdc5bba717fdea956c3ce9" gracePeriod=30
Sep 11 05:03:31.685789 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:31.685753    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c"]
Sep 11 05:03:31.686054 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:31.685982    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c" podUID="ecb7e910-5c0f-4850-abc5-7bf456c2c844" containerName="payload-pre-processing" containerID="cri-o://b18fd829e8b591d095b3dd3be61c3f87bdf45fc1669df1de83daddda7e346e63" gracePeriod=30
Sep 11 05:03:31.728716 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:31.728672    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:03:31.833734 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:31.833694    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:03:31.931782 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:31.931750    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:03:31.942994 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:31.942973    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg"
Sep 11 05:03:31.982806 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:31.982779    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c"
Sep 11 05:03:31.999491 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:31.999458    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kkdkp\" (UniqueName: \"kubernetes.io/projected/faf657f3-5806-4df4-9e6a-d791ac76c686-kube-api-access-kkdkp\") pod \"faf657f3-5806-4df4-9e6a-d791ac76c686\" (UID: \"faf657f3-5806-4df4-9e6a-d791ac76c686\") "
Sep 11 05:03:31.999671 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:31.999500    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faf657f3-5806-4df4-9e6a-d791ac76c686-plugins-config-volume\") pod \"faf657f3-5806-4df4-9e6a-d791ac76c686\" (UID: \"faf657f3-5806-4df4-9e6a-d791ac76c686\") "
Sep 11 05:03:32.000445 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.000153    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/faf657f3-5806-4df4-9e6a-d791ac76c686-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "faf657f3-5806-4df4-9e6a-d791ac76c686" (UID: "faf657f3-5806-4df4-9e6a-d791ac76c686"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:03:32.002775 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.002737    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faf657f3-5806-4df4-9e6a-d791ac76c686-kube-api-access-kkdkp" (OuterVolumeSpecName: "kube-api-access-kkdkp") pod "faf657f3-5806-4df4-9e6a-d791ac76c686" (UID: "faf657f3-5806-4df4-9e6a-d791ac76c686"). InnerVolumeSpecName "kube-api-access-kkdkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:03:32.100350 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.100318    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqrzx\" (UniqueName: \"kubernetes.io/projected/ecb7e910-5c0f-4850-abc5-7bf456c2c844-kube-api-access-kqrzx\") pod \"ecb7e910-5c0f-4850-abc5-7bf456c2c844\" (UID: \"ecb7e910-5c0f-4850-abc5-7bf456c2c844\") "
Sep 11 05:03:32.100551 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.100452    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecb7e910-5c0f-4850-abc5-7bf456c2c844-plugins-config-volume\") pod \"ecb7e910-5c0f-4850-abc5-7bf456c2c844\" (UID: \"ecb7e910-5c0f-4850-abc5-7bf456c2c844\") "
Sep 11 05:03:32.100710 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.100692    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kkdkp\" (UniqueName: \"kubernetes.io/projected/faf657f3-5806-4df4-9e6a-d791ac76c686-kube-api-access-kkdkp\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:03:32.100767 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.100715    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faf657f3-5806-4df4-9e6a-d791ac76c686-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:03:32.100835 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.100802    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecb7e910-5c0f-4850-abc5-7bf456c2c844-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ecb7e910-5c0f-4850-abc5-7bf456c2c844" (UID: "ecb7e910-5c0f-4850-abc5-7bf456c2c844"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:03:32.102577 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.102555    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecb7e910-5c0f-4850-abc5-7bf456c2c844-kube-api-access-kqrzx" (OuterVolumeSpecName: "kube-api-access-kqrzx") pod "ecb7e910-5c0f-4850-abc5-7bf456c2c844" (UID: "ecb7e910-5c0f-4850-abc5-7bf456c2c844"). InnerVolumeSpecName "kube-api-access-kqrzx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:03:32.202097 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.201999    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecb7e910-5c0f-4850-abc5-7bf456c2c844-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:03:32.202097 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.202028    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqrzx\" (UniqueName: \"kubernetes.io/projected/ecb7e910-5c0f-4850-abc5-7bf456c2c844-kube-api-access-kqrzx\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:03:32.221587 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.221554    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:03:32.249855 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.249823    2567 generic.go:356] "Generic (PLEG): container finished" podID="ecb7e910-5c0f-4850-abc5-7bf456c2c844" containerID="b18fd829e8b591d095b3dd3be61c3f87bdf45fc1669df1de83daddda7e346e63" exitCode=0
Sep 11 05:03:32.250140 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.249867    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c"
Sep 11 05:03:32.250140 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.249909    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c" event={"ID":"ecb7e910-5c0f-4850-abc5-7bf456c2c844","Type":"ContainerDied","Data":"b18fd829e8b591d095b3dd3be61c3f87bdf45fc1669df1de83daddda7e346e63"}
Sep 11 05:03:32.250140 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.249963    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c" event={"ID":"ecb7e910-5c0f-4850-abc5-7bf456c2c844","Type":"ContainerDied","Data":"35b732293ae66459cdfab57dbc98458bcb812fb8d572efc66761af58004aefb4"}
Sep 11 05:03:32.250140 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.249993    2567 scope.go:117] "RemoveContainer" containerID="b18fd829e8b591d095b3dd3be61c3f87bdf45fc1669df1de83daddda7e346e63"
Sep 11 05:03:32.251607 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.251581    2567 generic.go:356] "Generic (PLEG): container finished" podID="faf657f3-5806-4df4-9e6a-d791ac76c686" containerID="67e73d66aa43f2d845201192273251918c2091a80fcdc5bba717fdea956c3ce9" exitCode=0
Sep 11 05:03:32.251761 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.251626    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg" event={"ID":"faf657f3-5806-4df4-9e6a-d791ac76c686","Type":"ContainerDied","Data":"67e73d66aa43f2d845201192273251918c2091a80fcdc5bba717fdea956c3ce9"}
Sep 11 05:03:32.251761 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.251651    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg"
Sep 11 05:03:32.251761 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.251693    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg" event={"ID":"faf657f3-5806-4df4-9e6a-d791ac76c686","Type":"ContainerDied","Data":"39e23055463b3d68313786926b4f2c0f1696bba70fb19e809e9d3aeb39a5e0bc"}
Sep 11 05:03:32.261418 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.261365    2567 scope.go:117] "RemoveContainer" containerID="b18fd829e8b591d095b3dd3be61c3f87bdf45fc1669df1de83daddda7e346e63"
Sep 11 05:03:32.261680 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:03:32.261661    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b18fd829e8b591d095b3dd3be61c3f87bdf45fc1669df1de83daddda7e346e63\": container with ID starting with b18fd829e8b591d095b3dd3be61c3f87bdf45fc1669df1de83daddda7e346e63 not found: ID does not exist" containerID="b18fd829e8b591d095b3dd3be61c3f87bdf45fc1669df1de83daddda7e346e63"
Sep 11 05:03:32.261891 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.261690    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b18fd829e8b591d095b3dd3be61c3f87bdf45fc1669df1de83daddda7e346e63"} err="failed to get container status \"b18fd829e8b591d095b3dd3be61c3f87bdf45fc1669df1de83daddda7e346e63\": rpc error: code = NotFound desc = could not find container \"b18fd829e8b591d095b3dd3be61c3f87bdf45fc1669df1de83daddda7e346e63\": container with ID starting with b18fd829e8b591d095b3dd3be61c3f87bdf45fc1669df1de83daddda7e346e63 not found: ID does not exist"
Sep 11 05:03:32.261891 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.261712    2567 scope.go:117] "RemoveContainer" containerID="67e73d66aa43f2d845201192273251918c2091a80fcdc5bba717fdea956c3ce9"
Sep 11 05:03:32.269732 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.269715    2567 scope.go:117] "RemoveContainer" containerID="67e73d66aa43f2d845201192273251918c2091a80fcdc5bba717fdea956c3ce9"
Sep 11 05:03:32.270004 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:03:32.269987    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67e73d66aa43f2d845201192273251918c2091a80fcdc5bba717fdea956c3ce9\": container with ID starting with 67e73d66aa43f2d845201192273251918c2091a80fcdc5bba717fdea956c3ce9 not found: ID does not exist" containerID="67e73d66aa43f2d845201192273251918c2091a80fcdc5bba717fdea956c3ce9"
Sep 11 05:03:32.270048 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.270010    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67e73d66aa43f2d845201192273251918c2091a80fcdc5bba717fdea956c3ce9"} err="failed to get container status \"67e73d66aa43f2d845201192273251918c2091a80fcdc5bba717fdea956c3ce9\": rpc error: code = NotFound desc = could not find container \"67e73d66aa43f2d845201192273251918c2091a80fcdc5bba717fdea956c3ce9\": container with ID starting with 67e73d66aa43f2d845201192273251918c2091a80fcdc5bba717fdea956c3ce9 not found: ID does not exist"
Sep 11 05:03:32.274864 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.274838    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg"]
Sep 11 05:03:32.276813 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.276792    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-b9198b-6b44bf5d9-xmwlg"]
Sep 11 05:03:32.287652 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.287630    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c"]
Sep 11 05:03:32.291790 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:32.291769    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-b9198b-6cc45699fd-r6c7c"]
Sep 11 05:03:34.018772 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:34.018731    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecb7e910-5c0f-4850-abc5-7bf456c2c844" path="/var/lib/kubelet/pods/ecb7e910-5c0f-4850-abc5-7bf456c2c844/volumes"
Sep 11 05:03:34.019305 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:34.019286    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="faf657f3-5806-4df4-9e6a-d791ac76c686" path="/var/lib/kubelet/pods/faf657f3-5806-4df4-9e6a-d791ac76c686/volumes"
Sep 11 05:03:38.750949 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:38.750856    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:03:38.851486 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:38.851450    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:03:39.002670 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:39.002572    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:03:39.489495 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:39.489464    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:03:40.785317 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:40.785279    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:03:49.189590 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:49.189554    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:03:49.481020 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:49.480923    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:03:55.078817 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:55.078768    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:03:55.106786 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:55.106749    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9"]
Sep 11 05:03:55.107602 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:55.107443    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9" podUID="2955831b-a53d-4340-9023-13a00ad2b863" containerName="payload-pre-processing" containerID="cri-o://d1b24d38ee346733a28ef39f50f4080a41a13336d1869fe8894507cdd5126ed4" gracePeriod=30
Sep 11 05:03:55.182884 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:55.182849    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:03:55.282017 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:55.281978    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:03:55.349544 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:55.349458    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:03:55.377655 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:55.377626    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9"
Sep 11 05:03:55.381472 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:55.381444    2567 generic.go:356] "Generic (PLEG): container finished" podID="2955831b-a53d-4340-9023-13a00ad2b863" containerID="d1b24d38ee346733a28ef39f50f4080a41a13336d1869fe8894507cdd5126ed4" exitCode=0
Sep 11 05:03:55.381608 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:55.381475    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9" event={"ID":"2955831b-a53d-4340-9023-13a00ad2b863","Type":"ContainerDied","Data":"d1b24d38ee346733a28ef39f50f4080a41a13336d1869fe8894507cdd5126ed4"}
Sep 11 05:03:55.381608 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:55.381484    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9"
Sep 11 05:03:55.381608 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:55.381560    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9" event={"ID":"2955831b-a53d-4340-9023-13a00ad2b863","Type":"ContainerDied","Data":"6769b9c38d230bd89488293a26dab6072c3c217f3926cfa42cd29d49e1972038"}
Sep 11 05:03:55.381608 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:55.381588    2567 scope.go:117] "RemoveContainer" containerID="d1b24d38ee346733a28ef39f50f4080a41a13336d1869fe8894507cdd5126ed4"
Sep 11 05:03:55.394367 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:55.394340    2567 scope.go:117] "RemoveContainer" containerID="d1b24d38ee346733a28ef39f50f4080a41a13336d1869fe8894507cdd5126ed4"
Sep 11 05:03:55.394956 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:03:55.394872    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1b24d38ee346733a28ef39f50f4080a41a13336d1869fe8894507cdd5126ed4\": container with ID starting with d1b24d38ee346733a28ef39f50f4080a41a13336d1869fe8894507cdd5126ed4 not found: ID does not exist" containerID="d1b24d38ee346733a28ef39f50f4080a41a13336d1869fe8894507cdd5126ed4"
Sep 11 05:03:55.394956 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:55.394911    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1b24d38ee346733a28ef39f50f4080a41a13336d1869fe8894507cdd5126ed4"} err="failed to get container status \"d1b24d38ee346733a28ef39f50f4080a41a13336d1869fe8894507cdd5126ed4\": rpc error: code = NotFound desc = could not find container \"d1b24d38ee346733a28ef39f50f4080a41a13336d1869fe8894507cdd5126ed4\": container with ID starting with d1b24d38ee346733a28ef39f50f4080a41a13336d1869fe8894507cdd5126ed4 not found: ID does not exist"
Sep 11 05:03:55.535325 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:55.535290    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvnzb\" (UniqueName: \"kubernetes.io/projected/2955831b-a53d-4340-9023-13a00ad2b863-kube-api-access-zvnzb\") pod \"2955831b-a53d-4340-9023-13a00ad2b863\" (UID: \"2955831b-a53d-4340-9023-13a00ad2b863\") "
Sep 11 05:03:55.535489 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:55.535435    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2955831b-a53d-4340-9023-13a00ad2b863-plugins-config-volume\") pod \"2955831b-a53d-4340-9023-13a00ad2b863\" (UID: \"2955831b-a53d-4340-9023-13a00ad2b863\") "
Sep 11 05:03:55.535824 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:55.535790    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2955831b-a53d-4340-9023-13a00ad2b863-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2955831b-a53d-4340-9023-13a00ad2b863" (UID: "2955831b-a53d-4340-9023-13a00ad2b863"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:03:55.537389 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:55.537366    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2955831b-a53d-4340-9023-13a00ad2b863-kube-api-access-zvnzb" (OuterVolumeSpecName: "kube-api-access-zvnzb") pod "2955831b-a53d-4340-9023-13a00ad2b863" (UID: "2955831b-a53d-4340-9023-13a00ad2b863"). InnerVolumeSpecName "kube-api-access-zvnzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:03:55.636327 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:55.636247    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2955831b-a53d-4340-9023-13a00ad2b863-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:03:55.636327 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:55.636276    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvnzb\" (UniqueName: \"kubernetes.io/projected/2955831b-a53d-4340-9023-13a00ad2b863-kube-api-access-zvnzb\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:03:55.667455 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:55.667416    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:03:55.705602 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:55.705576    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9"]
Sep 11 05:03:55.710479 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:55.710454    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-9c1509-555dcfdbdb-tv4t9"]
Sep 11 05:03:56.020088 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:03:56.020037    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2955831b-a53d-4340-9023-13a00ad2b863" path="/var/lib/kubelet/pods/2955831b-a53d-4340-9023-13a00ad2b863/volumes"
Sep 11 05:04:01.699210 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:01.699172    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:01.800755 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:01.800715    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:01.945001 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:01.944968    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:02.513810 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:02.513776    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:12.732087 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:12.732031    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"]
Sep 11 05:04:12.732667 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:12.732245    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4" podUID="75ed05fd-87c0-4c62-91da-99bebc7445d8" containerName="maas-api" containerID="cri-o://9b962132e34a713fae56562f81f6e68ace5e4d958ac49db874ef295d884c2280" gracePeriod=30
Sep 11 05:04:12.762865 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:12.762785    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:12.795154 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:12.795116    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk"]
Sep 11 05:04:12.795363 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:12.795336    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk" podUID="44655566-d3ce-46d4-94c3-fd2123ffbe8d" containerName="payload-pre-processing" containerID="cri-o://0f3301d2d70befc41af202c1213e974248f855e13210b4f4325b9c86f452f74b" gracePeriod=30
Sep 11 05:04:12.860552 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:12.860506    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:12.943009 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:12.942811    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:13.040606 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.040571    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:13.076379 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.076359    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"
Sep 11 05:04:13.079471 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.079454    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk"
Sep 11 05:04:13.199363 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.199332    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wx7kv\" (UniqueName: \"kubernetes.io/projected/75ed05fd-87c0-4c62-91da-99bebc7445d8-kube-api-access-wx7kv\") pod \"75ed05fd-87c0-4c62-91da-99bebc7445d8\" (UID: \"75ed05fd-87c0-4c62-91da-99bebc7445d8\") "
Sep 11 05:04:13.199552 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.199369    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mg8sx\" (UniqueName: \"kubernetes.io/projected/44655566-d3ce-46d4-94c3-fd2123ffbe8d-kube-api-access-mg8sx\") pod \"44655566-d3ce-46d4-94c3-fd2123ffbe8d\" (UID: \"44655566-d3ce-46d4-94c3-fd2123ffbe8d\") "
Sep 11 05:04:13.199552 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.199469    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75ed05fd-87c0-4c62-91da-99bebc7445d8-metrics-certs\") pod \"75ed05fd-87c0-4c62-91da-99bebc7445d8\" (UID: \"75ed05fd-87c0-4c62-91da-99bebc7445d8\") "
Sep 11 05:04:13.199552 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.199498    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44655566-d3ce-46d4-94c3-fd2123ffbe8d-plugins-config-volume\") pod \"44655566-d3ce-46d4-94c3-fd2123ffbe8d\" (UID: \"44655566-d3ce-46d4-94c3-fd2123ffbe8d\") "
Sep 11 05:04:13.199552 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.199535    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75ed05fd-87c0-4c62-91da-99bebc7445d8-maas-api-tls\") pod \"75ed05fd-87c0-4c62-91da-99bebc7445d8\" (UID: \"75ed05fd-87c0-4c62-91da-99bebc7445d8\") "
Sep 11 05:04:13.199759 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.199576    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75ed05fd-87c0-4c62-91da-99bebc7445d8-openshift-service-ca\") pod \"75ed05fd-87c0-4c62-91da-99bebc7445d8\" (UID: \"75ed05fd-87c0-4c62-91da-99bebc7445d8\") "
Sep 11 05:04:13.199955 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.199926    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44655566-d3ce-46d4-94c3-fd2123ffbe8d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "44655566-d3ce-46d4-94c3-fd2123ffbe8d" (UID: "44655566-d3ce-46d4-94c3-fd2123ffbe8d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:04:13.200171 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.200129    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75ed05fd-87c0-4c62-91da-99bebc7445d8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "75ed05fd-87c0-4c62-91da-99bebc7445d8" (UID: "75ed05fd-87c0-4c62-91da-99bebc7445d8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:04:13.201857 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.201836    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75ed05fd-87c0-4c62-91da-99bebc7445d8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "75ed05fd-87c0-4c62-91da-99bebc7445d8" (UID: "75ed05fd-87c0-4c62-91da-99bebc7445d8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:04:13.201933 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.201859    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75ed05fd-87c0-4c62-91da-99bebc7445d8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "75ed05fd-87c0-4c62-91da-99bebc7445d8" (UID: "75ed05fd-87c0-4c62-91da-99bebc7445d8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:04:13.202194 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.202172    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75ed05fd-87c0-4c62-91da-99bebc7445d8-kube-api-access-wx7kv" (OuterVolumeSpecName: "kube-api-access-wx7kv") pod "75ed05fd-87c0-4c62-91da-99bebc7445d8" (UID: "75ed05fd-87c0-4c62-91da-99bebc7445d8"). InnerVolumeSpecName "kube-api-access-wx7kv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:04:13.202250 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.202172    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44655566-d3ce-46d4-94c3-fd2123ffbe8d-kube-api-access-mg8sx" (OuterVolumeSpecName: "kube-api-access-mg8sx") pod "44655566-d3ce-46d4-94c3-fd2123ffbe8d" (UID: "44655566-d3ce-46d4-94c3-fd2123ffbe8d"). InnerVolumeSpecName "kube-api-access-mg8sx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:04:13.299487 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.299457    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:13.300634 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.300614    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/75ed05fd-87c0-4c62-91da-99bebc7445d8-metrics-certs\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:04:13.300695 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.300638    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/44655566-d3ce-46d4-94c3-fd2123ffbe8d-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:04:13.300695 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.300648    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75ed05fd-87c0-4c62-91da-99bebc7445d8-maas-api-tls\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:04:13.300695 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.300656    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75ed05fd-87c0-4c62-91da-99bebc7445d8-openshift-service-ca\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:04:13.300695 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.300664    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wx7kv\" (UniqueName: \"kubernetes.io/projected/75ed05fd-87c0-4c62-91da-99bebc7445d8-kube-api-access-wx7kv\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:04:13.300695 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.300672    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mg8sx\" (UniqueName: \"kubernetes.io/projected/44655566-d3ce-46d4-94c3-fd2123ffbe8d-kube-api-access-mg8sx\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:04:13.476509 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.476479    2567 generic.go:356] "Generic (PLEG): container finished" podID="44655566-d3ce-46d4-94c3-fd2123ffbe8d" containerID="0f3301d2d70befc41af202c1213e974248f855e13210b4f4325b9c86f452f74b" exitCode=0
Sep 11 05:04:13.476682 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.476548    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk"
Sep 11 05:04:13.476682 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.476559    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk" event={"ID":"44655566-d3ce-46d4-94c3-fd2123ffbe8d","Type":"ContainerDied","Data":"0f3301d2d70befc41af202c1213e974248f855e13210b4f4325b9c86f452f74b"}
Sep 11 05:04:13.476682 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.476647    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk" event={"ID":"44655566-d3ce-46d4-94c3-fd2123ffbe8d","Type":"ContainerDied","Data":"7c45739b0940aec0ab4fed87536d4da70a6c0e92ec2946f3671c30e5f16d7389"}
Sep 11 05:04:13.476682 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.476677    2567 scope.go:117] "RemoveContainer" containerID="0f3301d2d70befc41af202c1213e974248f855e13210b4f4325b9c86f452f74b"
Sep 11 05:04:13.477882 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.477824    2567 generic.go:356] "Generic (PLEG): container finished" podID="75ed05fd-87c0-4c62-91da-99bebc7445d8" containerID="9b962132e34a713fae56562f81f6e68ace5e4d958ac49db874ef295d884c2280" exitCode=0
Sep 11 05:04:13.477882 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.477850    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4" event={"ID":"75ed05fd-87c0-4c62-91da-99bebc7445d8","Type":"ContainerDied","Data":"9b962132e34a713fae56562f81f6e68ace5e4d958ac49db874ef295d884c2280"}
Sep 11 05:04:13.478011 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.477891    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4" event={"ID":"75ed05fd-87c0-4c62-91da-99bebc7445d8","Type":"ContainerDied","Data":"0b30880dc538e6845bad6154886a8f9cec2fbe950df0347d50f9e4cd01591d5c"}
Sep 11 05:04:13.478011 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.477892    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"
Sep 11 05:04:13.487114 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.487095    2567 scope.go:117] "RemoveContainer" containerID="0f3301d2d70befc41af202c1213e974248f855e13210b4f4325b9c86f452f74b"
Sep 11 05:04:13.487427 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:04:13.487408    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f3301d2d70befc41af202c1213e974248f855e13210b4f4325b9c86f452f74b\": container with ID starting with 0f3301d2d70befc41af202c1213e974248f855e13210b4f4325b9c86f452f74b not found: ID does not exist" containerID="0f3301d2d70befc41af202c1213e974248f855e13210b4f4325b9c86f452f74b"
Sep 11 05:04:13.487505 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.487435    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f3301d2d70befc41af202c1213e974248f855e13210b4f4325b9c86f452f74b"} err="failed to get container status \"0f3301d2d70befc41af202c1213e974248f855e13210b4f4325b9c86f452f74b\": rpc error: code = NotFound desc = could not find container \"0f3301d2d70befc41af202c1213e974248f855e13210b4f4325b9c86f452f74b\": container with ID starting with 0f3301d2d70befc41af202c1213e974248f855e13210b4f4325b9c86f452f74b not found: ID does not exist"
Sep 11 05:04:13.487505 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.487451    2567 scope.go:117] "RemoveContainer" containerID="9b962132e34a713fae56562f81f6e68ace5e4d958ac49db874ef295d884c2280"
Sep 11 05:04:13.495864 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.495846    2567 scope.go:117] "RemoveContainer" containerID="9b962132e34a713fae56562f81f6e68ace5e4d958ac49db874ef295d884c2280"
Sep 11 05:04:13.496143 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:04:13.496122    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b962132e34a713fae56562f81f6e68ace5e4d958ac49db874ef295d884c2280\": container with ID starting with 9b962132e34a713fae56562f81f6e68ace5e4d958ac49db874ef295d884c2280 not found: ID does not exist" containerID="9b962132e34a713fae56562f81f6e68ace5e4d958ac49db874ef295d884c2280"
Sep 11 05:04:13.496200 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.496153    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b962132e34a713fae56562f81f6e68ace5e4d958ac49db874ef295d884c2280"} err="failed to get container status \"9b962132e34a713fae56562f81f6e68ace5e4d958ac49db874ef295d884c2280\": rpc error: code = NotFound desc = could not find container \"9b962132e34a713fae56562f81f6e68ace5e4d958ac49db874ef295d884c2280\": container with ID starting with 9b962132e34a713fae56562f81f6e68ace5e4d958ac49db874ef295d884c2280 not found: ID does not exist"
Sep 11 05:04:13.507678 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.507656    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"]
Sep 11 05:04:13.513747 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.513727    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-f5e6f7-7cbff769cf-mvlv4"]
Sep 11 05:04:13.524784 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.524763    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk"]
Sep 11 05:04:13.528856 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:13.528834    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f5e6f7-77dd4d6b58-jsnxk"]
Sep 11 05:04:14.017852 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:14.017816    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44655566-d3ce-46d4-94c3-fd2123ffbe8d" path="/var/lib/kubelet/pods/44655566-d3ce-46d4-94c3-fd2123ffbe8d/volumes"
Sep 11 05:04:14.018328 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:14.018301    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75ed05fd-87c0-4c62-91da-99bebc7445d8" path="/var/lib/kubelet/pods/75ed05fd-87c0-4c62-91da-99bebc7445d8/volumes"
Sep 11 05:04:18.875633 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:18.875597    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:18.934623 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:18.934591    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:19.129391 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:19.129304    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:19.552943 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:19.552902    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:19.851263 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:19.851165    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:20.985404 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:20.985374    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:21.859176 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:21.859142    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:22.284457 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:22.284424    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:22.567547 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:22.567457    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:22.762533 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:22.762496    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:25.422597 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:25.422560    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:25.519863 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:25.519832    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:26.538769 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.538736    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"]
Sep 11 05:04:26.539433 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.539406    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75ed05fd-87c0-4c62-91da-99bebc7445d8" containerName="maas-api"
Sep 11 05:04:26.539604 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.539583    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="75ed05fd-87c0-4c62-91da-99bebc7445d8" containerName="maas-api"
Sep 11 05:04:26.539698 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.539609    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faf657f3-5806-4df4-9e6a-d791ac76c686" containerName="payload-processing"
Sep 11 05:04:26.539698 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.539619    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="faf657f3-5806-4df4-9e6a-d791ac76c686" containerName="payload-processing"
Sep 11 05:04:26.539698 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.539639    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecb7e910-5c0f-4850-abc5-7bf456c2c844" containerName="payload-pre-processing"
Sep 11 05:04:26.539698 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.539647    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecb7e910-5c0f-4850-abc5-7bf456c2c844" containerName="payload-pre-processing"
Sep 11 05:04:26.539698 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.539662    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2955831b-a53d-4340-9023-13a00ad2b863" containerName="payload-pre-processing"
Sep 11 05:04:26.539698 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.539670    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2955831b-a53d-4340-9023-13a00ad2b863" containerName="payload-pre-processing"
Sep 11 05:04:26.539698 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.539692    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44655566-d3ce-46d4-94c3-fd2123ffbe8d" containerName="payload-pre-processing"
Sep 11 05:04:26.539698 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.539700    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="44655566-d3ce-46d4-94c3-fd2123ffbe8d" containerName="payload-pre-processing"
Sep 11 05:04:26.539949 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.539837    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2955831b-a53d-4340-9023-13a00ad2b863" containerName="payload-pre-processing"
Sep 11 05:04:26.539949 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.539852    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="faf657f3-5806-4df4-9e6a-d791ac76c686" containerName="payload-processing"
Sep 11 05:04:26.539949 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.539865    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="75ed05fd-87c0-4c62-91da-99bebc7445d8" containerName="maas-api"
Sep 11 05:04:26.539949 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.539878    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="44655566-d3ce-46d4-94c3-fd2123ffbe8d" containerName="payload-pre-processing"
Sep 11 05:04:26.539949 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.539888    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecb7e910-5c0f-4850-abc5-7bf456c2c844" containerName="payload-pre-processing"
Sep 11 05:04:26.543171 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.543145    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"
Sep 11 05:04:26.545748 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.545727    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-909c76-serving-cert\""
Sep 11 05:04:26.554454 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.554351    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"]
Sep 11 05:04:26.594531 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.594491    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:26.623689 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.623653    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pjq9\" (UniqueName: \"kubernetes.io/projected/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-kube-api-access-7pjq9\") pod \"maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv\" (UID: \"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"
Sep 11 05:04:26.623833 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.623714    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-metrics-certs\") pod \"maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv\" (UID: \"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"
Sep 11 05:04:26.623891 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.623868    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-maas-api-tls\") pod \"maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv\" (UID: \"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"
Sep 11 05:04:26.623948 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.623929    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv\" (UID: \"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"
Sep 11 05:04:26.694920 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.694885    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:26.725160 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.725132    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv\" (UID: \"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"
Sep 11 05:04:26.725302 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.725173    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pjq9\" (UniqueName: \"kubernetes.io/projected/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-kube-api-access-7pjq9\") pod \"maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv\" (UID: \"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"
Sep 11 05:04:26.725302 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.725213    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-metrics-certs\") pod \"maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv\" (UID: \"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"
Sep 11 05:04:26.725302 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.725288    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-maas-api-tls\") pod \"maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv\" (UID: \"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"
Sep 11 05:04:26.725454 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:04:26.725400    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-serving-cert: secret "maas-api-e2e-ipp-a-909c76-serving-cert" not found
Sep 11 05:04:26.725514 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:04:26.725466    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-maas-api-tls podName:0fb40efe-b37b-49c1-b7cb-6448b64d0ea9 nodeName:}" failed. No retries permitted until 2026-09-11 05:04:27.225447617 +0000 UTC m=+1481.696911473 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-maas-api-tls") pod "maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv" (UID: "0fb40efe-b37b-49c1-b7cb-6448b64d0ea9") : secret "maas-api-e2e-ipp-a-909c76-serving-cert" not found
Sep 11 05:04:26.725766 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.725744    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv\" (UID: \"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"
Sep 11 05:04:26.727934 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.727912    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-metrics-certs\") pod \"maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv\" (UID: \"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"
Sep 11 05:04:26.733543 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.733521    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pjq9\" (UniqueName: \"kubernetes.io/projected/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-kube-api-access-7pjq9\") pod \"maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv\" (UID: \"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"
Sep 11 05:04:26.849589 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:26.849489    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:27.099820 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:27.099726    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:27.171057 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:27.171023    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:27.231363 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:27.231277    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-maas-api-tls\") pod \"maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv\" (UID: \"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"
Sep 11 05:04:27.234756 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:27.234731    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-maas-api-tls\") pod \"maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv\" (UID: \"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"
Sep 11 05:04:27.457940 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:27.457845    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"
Sep 11 05:04:27.727646 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:27.727623    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"]
Sep 11 05:04:27.729528 ip-10-0-128-18 kubenswrapper[2567]: W0911 05:04:27.729499    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fb40efe_b37b_49c1_b7cb_6448b64d0ea9.slice/crio-1d86d97634d563df6aeeb4eb8dbd75ea19dfb232e610f0660636cba4a364fe9b WatchSource:0}: Error finding container 1d86d97634d563df6aeeb4eb8dbd75ea19dfb232e610f0660636cba4a364fe9b: Status 404 returned error can't find the container with id 1d86d97634d563df6aeeb4eb8dbd75ea19dfb232e610f0660636cba4a364fe9b
Sep 11 05:04:27.730975 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:27.730953    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 05:04:28.576876 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:28.576776    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv" event={"ID":"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9","Type":"ContainerStarted","Data":"91d18eed32c535c9c3e723c1f0089a4682a2f16a4cced138f705a4925bd1d140"}
Sep 11 05:04:28.576876 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:28.576851    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv" event={"ID":"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9","Type":"ContainerStarted","Data":"1d86d97634d563df6aeeb4eb8dbd75ea19dfb232e610f0660636cba4a364fe9b"}
Sep 11 05:04:28.577091 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:28.576879    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"
Sep 11 05:04:28.592550 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:28.592503    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv" podStartSLOduration=2.041267475 podStartE2EDuration="2.59248691s" totalImagesPullingTime="551.219435ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:04:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:04:28.592166296 +0000 UTC m=+1483.063630175" watchObservedRunningTime="2026-09-11 05:04:28.59248691 +0000 UTC m=+1483.063950785"
Sep 11 05:04:29.582776 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:29.582740    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:30.076169 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.076133    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"]
Sep 11 05:04:30.076386 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.076358    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5" podUID="24942598-c316-4136-b9f2-5dda388ae49c" containerName="maas-api" containerID="cri-o://a7d6c083ce0291cfb4489b7b59cbf2f48028b3750fe098e7330b1db1dcc4d174" gracePeriod=30
Sep 11 05:04:30.104414 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.104387    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:30.137231 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.137199    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx"]
Sep 11 05:04:30.137762 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.137413    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx" podUID="8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02" containerName="payload-processing" containerID="cri-o://1eecd81873f15aab09164eb04313e745a8ec0f90ef9475d01ae65d5766ae0ec9" gracePeriod=30
Sep 11 05:04:30.175875 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.175813    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:30.234844 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.234816    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:30.294007 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.293912    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:30.418464 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.418442    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"
Sep 11 05:04:30.421397 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.421382    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx"
Sep 11 05:04:30.563791 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.563759    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4gg4\" (UniqueName: \"kubernetes.io/projected/8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02-kube-api-access-k4gg4\") pod \"8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02\" (UID: \"8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02\") "
Sep 11 05:04:30.563978 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.563813    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24942598-c316-4136-b9f2-5dda388ae49c-metrics-certs\") pod \"24942598-c316-4136-b9f2-5dda388ae49c\" (UID: \"24942598-c316-4136-b9f2-5dda388ae49c\") "
Sep 11 05:04:30.563978 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.563885    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bbs8\" (UniqueName: \"kubernetes.io/projected/24942598-c316-4136-b9f2-5dda388ae49c-kube-api-access-5bbs8\") pod \"24942598-c316-4136-b9f2-5dda388ae49c\" (UID: \"24942598-c316-4136-b9f2-5dda388ae49c\") "
Sep 11 05:04:30.563978 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.563939    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02-plugins-config-volume\") pod \"8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02\" (UID: \"8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02\") "
Sep 11 05:04:30.563978 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.563972    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24942598-c316-4136-b9f2-5dda388ae49c-openshift-service-ca\") pod \"24942598-c316-4136-b9f2-5dda388ae49c\" (UID: \"24942598-c316-4136-b9f2-5dda388ae49c\") "
Sep 11 05:04:30.564242 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.564054    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24942598-c316-4136-b9f2-5dda388ae49c-maas-api-tls\") pod \"24942598-c316-4136-b9f2-5dda388ae49c\" (UID: \"24942598-c316-4136-b9f2-5dda388ae49c\") "
Sep 11 05:04:30.564418 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.564372    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24942598-c316-4136-b9f2-5dda388ae49c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "24942598-c316-4136-b9f2-5dda388ae49c" (UID: "24942598-c316-4136-b9f2-5dda388ae49c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:04:30.564418 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.564378    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02" (UID: "8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:04:30.566152 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.566128    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24942598-c316-4136-b9f2-5dda388ae49c-kube-api-access-5bbs8" (OuterVolumeSpecName: "kube-api-access-5bbs8") pod "24942598-c316-4136-b9f2-5dda388ae49c" (UID: "24942598-c316-4136-b9f2-5dda388ae49c"). InnerVolumeSpecName "kube-api-access-5bbs8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:04:30.566248 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.566150    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02-kube-api-access-k4gg4" (OuterVolumeSpecName: "kube-api-access-k4gg4") pod "8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02" (UID: "8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02"). InnerVolumeSpecName "kube-api-access-k4gg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:04:30.566248 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.566168    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24942598-c316-4136-b9f2-5dda388ae49c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "24942598-c316-4136-b9f2-5dda388ae49c" (UID: "24942598-c316-4136-b9f2-5dda388ae49c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:04:30.566423 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.566404    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24942598-c316-4136-b9f2-5dda388ae49c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "24942598-c316-4136-b9f2-5dda388ae49c" (UID: "24942598-c316-4136-b9f2-5dda388ae49c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:04:30.588551 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.588520    2567 generic.go:356] "Generic (PLEG): container finished" podID="8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02" containerID="1eecd81873f15aab09164eb04313e745a8ec0f90ef9475d01ae65d5766ae0ec9" exitCode=0
Sep 11 05:04:30.588909 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.588570    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx"
Sep 11 05:04:30.588909 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.588599    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx" event={"ID":"8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02","Type":"ContainerDied","Data":"1eecd81873f15aab09164eb04313e745a8ec0f90ef9475d01ae65d5766ae0ec9"}
Sep 11 05:04:30.588909 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.588649    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx" event={"ID":"8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02","Type":"ContainerDied","Data":"9eb947365a7f5064d13a43fd8a541e77efc2bf24748b70bbfab9bc770b38a6c5"}
Sep 11 05:04:30.588909 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.588673    2567 scope.go:117] "RemoveContainer" containerID="1eecd81873f15aab09164eb04313e745a8ec0f90ef9475d01ae65d5766ae0ec9"
Sep 11 05:04:30.597903 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.597841    2567 generic.go:356] "Generic (PLEG): container finished" podID="24942598-c316-4136-b9f2-5dda388ae49c" containerID="a7d6c083ce0291cfb4489b7b59cbf2f48028b3750fe098e7330b1db1dcc4d174" exitCode=0
Sep 11 05:04:30.598033 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.597928    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"
Sep 11 05:04:30.598033 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.597943    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5" event={"ID":"24942598-c316-4136-b9f2-5dda388ae49c","Type":"ContainerDied","Data":"a7d6c083ce0291cfb4489b7b59cbf2f48028b3750fe098e7330b1db1dcc4d174"}
Sep 11 05:04:30.598236 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.598034    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5" event={"ID":"24942598-c316-4136-b9f2-5dda388ae49c","Type":"ContainerDied","Data":"782806987d547cb06d92d8ffe72ac1af7dc9d8c52a6037c9e655b62e81b4b610"}
Sep 11 05:04:30.606470 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.606452    2567 scope.go:117] "RemoveContainer" containerID="1eecd81873f15aab09164eb04313e745a8ec0f90ef9475d01ae65d5766ae0ec9"
Sep 11 05:04:30.606788 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:04:30.606757    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1eecd81873f15aab09164eb04313e745a8ec0f90ef9475d01ae65d5766ae0ec9\": container with ID starting with 1eecd81873f15aab09164eb04313e745a8ec0f90ef9475d01ae65d5766ae0ec9 not found: ID does not exist" containerID="1eecd81873f15aab09164eb04313e745a8ec0f90ef9475d01ae65d5766ae0ec9"
Sep 11 05:04:30.606867 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.606789    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eecd81873f15aab09164eb04313e745a8ec0f90ef9475d01ae65d5766ae0ec9"} err="failed to get container status \"1eecd81873f15aab09164eb04313e745a8ec0f90ef9475d01ae65d5766ae0ec9\": rpc error: code = NotFound desc = could not find container \"1eecd81873f15aab09164eb04313e745a8ec0f90ef9475d01ae65d5766ae0ec9\": container with ID starting with 1eecd81873f15aab09164eb04313e745a8ec0f90ef9475d01ae65d5766ae0ec9 not found: ID does not exist"
Sep 11 05:04:30.606867 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.606813    2567 scope.go:117] "RemoveContainer" containerID="a7d6c083ce0291cfb4489b7b59cbf2f48028b3750fe098e7330b1db1dcc4d174"
Sep 11 05:04:30.616332 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.616313    2567 scope.go:117] "RemoveContainer" containerID="a7d6c083ce0291cfb4489b7b59cbf2f48028b3750fe098e7330b1db1dcc4d174"
Sep 11 05:04:30.616624 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:04:30.616601    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7d6c083ce0291cfb4489b7b59cbf2f48028b3750fe098e7330b1db1dcc4d174\": container with ID starting with a7d6c083ce0291cfb4489b7b59cbf2f48028b3750fe098e7330b1db1dcc4d174 not found: ID does not exist" containerID="a7d6c083ce0291cfb4489b7b59cbf2f48028b3750fe098e7330b1db1dcc4d174"
Sep 11 05:04:30.617124 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.616634    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7d6c083ce0291cfb4489b7b59cbf2f48028b3750fe098e7330b1db1dcc4d174"} err="failed to get container status \"a7d6c083ce0291cfb4489b7b59cbf2f48028b3750fe098e7330b1db1dcc4d174\": rpc error: code = NotFound desc = could not find container \"a7d6c083ce0291cfb4489b7b59cbf2f48028b3750fe098e7330b1db1dcc4d174\": container with ID starting with a7d6c083ce0291cfb4489b7b59cbf2f48028b3750fe098e7330b1db1dcc4d174 not found: ID does not exist"
Sep 11 05:04:30.618086 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.618049    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx"]
Sep 11 05:04:30.619439 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.619410    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:30.619993 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.619973    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-f81bf2-66bb46bb9-2rmkx"]
Sep 11 05:04:30.628431 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.628411    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"]
Sep 11 05:04:30.631917 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.631896    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-f81bf2-576cb84b4-ffrn5"]
Sep 11 05:04:30.665560 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.665531    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:04:30.665560 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.665560    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/24942598-c316-4136-b9f2-5dda388ae49c-openshift-service-ca\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:04:30.665707 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.665571    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/24942598-c316-4136-b9f2-5dda388ae49c-maas-api-tls\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:04:30.665707 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.665580    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4gg4\" (UniqueName: \"kubernetes.io/projected/8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02-kube-api-access-k4gg4\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:04:30.665707 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.665588    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24942598-c316-4136-b9f2-5dda388ae49c-metrics-certs\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:04:30.665707 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:30.665596    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bbs8\" (UniqueName: \"kubernetes.io/projected/24942598-c316-4136-b9f2-5dda388ae49c-kube-api-access-5bbs8\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:04:32.017962 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:32.017926    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24942598-c316-4136-b9f2-5dda388ae49c" path="/var/lib/kubelet/pods/24942598-c316-4136-b9f2-5dda388ae49c/volumes"
Sep 11 05:04:32.018391 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:32.018332    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02" path="/var/lib/kubelet/pods/8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02/volumes"
Sep 11 05:04:34.586816 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:34.586791    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"
Sep 11 05:04:36.556477 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:36.556450    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:36.627427 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:36.627395    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:36.817346 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:36.817230    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:47.449660 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:47.449614    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:48.962791 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:48.962752    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:50.097543 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:50.097498    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:50.382444 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:50.382354    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:51.023513 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:51.023470    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:51.327769 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:51.327693    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:52.094554 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:52.094505    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:54.047459 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:54.047421    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:54.150791 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:54.150749    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:55.155524 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.155487    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"]
Sep 11 05:04:55.155984 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.155963    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02" containerName="payload-processing"
Sep 11 05:04:55.155984 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.155982    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02" containerName="payload-processing"
Sep 11 05:04:55.155984 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.155991    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24942598-c316-4136-b9f2-5dda388ae49c" containerName="maas-api"
Sep 11 05:04:55.155984 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.155997    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="24942598-c316-4136-b9f2-5dda388ae49c" containerName="maas-api"
Sep 11 05:04:55.156316 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.156056    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="24942598-c316-4136-b9f2-5dda388ae49c" containerName="maas-api"
Sep 11 05:04:55.156316 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.156082    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f96fb36-62c6-4f29-87a5-1d0ba0b1ed02" containerName="payload-processing"
Sep 11 05:04:55.160201 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.160186    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"
Sep 11 05:04:55.162575 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.162556    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-c4902f-serving-cert\""
Sep 11 05:04:55.170175 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.170154    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"]
Sep 11 05:04:55.194625 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.194590    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/586458cb-fb1d-4851-9c6f-4a11d1223989-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d\" (UID: \"586458cb-fb1d-4851-9c6f-4a11d1223989\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"
Sep 11 05:04:55.194625 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.194622    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/586458cb-fb1d-4851-9c6f-4a11d1223989-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d\" (UID: \"586458cb-fb1d-4851-9c6f-4a11d1223989\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"
Sep 11 05:04:55.194846 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.194693    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xx65\" (UniqueName: \"kubernetes.io/projected/586458cb-fb1d-4851-9c6f-4a11d1223989-kube-api-access-5xx65\") pod \"maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d\" (UID: \"586458cb-fb1d-4851-9c6f-4a11d1223989\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"
Sep 11 05:04:55.194846 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.194816    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/586458cb-fb1d-4851-9c6f-4a11d1223989-metrics-certs\") pod \"maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d\" (UID: \"586458cb-fb1d-4851-9c6f-4a11d1223989\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"
Sep 11 05:04:55.251490 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.251436    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:55.295916 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.295884    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/586458cb-fb1d-4851-9c6f-4a11d1223989-metrics-certs\") pod \"maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d\" (UID: \"586458cb-fb1d-4851-9c6f-4a11d1223989\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"
Sep 11 05:04:55.296083 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.295923    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/586458cb-fb1d-4851-9c6f-4a11d1223989-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d\" (UID: \"586458cb-fb1d-4851-9c6f-4a11d1223989\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"
Sep 11 05:04:55.296083 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.295944    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/586458cb-fb1d-4851-9c6f-4a11d1223989-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d\" (UID: \"586458cb-fb1d-4851-9c6f-4a11d1223989\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"
Sep 11 05:04:55.296083 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.295986    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xx65\" (UniqueName: \"kubernetes.io/projected/586458cb-fb1d-4851-9c6f-4a11d1223989-kube-api-access-5xx65\") pod \"maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d\" (UID: \"586458cb-fb1d-4851-9c6f-4a11d1223989\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"
Sep 11 05:04:55.296083 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:04:55.296047    2567 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-serving-cert: secret "maas-api-e2e-ipp-b-c4902f-serving-cert" not found
Sep 11 05:04:55.296567 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:04:55.296529    2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/586458cb-fb1d-4851-9c6f-4a11d1223989-maas-api-tls podName:586458cb-fb1d-4851-9c6f-4a11d1223989 nodeName:}" failed. No retries permitted until 2026-09-11 05:04:55.796114123 +0000 UTC m=+1510.267577977 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/586458cb-fb1d-4851-9c6f-4a11d1223989-maas-api-tls") pod "maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d" (UID: "586458cb-fb1d-4851-9c6f-4a11d1223989") : secret "maas-api-e2e-ipp-b-c4902f-serving-cert" not found
Sep 11 05:04:55.305549 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.297234    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/586458cb-fb1d-4851-9c6f-4a11d1223989-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d\" (UID: \"586458cb-fb1d-4851-9c6f-4a11d1223989\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"
Sep 11 05:04:55.305549 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.299582    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/586458cb-fb1d-4851-9c6f-4a11d1223989-metrics-certs\") pod \"maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d\" (UID: \"586458cb-fb1d-4851-9c6f-4a11d1223989\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"
Sep 11 05:04:55.305549 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.305481    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xx65\" (UniqueName: \"kubernetes.io/projected/586458cb-fb1d-4851-9c6f-4a11d1223989-kube-api-access-5xx65\") pod \"maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d\" (UID: \"586458cb-fb1d-4851-9c6f-4a11d1223989\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"
Sep 11 05:04:55.353253 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.353204    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:55.452947 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.452871    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:55.681662 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.681620    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:55.786049 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.786014    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p"]
Sep 11 05:04:55.790030 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.790009    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p"
Sep 11 05:04:55.792231 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.792214    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-c4902f-dockercfg-cjmz6\""
Sep 11 05:04:55.792299 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.792242    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-c4902f\""
Sep 11 05:04:55.801377 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.801350    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/586458cb-fb1d-4851-9c6f-4a11d1223989-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d\" (UID: \"586458cb-fb1d-4851-9c6f-4a11d1223989\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"
Sep 11 05:04:55.801728 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.801711    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p"]
Sep 11 05:04:55.803981 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.803960    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/586458cb-fb1d-4851-9c6f-4a11d1223989-maas-api-tls\") pod \"maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d\" (UID: \"586458cb-fb1d-4851-9c6f-4a11d1223989\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"
Sep 11 05:04:55.807658 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.807638    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw"]
Sep 11 05:04:55.811088 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.811051    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw"
Sep 11 05:04:55.823661 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.823638    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw"]
Sep 11 05:04:55.902486 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.902449    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/866ad27d-598c-4156-b02c-0f842e988593-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw\" (UID: \"866ad27d-598c-4156-b02c-0f842e988593\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw"
Sep 11 05:04:55.902656 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.902553    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f568534-d42f-4808-86a0-f23f22861913-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p\" (UID: \"7f568534-d42f-4808-86a0-f23f22861913\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p"
Sep 11 05:04:55.902656 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.902604    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2wmz\" (UniqueName: \"kubernetes.io/projected/866ad27d-598c-4156-b02c-0f842e988593-kube-api-access-x2wmz\") pod \"payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw\" (UID: \"866ad27d-598c-4156-b02c-0f842e988593\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw"
Sep 11 05:04:55.902745 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:55.902683    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swl25\" (UniqueName: \"kubernetes.io/projected/7f568534-d42f-4808-86a0-f23f22861913-kube-api-access-swl25\") pod \"payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p\" (UID: \"7f568534-d42f-4808-86a0-f23f22861913\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p"
Sep 11 05:04:56.003975 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.003942    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f568534-d42f-4808-86a0-f23f22861913-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p\" (UID: \"7f568534-d42f-4808-86a0-f23f22861913\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p"
Sep 11 05:04:56.004189 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.003996    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2wmz\" (UniqueName: \"kubernetes.io/projected/866ad27d-598c-4156-b02c-0f842e988593-kube-api-access-x2wmz\") pod \"payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw\" (UID: \"866ad27d-598c-4156-b02c-0f842e988593\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw"
Sep 11 05:04:56.004189 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.004040    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swl25\" (UniqueName: \"kubernetes.io/projected/7f568534-d42f-4808-86a0-f23f22861913-kube-api-access-swl25\") pod \"payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p\" (UID: \"7f568534-d42f-4808-86a0-f23f22861913\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p"
Sep 11 05:04:56.004189 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.004081    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/866ad27d-598c-4156-b02c-0f842e988593-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw\" (UID: \"866ad27d-598c-4156-b02c-0f842e988593\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw"
Sep 11 05:04:56.004727 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.004704    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/866ad27d-598c-4156-b02c-0f842e988593-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw\" (UID: \"866ad27d-598c-4156-b02c-0f842e988593\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw"
Sep 11 05:04:56.004727 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.004717    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f568534-d42f-4808-86a0-f23f22861913-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p\" (UID: \"7f568534-d42f-4808-86a0-f23f22861913\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p"
Sep 11 05:04:56.012325 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.012298    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2wmz\" (UniqueName: \"kubernetes.io/projected/866ad27d-598c-4156-b02c-0f842e988593-kube-api-access-x2wmz\") pod \"payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw\" (UID: \"866ad27d-598c-4156-b02c-0f842e988593\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw"
Sep 11 05:04:56.012809 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.012788    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-swl25\" (UniqueName: \"kubernetes.io/projected/7f568534-d42f-4808-86a0-f23f22861913-kube-api-access-swl25\") pod \"payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p\" (UID: \"7f568534-d42f-4808-86a0-f23f22861913\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p"
Sep 11 05:04:56.071804 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.071731    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"
Sep 11 05:04:56.101512 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.101486    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p"
Sep 11 05:04:56.124802 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.124771    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw"
Sep 11 05:04:56.359770 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.359734    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"]
Sep 11 05:04:56.361464 ip-10-0-128-18 kubenswrapper[2567]: W0911 05:04:56.361442    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod586458cb_fb1d_4851_9c6f_4a11d1223989.slice/crio-9e86ebbbb00e3149ffb3badc40b976906d20d743af96c047d5f5201b6d8b6769 WatchSource:0}: Error finding container 9e86ebbbb00e3149ffb3badc40b976906d20d743af96c047d5f5201b6d8b6769: Status 404 returned error can't find the container with id 9e86ebbbb00e3149ffb3badc40b976906d20d743af96c047d5f5201b6d8b6769
Sep 11 05:04:56.406261 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.406233    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p"]
Sep 11 05:04:56.406641 ip-10-0-128-18 kubenswrapper[2567]: W0911 05:04:56.406614    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f568534_d42f_4808_86a0_f23f22861913.slice/crio-ab06f217bd02f267e8e1d47f519307b01ccad3d1da5d82777a8fce2325ec5065 WatchSource:0}: Error finding container ab06f217bd02f267e8e1d47f519307b01ccad3d1da5d82777a8fce2325ec5065: Status 404 returned error can't find the container with id ab06f217bd02f267e8e1d47f519307b01ccad3d1da5d82777a8fce2325ec5065
Sep 11 05:04:56.433732 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.433706    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw"]
Sep 11 05:04:56.434563 ip-10-0-128-18 kubenswrapper[2567]: W0911 05:04:56.434537    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866ad27d_598c_4156_b02c_0f842e988593.slice/crio-8481fff8c1103ded35097e306b124e7dc8f79123c7186353a588501e2397af46 WatchSource:0}: Error finding container 8481fff8c1103ded35097e306b124e7dc8f79123c7186353a588501e2397af46: Status 404 returned error can't find the container with id 8481fff8c1103ded35097e306b124e7dc8f79123c7186353a588501e2397af46
Sep 11 05:04:56.473807 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.473783    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:04:56.735646 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.735616    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p" event={"ID":"7f568534-d42f-4808-86a0-f23f22861913","Type":"ContainerStarted","Data":"ee9160400461c39fab2430b1f690353753b909a128d3da6526efa6a5546c3a7f"}
Sep 11 05:04:56.735764 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.735683    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p" event={"ID":"7f568534-d42f-4808-86a0-f23f22861913","Type":"ContainerStarted","Data":"ab06f217bd02f267e8e1d47f519307b01ccad3d1da5d82777a8fce2325ec5065"}
Sep 11 05:04:56.735868 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.735844    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p"
Sep 11 05:04:56.737587 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.737563    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw" event={"ID":"866ad27d-598c-4156-b02c-0f842e988593","Type":"ContainerStarted","Data":"f59ee375608af4fb356ee0bc01dfcae04bc67b6274892b37754bd21822cd512d"}
Sep 11 05:04:56.737690 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.737612    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw" event={"ID":"866ad27d-598c-4156-b02c-0f842e988593","Type":"ContainerStarted","Data":"8481fff8c1103ded35097e306b124e7dc8f79123c7186353a588501e2397af46"}
Sep 11 05:04:56.737690 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.737639    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw"
Sep 11 05:04:56.739105 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.739083    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d" event={"ID":"586458cb-fb1d-4851-9c6f-4a11d1223989","Type":"ContainerStarted","Data":"9e86ebbbb00e3149ffb3badc40b976906d20d743af96c047d5f5201b6d8b6769"}
Sep 11 05:04:56.751432 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.751389    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p" podStartSLOduration=1.751373402 podStartE2EDuration="1.751373402s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:04:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:04:56.74977884 +0000 UTC m=+1511.221242730" watchObservedRunningTime="2026-09-11 05:04:56.751373402 +0000 UTC m=+1511.222837342"
Sep 11 05:04:56.765722 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:56.765679    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw" podStartSLOduration=1.7656651719999998 podStartE2EDuration="1.765665172s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:04:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:04:56.76441482 +0000 UTC m=+1511.235878696" watchObservedRunningTime="2026-09-11 05:04:56.765665172 +0000 UTC m=+1511.237129047"
Sep 11 05:04:57.746500 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:57.746460    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d" event={"ID":"586458cb-fb1d-4851-9c6f-4a11d1223989","Type":"ContainerStarted","Data":"41987aef6b3c06ef02fc73db0fee7003e2e325c0ed5fba2075011eddbca6e261"}
Sep 11 05:04:57.746995 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:57.746859    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"
Sep 11 05:04:57.765506 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:04:57.765460    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d" podStartSLOduration=2.412350629 podStartE2EDuration="2.765445022s" totalImagesPullingTime="353.094393ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:04:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:04:57.763651768 +0000 UTC m=+1512.235115645" watchObservedRunningTime="2026-09-11 05:04:57.765445022 +0000 UTC m=+1512.236908898"
Sep 11 05:05:03.756673 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:05:03.756643    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"
Sep 11 05:05:07.748894 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:05:07.748819    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw"
Sep 11 05:05:07.749287 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:05:07.748990    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p"
Sep 11 05:05:19.768374 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:05:19.768335    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:05:23.921536 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:05:23.921501    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-4wnbl_9d36da1d-c48a-4391-b2eb-5ffc687f1164/payload-processing/0.log"
Sep 11 05:05:24.030818 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:05:24.030774    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw_866ad27d-598c-4156-b02c-0f842e988593/payload-processing/0.log"
Sep 11 05:05:28.361903 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:05:28.361867    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:05:29.420094 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:05:29.420038    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:05:41.297691 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:05:41.297650    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:05:41.514912 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:05:41.514868    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:05:54.567866 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:05:54.567827    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-4wnbl_9d36da1d-c48a-4391-b2eb-5ffc687f1164/payload-processing/0.log"
Sep 11 05:05:54.683093 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:05:54.683041    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw_866ad27d-598c-4156-b02c-0f842e988593/payload-processing/0.log"
Sep 11 05:05:54.959649 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:05:54.959547    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:05:56.152282 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:05:56.152229    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:05:58.422059 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:05:58.422020    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:05:58.525050 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:05:58.524984    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:05:59.686041 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:05:59.686011    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:05:59.783176 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:05:59.783125    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:06:00.223898 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.223865    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:06:00.323763 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.323733    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:06:00.339558 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.339530    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf"]
Sep 11 05:06:00.343354 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.343339    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf"
Sep 11 05:06:00.345722 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.345702    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-d69099-dockercfg-qr8xn\""
Sep 11 05:06:00.345841 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.345750    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-d69099\""
Sep 11 05:06:00.355152 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.355091    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf"]
Sep 11 05:06:00.366692 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.366661    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl"]
Sep 11 05:06:00.371039 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.371018    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl"
Sep 11 05:06:00.381960 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.381935    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl"]
Sep 11 05:06:00.430344 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.430309    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb692b75-60df-413a-92c9-2aed7ba93b20-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl\" (UID: \"eb692b75-60df-413a-92c9-2aed7ba93b20\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl"
Sep 11 05:06:00.430502 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.430437    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79875611-ced2-408c-89dc-461f0e2fa730-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf\" (UID: \"79875611-ced2-408c-89dc-461f0e2fa730\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf"
Sep 11 05:06:00.430550 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.430505    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfn89\" (UniqueName: \"kubernetes.io/projected/79875611-ced2-408c-89dc-461f0e2fa730-kube-api-access-cfn89\") pod \"payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf\" (UID: \"79875611-ced2-408c-89dc-461f0e2fa730\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf"
Sep 11 05:06:00.430550 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.430528    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stfx9\" (UniqueName: \"kubernetes.io/projected/eb692b75-60df-413a-92c9-2aed7ba93b20-kube-api-access-stfx9\") pod \"payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl\" (UID: \"eb692b75-60df-413a-92c9-2aed7ba93b20\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl"
Sep 11 05:06:00.532047 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.532018    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stfx9\" (UniqueName: \"kubernetes.io/projected/eb692b75-60df-413a-92c9-2aed7ba93b20-kube-api-access-stfx9\") pod \"payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl\" (UID: \"eb692b75-60df-413a-92c9-2aed7ba93b20\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl"
Sep 11 05:06:00.532235 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.532096    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb692b75-60df-413a-92c9-2aed7ba93b20-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl\" (UID: \"eb692b75-60df-413a-92c9-2aed7ba93b20\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl"
Sep 11 05:06:00.532235 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.532178    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79875611-ced2-408c-89dc-461f0e2fa730-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf\" (UID: \"79875611-ced2-408c-89dc-461f0e2fa730\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf"
Sep 11 05:06:00.532235 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.532221    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfn89\" (UniqueName: \"kubernetes.io/projected/79875611-ced2-408c-89dc-461f0e2fa730-kube-api-access-cfn89\") pod \"payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf\" (UID: \"79875611-ced2-408c-89dc-461f0e2fa730\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf"
Sep 11 05:06:00.532885 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.532863    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb692b75-60df-413a-92c9-2aed7ba93b20-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl\" (UID: \"eb692b75-60df-413a-92c9-2aed7ba93b20\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl"
Sep 11 05:06:00.532948 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.532891    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79875611-ced2-408c-89dc-461f0e2fa730-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf\" (UID: \"79875611-ced2-408c-89dc-461f0e2fa730\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf"
Sep 11 05:06:00.539871 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.539849    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfn89\" (UniqueName: \"kubernetes.io/projected/79875611-ced2-408c-89dc-461f0e2fa730-kube-api-access-cfn89\") pod \"payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf\" (UID: \"79875611-ced2-408c-89dc-461f0e2fa730\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf"
Sep 11 05:06:00.539981 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.539955    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stfx9\" (UniqueName: \"kubernetes.io/projected/eb692b75-60df-413a-92c9-2aed7ba93b20-kube-api-access-stfx9\") pod \"payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl\" (UID: \"eb692b75-60df-413a-92c9-2aed7ba93b20\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl"
Sep 11 05:06:00.657926 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.657884    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf"
Sep 11 05:06:00.683389 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.683362    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl"
Sep 11 05:06:00.931948 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.931925    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf"]
Sep 11 05:06:00.934218 ip-10-0-128-18 kubenswrapper[2567]: W0911 05:06:00.934190    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79875611_ced2_408c_89dc_461f0e2fa730.slice/crio-7424daa80bc34fbf244720f28df600c37b9403dcf5a03a50d7a69ab6de169230 WatchSource:0}: Error finding container 7424daa80bc34fbf244720f28df600c37b9403dcf5a03a50d7a69ab6de169230: Status 404 returned error can't find the container with id 7424daa80bc34fbf244720f28df600c37b9403dcf5a03a50d7a69ab6de169230
Sep 11 05:06:00.973979 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:00.973953    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl"]
Sep 11 05:06:00.975732 ip-10-0-128-18 kubenswrapper[2567]: W0911 05:06:00.975710    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb692b75_60df_413a_92c9_2aed7ba93b20.slice/crio-4236fea500b5803bf8af1579c5abf8d75b78391ab19c79356c105ba1c06e8178 WatchSource:0}: Error finding container 4236fea500b5803bf8af1579c5abf8d75b78391ab19c79356c105ba1c06e8178: Status 404 returned error can't find the container with id 4236fea500b5803bf8af1579c5abf8d75b78391ab19c79356c105ba1c06e8178
Sep 11 05:06:01.073411 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:01.073372    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl" event={"ID":"eb692b75-60df-413a-92c9-2aed7ba93b20","Type":"ContainerStarted","Data":"c463b11a68bce516750e4c123f0ca627420770b77e7ca166fdebdd69806ff76b"}
Sep 11 05:06:01.073621 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:01.073442    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl" event={"ID":"eb692b75-60df-413a-92c9-2aed7ba93b20","Type":"ContainerStarted","Data":"4236fea500b5803bf8af1579c5abf8d75b78391ab19c79356c105ba1c06e8178"}
Sep 11 05:06:01.073621 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:01.073508    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl"
Sep 11 05:06:01.075215 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:01.075191    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf" event={"ID":"79875611-ced2-408c-89dc-461f0e2fa730","Type":"ContainerStarted","Data":"dafe84ba7cb3f19521c30e0ed6d26d9516a845e42a415c5304e63a61b578f04f"}
Sep 11 05:06:01.075332 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:01.075229    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf" event={"ID":"79875611-ced2-408c-89dc-461f0e2fa730","Type":"ContainerStarted","Data":"7424daa80bc34fbf244720f28df600c37b9403dcf5a03a50d7a69ab6de169230"}
Sep 11 05:06:01.075332 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:01.075310    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf"
Sep 11 05:06:01.089249 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:01.089196    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl" podStartSLOduration=1.089182381 podStartE2EDuration="1.089182381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:06:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:06:01.0876639 +0000 UTC m=+1575.559127783" watchObservedRunningTime="2026-09-11 05:06:01.089182381 +0000 UTC m=+1575.560646257"
Sep 11 05:06:01.104019 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:01.103971    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf" podStartSLOduration=1.103952488 podStartE2EDuration="1.103952488s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:06:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:06:01.101660509 +0000 UTC m=+1575.573124386" watchObservedRunningTime="2026-09-11 05:06:01.103952488 +0000 UTC m=+1575.575416363"
Sep 11 05:06:12.082807 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:12.082769    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf"
Sep 11 05:06:12.083318 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:12.082855    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl"
Sep 11 05:06:23.393773 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:23.393728    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:06:30.161705 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:30.161668    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:06:30.190034 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:30.190000    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl"]
Sep 11 05:06:30.190733 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:30.190698    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl" podUID="eb692b75-60df-413a-92c9-2aed7ba93b20" containerName="payload-processing" containerID="cri-o://c463b11a68bce516750e4c123f0ca627420770b77e7ca166fdebdd69806ff76b" gracePeriod=30
Sep 11 05:06:30.193791 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:30.193764    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf"]
Sep 11 05:06:30.193964 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:30.193942    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf" podUID="79875611-ced2-408c-89dc-461f0e2fa730" containerName="payload-pre-processing" containerID="cri-o://dafe84ba7cb3f19521c30e0ed6d26d9516a845e42a415c5304e63a61b578f04f" gracePeriod=30
Sep 11 05:06:30.228242 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:30.228205    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:06:30.324730 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:30.324696    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:06:30.380817 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:30.380786    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:06:30.487909 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:30.487884    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl"
Sep 11 05:06:30.490683 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:30.490665    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf"
Sep 11 05:06:30.623853 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:30.623824    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stfx9\" (UniqueName: \"kubernetes.io/projected/eb692b75-60df-413a-92c9-2aed7ba93b20-kube-api-access-stfx9\") pod \"eb692b75-60df-413a-92c9-2aed7ba93b20\" (UID: \"eb692b75-60df-413a-92c9-2aed7ba93b20\") "
Sep 11 05:06:30.624042 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:30.623891    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb692b75-60df-413a-92c9-2aed7ba93b20-plugins-config-volume\") pod \"eb692b75-60df-413a-92c9-2aed7ba93b20\" (UID: \"eb692b75-60df-413a-92c9-2aed7ba93b20\") "
Sep 11 05:06:30.624042 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:30.623926    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79875611-ced2-408c-89dc-461f0e2fa730-plugins-config-volume\") pod \"79875611-ced2-408c-89dc-461f0e2fa730\" (UID: \"79875611-ced2-408c-89dc-461f0e2fa730\") "
Sep 11 05:06:30.624042 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:30.623954    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfn89\" (UniqueName: \"kubernetes.io/projected/79875611-ced2-408c-89dc-461f0e2fa730-kube-api-access-cfn89\") pod \"79875611-ced2-408c-89dc-461f0e2fa730\" (UID: \"79875611-ced2-408c-89dc-461f0e2fa730\") "
Sep 11 05:06:30.624363 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:30.624339    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79875611-ced2-408c-89dc-461f0e2fa730-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "79875611-ced2-408c-89dc-461f0e2fa730" (UID: "79875611-ced2-408c-89dc-461f0e2fa730"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:06:30.624430 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:30.624341    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb692b75-60df-413a-92c9-2aed7ba93b20-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eb692b75-60df-413a-92c9-2aed7ba93b20" (UID: "eb692b75-60df-413a-92c9-2aed7ba93b20"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:06:30.626325 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:30.626293    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb692b75-60df-413a-92c9-2aed7ba93b20-kube-api-access-stfx9" (OuterVolumeSpecName: "kube-api-access-stfx9") pod "eb692b75-60df-413a-92c9-2aed7ba93b20" (UID: "eb692b75-60df-413a-92c9-2aed7ba93b20"). InnerVolumeSpecName "kube-api-access-stfx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:06:30.626434 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:30.626328    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79875611-ced2-408c-89dc-461f0e2fa730-kube-api-access-cfn89" (OuterVolumeSpecName: "kube-api-access-cfn89") pod "79875611-ced2-408c-89dc-461f0e2fa730" (UID: "79875611-ced2-408c-89dc-461f0e2fa730"). InnerVolumeSpecName "kube-api-access-cfn89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:06:30.680924 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:30.680892    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:06:30.725663 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:30.725583    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79875611-ced2-408c-89dc-461f0e2fa730-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:06:30.725663 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:30.725608    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfn89\" (UniqueName: \"kubernetes.io/projected/79875611-ced2-408c-89dc-461f0e2fa730-kube-api-access-cfn89\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:06:30.725663 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:30.725622    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stfx9\" (UniqueName: \"kubernetes.io/projected/eb692b75-60df-413a-92c9-2aed7ba93b20-kube-api-access-stfx9\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:06:30.725663 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:30.725633    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb692b75-60df-413a-92c9-2aed7ba93b20-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:06:31.234542 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:31.234502    2567 generic.go:356] "Generic (PLEG): container finished" podID="eb692b75-60df-413a-92c9-2aed7ba93b20" containerID="c463b11a68bce516750e4c123f0ca627420770b77e7ca166fdebdd69806ff76b" exitCode=0
Sep 11 05:06:31.235002 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:31.234547    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl"
Sep 11 05:06:31.235002 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:31.234593    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl" event={"ID":"eb692b75-60df-413a-92c9-2aed7ba93b20","Type":"ContainerDied","Data":"c463b11a68bce516750e4c123f0ca627420770b77e7ca166fdebdd69806ff76b"}
Sep 11 05:06:31.235002 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:31.234651    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl" event={"ID":"eb692b75-60df-413a-92c9-2aed7ba93b20","Type":"ContainerDied","Data":"4236fea500b5803bf8af1579c5abf8d75b78391ab19c79356c105ba1c06e8178"}
Sep 11 05:06:31.235002 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:31.234686    2567 scope.go:117] "RemoveContainer" containerID="c463b11a68bce516750e4c123f0ca627420770b77e7ca166fdebdd69806ff76b"
Sep 11 05:06:31.236198 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:31.236099    2567 generic.go:356] "Generic (PLEG): container finished" podID="79875611-ced2-408c-89dc-461f0e2fa730" containerID="dafe84ba7cb3f19521c30e0ed6d26d9516a845e42a415c5304e63a61b578f04f" exitCode=0
Sep 11 05:06:31.236198 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:31.236192    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf"
Sep 11 05:06:31.236292 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:31.236245    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf" event={"ID":"79875611-ced2-408c-89dc-461f0e2fa730","Type":"ContainerDied","Data":"dafe84ba7cb3f19521c30e0ed6d26d9516a845e42a415c5304e63a61b578f04f"}
Sep 11 05:06:31.236331 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:31.236291    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf" event={"ID":"79875611-ced2-408c-89dc-461f0e2fa730","Type":"ContainerDied","Data":"7424daa80bc34fbf244720f28df600c37b9403dcf5a03a50d7a69ab6de169230"}
Sep 11 05:06:31.253504 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:31.253469    2567 scope.go:117] "RemoveContainer" containerID="c463b11a68bce516750e4c123f0ca627420770b77e7ca166fdebdd69806ff76b"
Sep 11 05:06:31.254713 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:06:31.254679    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c463b11a68bce516750e4c123f0ca627420770b77e7ca166fdebdd69806ff76b\": container with ID starting with c463b11a68bce516750e4c123f0ca627420770b77e7ca166fdebdd69806ff76b not found: ID does not exist" containerID="c463b11a68bce516750e4c123f0ca627420770b77e7ca166fdebdd69806ff76b"
Sep 11 05:06:31.254837 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:31.254725    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c463b11a68bce516750e4c123f0ca627420770b77e7ca166fdebdd69806ff76b"} err="failed to get container status \"c463b11a68bce516750e4c123f0ca627420770b77e7ca166fdebdd69806ff76b\": rpc error: code = NotFound desc = could not find container \"c463b11a68bce516750e4c123f0ca627420770b77e7ca166fdebdd69806ff76b\": container with ID starting with c463b11a68bce516750e4c123f0ca627420770b77e7ca166fdebdd69806ff76b not found: ID does not exist"
Sep 11 05:06:31.254837 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:31.254751    2567 scope.go:117] "RemoveContainer" containerID="dafe84ba7cb3f19521c30e0ed6d26d9516a845e42a415c5304e63a61b578f04f"
Sep 11 05:06:31.263614 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:31.263589    2567 scope.go:117] "RemoveContainer" containerID="dafe84ba7cb3f19521c30e0ed6d26d9516a845e42a415c5304e63a61b578f04f"
Sep 11 05:06:31.263921 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:06:31.263903    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dafe84ba7cb3f19521c30e0ed6d26d9516a845e42a415c5304e63a61b578f04f\": container with ID starting with dafe84ba7cb3f19521c30e0ed6d26d9516a845e42a415c5304e63a61b578f04f not found: ID does not exist" containerID="dafe84ba7cb3f19521c30e0ed6d26d9516a845e42a415c5304e63a61b578f04f"
Sep 11 05:06:31.263968 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:31.263931    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dafe84ba7cb3f19521c30e0ed6d26d9516a845e42a415c5304e63a61b578f04f"} err="failed to get container status \"dafe84ba7cb3f19521c30e0ed6d26d9516a845e42a415c5304e63a61b578f04f\": rpc error: code = NotFound desc = could not find container \"dafe84ba7cb3f19521c30e0ed6d26d9516a845e42a415c5304e63a61b578f04f\": container with ID starting with dafe84ba7cb3f19521c30e0ed6d26d9516a845e42a415c5304e63a61b578f04f not found: ID does not exist"
Sep 11 05:06:31.268081 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:31.268046    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl"]
Sep 11 05:06:31.271448 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:31.271426    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d69099-5449b6d65c-b4wdl"]
Sep 11 05:06:31.281335 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:31.281314    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf"]
Sep 11 05:06:31.284375 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:31.284355    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d69099-6ddc577858-6njxf"]
Sep 11 05:06:32.017583 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:32.017547    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79875611-ced2-408c-89dc-461f0e2fa730" path="/var/lib/kubelet/pods/79875611-ced2-408c-89dc-461f0e2fa730/volumes"
Sep 11 05:06:32.017917 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:32.017904    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb692b75-60df-413a-92c9-2aed7ba93b20" path="/var/lib/kubelet/pods/eb692b75-60df-413a-92c9-2aed7ba93b20/volumes"
Sep 11 05:06:36.327834 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:36.327793    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:06:36.429575 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:36.429537    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:06:36.582787 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:36.582671    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:06:40.116199 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:40.116168    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:06:55.424042 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:55.424004    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"]
Sep 11 05:06:55.424532 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:55.424293    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv" podUID="0fb40efe-b37b-49c1-b7cb-6448b64d0ea9" containerName="maas-api" containerID="cri-o://91d18eed32c535c9c3e723c1f0089a4682a2f16a4cced138f705a4925bd1d140" gracePeriod=30
Sep 11 05:06:55.456163 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:55.456098    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:06:55.554119 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:55.554058    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:06:55.627861 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:55.627826    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:06:55.681013 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:55.680936    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:06:55.684952 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:55.684931    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"
Sep 11 05:06:55.759692 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:55.759664    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-openshift-service-ca\") pod \"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9\" (UID: \"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9\") "
Sep 11 05:06:55.759887 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:55.759716    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-maas-api-tls\") pod \"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9\" (UID: \"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9\") "
Sep 11 05:06:55.759887 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:55.759804    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7pjq9\" (UniqueName: \"kubernetes.io/projected/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-kube-api-access-7pjq9\") pod \"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9\" (UID: \"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9\") "
Sep 11 05:06:55.760016 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:55.759925    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-metrics-certs\") pod \"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9\" (UID: \"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9\") "
Sep 11 05:06:55.760114 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:55.760041    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0fb40efe-b37b-49c1-b7cb-6448b64d0ea9" (UID: "0fb40efe-b37b-49c1-b7cb-6448b64d0ea9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:06:55.760330 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:55.760307    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-openshift-service-ca\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:06:55.762123 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:55.762099    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-kube-api-access-7pjq9" (OuterVolumeSpecName: "kube-api-access-7pjq9") pod "0fb40efe-b37b-49c1-b7cb-6448b64d0ea9" (UID: "0fb40efe-b37b-49c1-b7cb-6448b64d0ea9"). InnerVolumeSpecName "kube-api-access-7pjq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:06:55.762217 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:55.762195    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0fb40efe-b37b-49c1-b7cb-6448b64d0ea9" (UID: "0fb40efe-b37b-49c1-b7cb-6448b64d0ea9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:06:55.762282 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:55.762246    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0fb40efe-b37b-49c1-b7cb-6448b64d0ea9" (UID: "0fb40efe-b37b-49c1-b7cb-6448b64d0ea9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:06:55.860831 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:55.860795    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pjq9\" (UniqueName: \"kubernetes.io/projected/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-kube-api-access-7pjq9\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:06:55.860831 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:55.860825    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-metrics-certs\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:06:55.860831 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:55.860837    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9-maas-api-tls\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:06:55.989292 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:55.989202    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:06:56.366127 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:56.366087    2567 generic.go:356] "Generic (PLEG): container finished" podID="0fb40efe-b37b-49c1-b7cb-6448b64d0ea9" containerID="91d18eed32c535c9c3e723c1f0089a4682a2f16a4cced138f705a4925bd1d140" exitCode=0
Sep 11 05:06:56.366127 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:56.366123    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv" event={"ID":"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9","Type":"ContainerDied","Data":"91d18eed32c535c9c3e723c1f0089a4682a2f16a4cced138f705a4925bd1d140"}
Sep 11 05:06:56.366365 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:56.366145    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"
Sep 11 05:06:56.366365 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:56.366182    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv" event={"ID":"0fb40efe-b37b-49c1-b7cb-6448b64d0ea9","Type":"ContainerDied","Data":"1d86d97634d563df6aeeb4eb8dbd75ea19dfb232e610f0660636cba4a364fe9b"}
Sep 11 05:06:56.366365 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:56.366218    2567 scope.go:117] "RemoveContainer" containerID="91d18eed32c535c9c3e723c1f0089a4682a2f16a4cced138f705a4925bd1d140"
Sep 11 05:06:56.375376 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:56.375360    2567 scope.go:117] "RemoveContainer" containerID="91d18eed32c535c9c3e723c1f0089a4682a2f16a4cced138f705a4925bd1d140"
Sep 11 05:06:56.375645 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:06:56.375619    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91d18eed32c535c9c3e723c1f0089a4682a2f16a4cced138f705a4925bd1d140\": container with ID starting with 91d18eed32c535c9c3e723c1f0089a4682a2f16a4cced138f705a4925bd1d140 not found: ID does not exist" containerID="91d18eed32c535c9c3e723c1f0089a4682a2f16a4cced138f705a4925bd1d140"
Sep 11 05:06:56.375713 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:56.375653    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91d18eed32c535c9c3e723c1f0089a4682a2f16a4cced138f705a4925bd1d140"} err="failed to get container status \"91d18eed32c535c9c3e723c1f0089a4682a2f16a4cced138f705a4925bd1d140\": rpc error: code = NotFound desc = could not find container \"91d18eed32c535c9c3e723c1f0089a4682a2f16a4cced138f705a4925bd1d140\": container with ID starting with 91d18eed32c535c9c3e723c1f0089a4682a2f16a4cced138f705a4925bd1d140 not found: ID does not exist"
Sep 11 05:06:56.384947 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:56.384926    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"]
Sep 11 05:06:56.386767 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:56.386747    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-909c76-79f79f87f4-4mdsv"]
Sep 11 05:06:58.017271 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:06:58.017234    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fb40efe-b37b-49c1-b7cb-6448b64d0ea9" path="/var/lib/kubelet/pods/0fb40efe-b37b-49c1-b7cb-6448b64d0ea9/volumes"
Sep 11 05:07:01.009425 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:01.009362    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:07:22.962279 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:22.962235    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:07:37.544872 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:37.544837    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:07:37.868598 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:37.868507    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:07:43.288892 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:43.288852    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:07:43.392045 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:43.392011    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:07:43.548095 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:43.547973    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:07:43.971589 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:43.971501    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:07:49.779578 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:49.779541    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:07:54.177291 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:54.177226    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:07:54.732515 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:54.732480    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"]
Sep 11 05:07:54.732781 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:54.732728    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d" podUID="586458cb-fb1d-4851-9c6f-4a11d1223989" containerName="maas-api" containerID="cri-o://41987aef6b3c06ef02fc73db0fee7003e2e325c0ed5fba2075011eddbca6e261" gracePeriod=30
Sep 11 05:07:54.761258 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:54.761233    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:07:54.789486 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:54.789458    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw"]
Sep 11 05:07:54.789711 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:54.789686    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw" podUID="866ad27d-598c-4156-b02c-0f842e988593" containerName="payload-processing" containerID="cri-o://f59ee375608af4fb356ee0bc01dfcae04bc67b6274892b37754bd21822cd512d" gracePeriod=30
Sep 11 05:07:54.796275 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:54.796230    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p"]
Sep 11 05:07:54.796549 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:54.796485    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p" podUID="7f568534-d42f-4808-86a0-f23f22861913" containerName="payload-pre-processing" containerID="cri-o://ee9160400461c39fab2430b1f690353753b909a128d3da6526efa6a5546c3a7f" gracePeriod=30
Sep 11 05:07:54.845663 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:54.845631    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:07:54.933561 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:54.933479    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:07:54.998649 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:54.998488    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:07:55.053297 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.053243    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:07:55.058940 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.058914    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"
Sep 11 05:07:55.101967 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.101944    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw"
Sep 11 05:07:55.109250 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.109225    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p"
Sep 11 05:07:55.209374 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.209343    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/866ad27d-598c-4156-b02c-0f842e988593-plugins-config-volume\") pod \"866ad27d-598c-4156-b02c-0f842e988593\" (UID: \"866ad27d-598c-4156-b02c-0f842e988593\") "
Sep 11 05:07:55.209819 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.209385    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xx65\" (UniqueName: \"kubernetes.io/projected/586458cb-fb1d-4851-9c6f-4a11d1223989-kube-api-access-5xx65\") pod \"586458cb-fb1d-4851-9c6f-4a11d1223989\" (UID: \"586458cb-fb1d-4851-9c6f-4a11d1223989\") "
Sep 11 05:07:55.209819 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.209434    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2wmz\" (UniqueName: \"kubernetes.io/projected/866ad27d-598c-4156-b02c-0f842e988593-kube-api-access-x2wmz\") pod \"866ad27d-598c-4156-b02c-0f842e988593\" (UID: \"866ad27d-598c-4156-b02c-0f842e988593\") "
Sep 11 05:07:55.209819 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.209450    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swl25\" (UniqueName: \"kubernetes.io/projected/7f568534-d42f-4808-86a0-f23f22861913-kube-api-access-swl25\") pod \"7f568534-d42f-4808-86a0-f23f22861913\" (UID: \"7f568534-d42f-4808-86a0-f23f22861913\") "
Sep 11 05:07:55.209819 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.209474    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f568534-d42f-4808-86a0-f23f22861913-plugins-config-volume\") pod \"7f568534-d42f-4808-86a0-f23f22861913\" (UID: \"7f568534-d42f-4808-86a0-f23f22861913\") "
Sep 11 05:07:55.209819 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.209513    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/586458cb-fb1d-4851-9c6f-4a11d1223989-openshift-service-ca\") pod \"586458cb-fb1d-4851-9c6f-4a11d1223989\" (UID: \"586458cb-fb1d-4851-9c6f-4a11d1223989\") "
Sep 11 05:07:55.209819 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.209536    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/586458cb-fb1d-4851-9c6f-4a11d1223989-metrics-certs\") pod \"586458cb-fb1d-4851-9c6f-4a11d1223989\" (UID: \"586458cb-fb1d-4851-9c6f-4a11d1223989\") "
Sep 11 05:07:55.209819 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.209581    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/586458cb-fb1d-4851-9c6f-4a11d1223989-maas-api-tls\") pod \"586458cb-fb1d-4851-9c6f-4a11d1223989\" (UID: \"586458cb-fb1d-4851-9c6f-4a11d1223989\") "
Sep 11 05:07:55.210247 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.209855    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/866ad27d-598c-4156-b02c-0f842e988593-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "866ad27d-598c-4156-b02c-0f842e988593" (UID: "866ad27d-598c-4156-b02c-0f842e988593"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:07:55.210247 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.209902    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f568534-d42f-4808-86a0-f23f22861913-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7f568534-d42f-4808-86a0-f23f22861913" (UID: "7f568534-d42f-4808-86a0-f23f22861913"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:07:55.210247 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.210147    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/586458cb-fb1d-4851-9c6f-4a11d1223989-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "586458cb-fb1d-4851-9c6f-4a11d1223989" (UID: "586458cb-fb1d-4851-9c6f-4a11d1223989"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:07:55.210546 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.210523    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f568534-d42f-4808-86a0-f23f22861913-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:07:55.210625 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.210551    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/586458cb-fb1d-4851-9c6f-4a11d1223989-openshift-service-ca\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:07:55.210625 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.210565    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/866ad27d-598c-4156-b02c-0f842e988593-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:07:55.212000 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.211966    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/586458cb-fb1d-4851-9c6f-4a11d1223989-kube-api-access-5xx65" (OuterVolumeSpecName: "kube-api-access-5xx65") pod "586458cb-fb1d-4851-9c6f-4a11d1223989" (UID: "586458cb-fb1d-4851-9c6f-4a11d1223989"). InnerVolumeSpecName "kube-api-access-5xx65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:07:55.212177 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.211997    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f568534-d42f-4808-86a0-f23f22861913-kube-api-access-swl25" (OuterVolumeSpecName: "kube-api-access-swl25") pod "7f568534-d42f-4808-86a0-f23f22861913" (UID: "7f568534-d42f-4808-86a0-f23f22861913"). InnerVolumeSpecName "kube-api-access-swl25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:07:55.212276 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.212254    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/586458cb-fb1d-4851-9c6f-4a11d1223989-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "586458cb-fb1d-4851-9c6f-4a11d1223989" (UID: "586458cb-fb1d-4851-9c6f-4a11d1223989"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:07:55.212375 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.212359    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/866ad27d-598c-4156-b02c-0f842e988593-kube-api-access-x2wmz" (OuterVolumeSpecName: "kube-api-access-x2wmz") pod "866ad27d-598c-4156-b02c-0f842e988593" (UID: "866ad27d-598c-4156-b02c-0f842e988593"). InnerVolumeSpecName "kube-api-access-x2wmz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:07:55.212457 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.212444    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/586458cb-fb1d-4851-9c6f-4a11d1223989-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "586458cb-fb1d-4851-9c6f-4a11d1223989" (UID: "586458cb-fb1d-4851-9c6f-4a11d1223989"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:07:55.312101 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.312052    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xx65\" (UniqueName: \"kubernetes.io/projected/586458cb-fb1d-4851-9c6f-4a11d1223989-kube-api-access-5xx65\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:07:55.312101 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.312101    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2wmz\" (UniqueName: \"kubernetes.io/projected/866ad27d-598c-4156-b02c-0f842e988593-kube-api-access-x2wmz\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:07:55.312351 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.312115    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swl25\" (UniqueName: \"kubernetes.io/projected/7f568534-d42f-4808-86a0-f23f22861913-kube-api-access-swl25\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:07:55.312351 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.312128    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/586458cb-fb1d-4851-9c6f-4a11d1223989-metrics-certs\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:07:55.312351 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.312140    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/586458cb-fb1d-4851-9c6f-4a11d1223989-maas-api-tls\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:07:55.370298 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.370264    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:07:55.667678 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.667579    2567 generic.go:356] "Generic (PLEG): container finished" podID="866ad27d-598c-4156-b02c-0f842e988593" containerID="f59ee375608af4fb356ee0bc01dfcae04bc67b6274892b37754bd21822cd512d" exitCode=0
Sep 11 05:07:55.667678 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.667630    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw"
Sep 11 05:07:55.667916 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.667672    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw" event={"ID":"866ad27d-598c-4156-b02c-0f842e988593","Type":"ContainerDied","Data":"f59ee375608af4fb356ee0bc01dfcae04bc67b6274892b37754bd21822cd512d"}
Sep 11 05:07:55.667916 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.667732    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw" event={"ID":"866ad27d-598c-4156-b02c-0f842e988593","Type":"ContainerDied","Data":"8481fff8c1103ded35097e306b124e7dc8f79123c7186353a588501e2397af46"}
Sep 11 05:07:55.667916 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.667765    2567 scope.go:117] "RemoveContainer" containerID="f59ee375608af4fb356ee0bc01dfcae04bc67b6274892b37754bd21822cd512d"
Sep 11 05:07:55.669267 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.669239    2567 generic.go:356] "Generic (PLEG): container finished" podID="586458cb-fb1d-4851-9c6f-4a11d1223989" containerID="41987aef6b3c06ef02fc73db0fee7003e2e325c0ed5fba2075011eddbca6e261" exitCode=0
Sep 11 05:07:55.669384 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.669279    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"
Sep 11 05:07:55.669384 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.669338    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d" event={"ID":"586458cb-fb1d-4851-9c6f-4a11d1223989","Type":"ContainerDied","Data":"41987aef6b3c06ef02fc73db0fee7003e2e325c0ed5fba2075011eddbca6e261"}
Sep 11 05:07:55.669384 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.669377    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d" event={"ID":"586458cb-fb1d-4851-9c6f-4a11d1223989","Type":"ContainerDied","Data":"9e86ebbbb00e3149ffb3badc40b976906d20d743af96c047d5f5201b6d8b6769"}
Sep 11 05:07:55.670971 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.670932    2567 generic.go:356] "Generic (PLEG): container finished" podID="7f568534-d42f-4808-86a0-f23f22861913" containerID="ee9160400461c39fab2430b1f690353753b909a128d3da6526efa6a5546c3a7f" exitCode=0
Sep 11 05:07:55.671100 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.670974    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p"
Sep 11 05:07:55.671100 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.670991    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p" event={"ID":"7f568534-d42f-4808-86a0-f23f22861913","Type":"ContainerDied","Data":"ee9160400461c39fab2430b1f690353753b909a128d3da6526efa6a5546c3a7f"}
Sep 11 05:07:55.671100 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.671031    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p" event={"ID":"7f568534-d42f-4808-86a0-f23f22861913","Type":"ContainerDied","Data":"ab06f217bd02f267e8e1d47f519307b01ccad3d1da5d82777a8fce2325ec5065"}
Sep 11 05:07:55.681308 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.681284    2567 scope.go:117] "RemoveContainer" containerID="f59ee375608af4fb356ee0bc01dfcae04bc67b6274892b37754bd21822cd512d"
Sep 11 05:07:55.681611 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:07:55.681584    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f59ee375608af4fb356ee0bc01dfcae04bc67b6274892b37754bd21822cd512d\": container with ID starting with f59ee375608af4fb356ee0bc01dfcae04bc67b6274892b37754bd21822cd512d not found: ID does not exist" containerID="f59ee375608af4fb356ee0bc01dfcae04bc67b6274892b37754bd21822cd512d"
Sep 11 05:07:55.681611 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.681622    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f59ee375608af4fb356ee0bc01dfcae04bc67b6274892b37754bd21822cd512d"} err="failed to get container status \"f59ee375608af4fb356ee0bc01dfcae04bc67b6274892b37754bd21822cd512d\": rpc error: code = NotFound desc = could not find container \"f59ee375608af4fb356ee0bc01dfcae04bc67b6274892b37754bd21822cd512d\": container with ID starting with f59ee375608af4fb356ee0bc01dfcae04bc67b6274892b37754bd21822cd512d not found: ID does not exist"
Sep 11 05:07:55.681611 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.681646    2567 scope.go:117] "RemoveContainer" containerID="41987aef6b3c06ef02fc73db0fee7003e2e325c0ed5fba2075011eddbca6e261"
Sep 11 05:07:55.691004 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.690969    2567 scope.go:117] "RemoveContainer" containerID="41987aef6b3c06ef02fc73db0fee7003e2e325c0ed5fba2075011eddbca6e261"
Sep 11 05:07:55.691514 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:07:55.691421    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41987aef6b3c06ef02fc73db0fee7003e2e325c0ed5fba2075011eddbca6e261\": container with ID starting with 41987aef6b3c06ef02fc73db0fee7003e2e325c0ed5fba2075011eddbca6e261 not found: ID does not exist" containerID="41987aef6b3c06ef02fc73db0fee7003e2e325c0ed5fba2075011eddbca6e261"
Sep 11 05:07:55.691514 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.691460    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41987aef6b3c06ef02fc73db0fee7003e2e325c0ed5fba2075011eddbca6e261"} err="failed to get container status \"41987aef6b3c06ef02fc73db0fee7003e2e325c0ed5fba2075011eddbca6e261\": rpc error: code = NotFound desc = could not find container \"41987aef6b3c06ef02fc73db0fee7003e2e325c0ed5fba2075011eddbca6e261\": container with ID starting with 41987aef6b3c06ef02fc73db0fee7003e2e325c0ed5fba2075011eddbca6e261 not found: ID does not exist"
Sep 11 05:07:55.691514 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.691479    2567 scope.go:117] "RemoveContainer" containerID="ee9160400461c39fab2430b1f690353753b909a128d3da6526efa6a5546c3a7f"
Sep 11 05:07:55.693487 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.693466    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw"]
Sep 11 05:07:55.699803 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.699784    2567 scope.go:117] "RemoveContainer" containerID="ee9160400461c39fab2430b1f690353753b909a128d3da6526efa6a5546c3a7f"
Sep 11 05:07:55.699998 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.699983    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-c4902f-8564c4797f-t5xnw"]
Sep 11 05:07:55.700111 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:07:55.700092    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee9160400461c39fab2430b1f690353753b909a128d3da6526efa6a5546c3a7f\": container with ID starting with ee9160400461c39fab2430b1f690353753b909a128d3da6526efa6a5546c3a7f not found: ID does not exist" containerID="ee9160400461c39fab2430b1f690353753b909a128d3da6526efa6a5546c3a7f"
Sep 11 05:07:55.700162 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.700115    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee9160400461c39fab2430b1f690353753b909a128d3da6526efa6a5546c3a7f"} err="failed to get container status \"ee9160400461c39fab2430b1f690353753b909a128d3da6526efa6a5546c3a7f\": rpc error: code = NotFound desc = could not find container \"ee9160400461c39fab2430b1f690353753b909a128d3da6526efa6a5546c3a7f\": container with ID starting with ee9160400461c39fab2430b1f690353753b909a128d3da6526efa6a5546c3a7f not found: ID does not exist"
Sep 11 05:07:55.711089 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.711053    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"]
Sep 11 05:07:55.714602 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.714582    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-c4902f-56754cb5f7-tpk4d"]
Sep 11 05:07:55.724139 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.724117    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p"]
Sep 11 05:07:55.726392 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:55.726374    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-c4902f-5c6f7c79db-vk74p"]
Sep 11 05:07:56.019307 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:56.019279    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="586458cb-fb1d-4851-9c6f-4a11d1223989" path="/var/lib/kubelet/pods/586458cb-fb1d-4851-9c6f-4a11d1223989/volumes"
Sep 11 05:07:56.019621 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:56.019608    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f568534-d42f-4808-86a0-f23f22861913" path="/var/lib/kubelet/pods/7f568534-d42f-4808-86a0-f23f22861913/volumes"
Sep 11 05:07:56.019927 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:07:56.019916    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="866ad27d-598c-4156-b02c-0f842e988593" path="/var/lib/kubelet/pods/866ad27d-598c-4156-b02c-0f842e988593/volumes"
Sep 11 05:08:02.175538 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:02.175502    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:08:02.276746 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:02.276715    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:08:11.993182 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:11.993095    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"]
Sep 11 05:08:11.993640 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:11.993375    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk" podUID="7dfc98bf-1183-4a35-93e7-d57204bed9d4" containerName="maas-api" containerID="cri-o://25a043661e4b9062f52aeac2e548c0cfdbfb9893efde89b6c88b35b6c5152a83" gracePeriod=30
Sep 11 05:08:12.025009 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.024982    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:08:12.048878 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.048841    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp"]
Sep 11 05:08:12.049125 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.049097    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp" podUID="12178127-9faf-4eda-b496-a8e4766592da" containerName="payload-processing" containerID="cri-o://a16b3a4656abb183ebac85d78f193655bde281f3a106d4e8f9bbc077f13d1690" gracePeriod=30
Sep 11 05:08:12.084182 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.084148    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:08:12.161186 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.161148    2567 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk" podUID="7dfc98bf-1183-4a35-93e7-d57204bed9d4" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.53:8443/health\": dial tcp 10.132.0.53:8443: connect: connection refused"
Sep 11 05:08:12.185348 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.185313    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:08:12.240563 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.240512    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:08:12.293253 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.293223    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"
Sep 11 05:08:12.326776 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.326748    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp"
Sep 11 05:08:12.366088 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.366042    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4v9l8\" (UniqueName: \"kubernetes.io/projected/12178127-9faf-4eda-b496-a8e4766592da-kube-api-access-4v9l8\") pod \"12178127-9faf-4eda-b496-a8e4766592da\" (UID: \"12178127-9faf-4eda-b496-a8e4766592da\") "
Sep 11 05:08:12.366257 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.366135    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dfc98bf-1183-4a35-93e7-d57204bed9d4-openshift-service-ca\") pod \"7dfc98bf-1183-4a35-93e7-d57204bed9d4\" (UID: \"7dfc98bf-1183-4a35-93e7-d57204bed9d4\") "
Sep 11 05:08:12.366257 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.366176    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7dfc98bf-1183-4a35-93e7-d57204bed9d4-maas-api-tls\") pod \"7dfc98bf-1183-4a35-93e7-d57204bed9d4\" (UID: \"7dfc98bf-1183-4a35-93e7-d57204bed9d4\") "
Sep 11 05:08:12.366416 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.366267    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12178127-9faf-4eda-b496-a8e4766592da-plugins-config-volume\") pod \"12178127-9faf-4eda-b496-a8e4766592da\" (UID: \"12178127-9faf-4eda-b496-a8e4766592da\") "
Sep 11 05:08:12.366416 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.366311    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpkkp\" (UniqueName: \"kubernetes.io/projected/7dfc98bf-1183-4a35-93e7-d57204bed9d4-kube-api-access-rpkkp\") pod \"7dfc98bf-1183-4a35-93e7-d57204bed9d4\" (UID: \"7dfc98bf-1183-4a35-93e7-d57204bed9d4\") "
Sep 11 05:08:12.366416 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.366344    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7dfc98bf-1183-4a35-93e7-d57204bed9d4-metrics-certs\") pod \"7dfc98bf-1183-4a35-93e7-d57204bed9d4\" (UID: \"7dfc98bf-1183-4a35-93e7-d57204bed9d4\") "
Sep 11 05:08:12.366733 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.366675    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12178127-9faf-4eda-b496-a8e4766592da-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "12178127-9faf-4eda-b496-a8e4766592da" (UID: "12178127-9faf-4eda-b496-a8e4766592da"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:08:12.366877 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.366805    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dfc98bf-1183-4a35-93e7-d57204bed9d4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7dfc98bf-1183-4a35-93e7-d57204bed9d4" (UID: "7dfc98bf-1183-4a35-93e7-d57204bed9d4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:08:12.368583 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.368549    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7dfc98bf-1183-4a35-93e7-d57204bed9d4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7dfc98bf-1183-4a35-93e7-d57204bed9d4" (UID: "7dfc98bf-1183-4a35-93e7-d57204bed9d4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:08:12.368583 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.368560    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12178127-9faf-4eda-b496-a8e4766592da-kube-api-access-4v9l8" (OuterVolumeSpecName: "kube-api-access-4v9l8") pod "12178127-9faf-4eda-b496-a8e4766592da" (UID: "12178127-9faf-4eda-b496-a8e4766592da"). InnerVolumeSpecName "kube-api-access-4v9l8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:08:12.368861 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.368843    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7dfc98bf-1183-4a35-93e7-d57204bed9d4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7dfc98bf-1183-4a35-93e7-d57204bed9d4" (UID: "7dfc98bf-1183-4a35-93e7-d57204bed9d4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:08:12.368984 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.368965    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dfc98bf-1183-4a35-93e7-d57204bed9d4-kube-api-access-rpkkp" (OuterVolumeSpecName: "kube-api-access-rpkkp") pod "7dfc98bf-1183-4a35-93e7-d57204bed9d4" (UID: "7dfc98bf-1183-4a35-93e7-d57204bed9d4"). InnerVolumeSpecName "kube-api-access-rpkkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:08:12.467900 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.467868    2567 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/12178127-9faf-4eda-b496-a8e4766592da-plugins-config-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:08:12.467900 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.467897    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpkkp\" (UniqueName: \"kubernetes.io/projected/7dfc98bf-1183-4a35-93e7-d57204bed9d4-kube-api-access-rpkkp\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:08:12.468120 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.467909    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7dfc98bf-1183-4a35-93e7-d57204bed9d4-metrics-certs\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:08:12.468120 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.467918    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4v9l8\" (UniqueName: \"kubernetes.io/projected/12178127-9faf-4eda-b496-a8e4766592da-kube-api-access-4v9l8\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:08:12.468120 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.467927    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7dfc98bf-1183-4a35-93e7-d57204bed9d4-openshift-service-ca\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:08:12.468120 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.467935    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7dfc98bf-1183-4a35-93e7-d57204bed9d4-maas-api-tls\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:08:12.570600 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.570514    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:08:12.761977 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.761945    2567 generic.go:356] "Generic (PLEG): container finished" podID="7dfc98bf-1183-4a35-93e7-d57204bed9d4" containerID="25a043661e4b9062f52aeac2e548c0cfdbfb9893efde89b6c88b35b6c5152a83" exitCode=0
Sep 11 05:08:12.762179 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.761981    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk" event={"ID":"7dfc98bf-1183-4a35-93e7-d57204bed9d4","Type":"ContainerDied","Data":"25a043661e4b9062f52aeac2e548c0cfdbfb9893efde89b6c88b35b6c5152a83"}
Sep 11 05:08:12.762179 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.762027    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"
Sep 11 05:08:12.762179 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.762043    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk" event={"ID":"7dfc98bf-1183-4a35-93e7-d57204bed9d4","Type":"ContainerDied","Data":"4c848fdb2f23a206463c83c1779a9e3f29f039816dab6e53c3550147a8f49214"}
Sep 11 05:08:12.762179 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.762080    2567 scope.go:117] "RemoveContainer" containerID="25a043661e4b9062f52aeac2e548c0cfdbfb9893efde89b6c88b35b6c5152a83"
Sep 11 05:08:12.763512 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.763464    2567 generic.go:356] "Generic (PLEG): container finished" podID="12178127-9faf-4eda-b496-a8e4766592da" containerID="a16b3a4656abb183ebac85d78f193655bde281f3a106d4e8f9bbc077f13d1690" exitCode=0
Sep 11 05:08:12.763512 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.763523    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp" event={"ID":"12178127-9faf-4eda-b496-a8e4766592da","Type":"ContainerDied","Data":"a16b3a4656abb183ebac85d78f193655bde281f3a106d4e8f9bbc077f13d1690"}
Sep 11 05:08:12.763806 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.763556    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp" event={"ID":"12178127-9faf-4eda-b496-a8e4766592da","Type":"ContainerDied","Data":"8993757841e3e0a3852dd3490a4158ea389d2b39ec412698f1664d86e18fe1e8"}
Sep 11 05:08:12.763806 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.763589    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp"
Sep 11 05:08:12.771190 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.771171    2567 scope.go:117] "RemoveContainer" containerID="25a043661e4b9062f52aeac2e548c0cfdbfb9893efde89b6c88b35b6c5152a83"
Sep 11 05:08:12.771515 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:08:12.771490    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25a043661e4b9062f52aeac2e548c0cfdbfb9893efde89b6c88b35b6c5152a83\": container with ID starting with 25a043661e4b9062f52aeac2e548c0cfdbfb9893efde89b6c88b35b6c5152a83 not found: ID does not exist" containerID="25a043661e4b9062f52aeac2e548c0cfdbfb9893efde89b6c88b35b6c5152a83"
Sep 11 05:08:12.771605 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.771540    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25a043661e4b9062f52aeac2e548c0cfdbfb9893efde89b6c88b35b6c5152a83"} err="failed to get container status \"25a043661e4b9062f52aeac2e548c0cfdbfb9893efde89b6c88b35b6c5152a83\": rpc error: code = NotFound desc = could not find container \"25a043661e4b9062f52aeac2e548c0cfdbfb9893efde89b6c88b35b6c5152a83\": container with ID starting with 25a043661e4b9062f52aeac2e548c0cfdbfb9893efde89b6c88b35b6c5152a83 not found: ID does not exist"
Sep 11 05:08:12.771605 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.771563    2567 scope.go:117] "RemoveContainer" containerID="a16b3a4656abb183ebac85d78f193655bde281f3a106d4e8f9bbc077f13d1690"
Sep 11 05:08:12.782037 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.782018    2567 scope.go:117] "RemoveContainer" containerID="a16b3a4656abb183ebac85d78f193655bde281f3a106d4e8f9bbc077f13d1690"
Sep 11 05:08:12.782369 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:08:12.782346    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a16b3a4656abb183ebac85d78f193655bde281f3a106d4e8f9bbc077f13d1690\": container with ID starting with a16b3a4656abb183ebac85d78f193655bde281f3a106d4e8f9bbc077f13d1690 not found: ID does not exist" containerID="a16b3a4656abb183ebac85d78f193655bde281f3a106d4e8f9bbc077f13d1690"
Sep 11 05:08:12.782424 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.782380    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a16b3a4656abb183ebac85d78f193655bde281f3a106d4e8f9bbc077f13d1690"} err="failed to get container status \"a16b3a4656abb183ebac85d78f193655bde281f3a106d4e8f9bbc077f13d1690\": rpc error: code = NotFound desc = could not find container \"a16b3a4656abb183ebac85d78f193655bde281f3a106d4e8f9bbc077f13d1690\": container with ID starting with a16b3a4656abb183ebac85d78f193655bde281f3a106d4e8f9bbc077f13d1690 not found: ID does not exist"
Sep 11 05:08:12.789343 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.789323    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"]
Sep 11 05:08:12.795103 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.795084    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-c1858a-6f9c5d6455-qh8zk"]
Sep 11 05:08:12.804142 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.804123    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp"]
Sep 11 05:08:12.807713 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:12.807694    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-c1858a-579bcf7dbc-svzbp"]
Sep 11 05:08:14.017058 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:14.017025    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12178127-9faf-4eda-b496-a8e4766592da" path="/var/lib/kubelet/pods/12178127-9faf-4eda-b496-a8e4766592da/volumes"
Sep 11 05:08:14.017455 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:14.017441    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7dfc98bf-1183-4a35-93e7-d57204bed9d4" path="/var/lib/kubelet/pods/7dfc98bf-1183-4a35-93e7-d57204bed9d4/volumes"
Sep 11 05:08:18.460417 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:18.460380    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:08:18.561465 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:18.561421    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:08:18.723411 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:18.723328    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:08:19.262720 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:19.262685    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:08:29.414684 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.414651    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"]
Sep 11 05:08:29.415612 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.415559    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj" podUID="e71f04c8-1bc1-463e-a6ab-111adbca0b2d" containerName="maas-api" containerID="cri-o://24019fdab57084b256da74c9d0f479cb1b3df5708b0e856aee0f2df23efcec01" gracePeriod=30
Sep 11 05:08:29.438768 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.438735    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:08:29.503305 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.503276    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:08:29.562880 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.562842    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:08:29.624508 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.624479    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:08:29.663042 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.663017    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"
Sep 11 05:08:29.826577 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.826542    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-openshift-service-ca\") pod \"e71f04c8-1bc1-463e-a6ab-111adbca0b2d\" (UID: \"e71f04c8-1bc1-463e-a6ab-111adbca0b2d\") "
Sep 11 05:08:29.826745 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.826603    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wqm8c\" (UniqueName: \"kubernetes.io/projected/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-kube-api-access-wqm8c\") pod \"e71f04c8-1bc1-463e-a6ab-111adbca0b2d\" (UID: \"e71f04c8-1bc1-463e-a6ab-111adbca0b2d\") "
Sep 11 05:08:29.826745 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.826643    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-maas-api-tls\") pod \"e71f04c8-1bc1-463e-a6ab-111adbca0b2d\" (UID: \"e71f04c8-1bc1-463e-a6ab-111adbca0b2d\") "
Sep 11 05:08:29.826745 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.826689    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-metrics-certs\") pod \"e71f04c8-1bc1-463e-a6ab-111adbca0b2d\" (UID: \"e71f04c8-1bc1-463e-a6ab-111adbca0b2d\") "
Sep 11 05:08:29.826951 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.826922    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e71f04c8-1bc1-463e-a6ab-111adbca0b2d" (UID: "e71f04c8-1bc1-463e-a6ab-111adbca0b2d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:08:29.828990 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.828959    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e71f04c8-1bc1-463e-a6ab-111adbca0b2d" (UID: "e71f04c8-1bc1-463e-a6ab-111adbca0b2d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:08:29.828990 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.828979    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-kube-api-access-wqm8c" (OuterVolumeSpecName: "kube-api-access-wqm8c") pod "e71f04c8-1bc1-463e-a6ab-111adbca0b2d" (UID: "e71f04c8-1bc1-463e-a6ab-111adbca0b2d"). InnerVolumeSpecName "kube-api-access-wqm8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:08:29.829211 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.829004    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e71f04c8-1bc1-463e-a6ab-111adbca0b2d" (UID: "e71f04c8-1bc1-463e-a6ab-111adbca0b2d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:08:29.848976 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.848953    2567 generic.go:356] "Generic (PLEG): container finished" podID="e71f04c8-1bc1-463e-a6ab-111adbca0b2d" containerID="24019fdab57084b256da74c9d0f479cb1b3df5708b0e856aee0f2df23efcec01" exitCode=0
Sep 11 05:08:29.849112 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.849015    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"
Sep 11 05:08:29.849112 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.849034    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj" event={"ID":"e71f04c8-1bc1-463e-a6ab-111adbca0b2d","Type":"ContainerDied","Data":"24019fdab57084b256da74c9d0f479cb1b3df5708b0e856aee0f2df23efcec01"}
Sep 11 05:08:29.849112 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.849108    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj" event={"ID":"e71f04c8-1bc1-463e-a6ab-111adbca0b2d","Type":"ContainerDied","Data":"42a3d2aa40770f164b66c73a6e26b5162c0c0a969ff3a8f5161cc9ac7cfbf836"}
Sep 11 05:08:29.849237 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.849131    2567 scope.go:117] "RemoveContainer" containerID="24019fdab57084b256da74c9d0f479cb1b3df5708b0e856aee0f2df23efcec01"
Sep 11 05:08:29.858672 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.858653    2567 scope.go:117] "RemoveContainer" containerID="24019fdab57084b256da74c9d0f479cb1b3df5708b0e856aee0f2df23efcec01"
Sep 11 05:08:29.858991 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:08:29.858974    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24019fdab57084b256da74c9d0f479cb1b3df5708b0e856aee0f2df23efcec01\": container with ID starting with 24019fdab57084b256da74c9d0f479cb1b3df5708b0e856aee0f2df23efcec01 not found: ID does not exist" containerID="24019fdab57084b256da74c9d0f479cb1b3df5708b0e856aee0f2df23efcec01"
Sep 11 05:08:29.859044 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.858999    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24019fdab57084b256da74c9d0f479cb1b3df5708b0e856aee0f2df23efcec01"} err="failed to get container status \"24019fdab57084b256da74c9d0f479cb1b3df5708b0e856aee0f2df23efcec01\": rpc error: code = NotFound desc = could not find container \"24019fdab57084b256da74c9d0f479cb1b3df5708b0e856aee0f2df23efcec01\": container with ID starting with 24019fdab57084b256da74c9d0f479cb1b3df5708b0e856aee0f2df23efcec01 not found: ID does not exist"
Sep 11 05:08:29.869965 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.869942    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"]
Sep 11 05:08:29.873365 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.873345    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-16b799-7f65975f7d-qtqxj"]
Sep 11 05:08:29.928264 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.928238    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-openshift-service-ca\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:08:29.928264 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.928260    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqm8c\" (UniqueName: \"kubernetes.io/projected/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-kube-api-access-wqm8c\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:08:29.928403 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.928273    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-maas-api-tls\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:08:29.928403 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.928286    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e71f04c8-1bc1-463e-a6ab-111adbca0b2d-metrics-certs\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:08:29.972720 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:29.972691    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:08:30.018329 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:30.018295    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e71f04c8-1bc1-463e-a6ab-111adbca0b2d" path="/var/lib/kubelet/pods/e71f04c8-1bc1-463e-a6ab-111adbca0b2d/volumes"
Sep 11 05:08:35.844632 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:35.844584    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:08:35.899258 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:35.899223    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:08:36.085406 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:08:36.085368    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:09:10.646874 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:09:10.646839    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:09:10.694348 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:09:10.694311    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:09:12.247456 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:09:12.247419    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:09:59.204966 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:09:59.204924    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:09:59.442631 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:09:59.442586    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:10:01.102440 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:10:01.102404    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:10:18.353597 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:10:18.353559    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:11:23.938216 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:11:23.938136    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7"]
Sep 11 05:11:23.938727 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:11:23.938343    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7" podUID="fcc8e92d-9145-4558-84d0-023f8428b478" containerName="manager" containerID="cri-o://824e5df02a03ea05b47f045a0e7ec58b1a2144a760d0903f54ec7b16b5bbc793" gracePeriod=10
Sep 11 05:11:24.192414 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:11:24.192357    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7"
Sep 11 05:11:24.231193 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:11:24.231167    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2774\" (UniqueName: \"kubernetes.io/projected/fcc8e92d-9145-4558-84d0-023f8428b478-kube-api-access-h2774\") pod \"fcc8e92d-9145-4558-84d0-023f8428b478\" (UID: \"fcc8e92d-9145-4558-84d0-023f8428b478\") "
Sep 11 05:11:24.231327 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:11:24.231274    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcc8e92d-9145-4558-84d0-023f8428b478-extensions-socket-volume\") pod \"fcc8e92d-9145-4558-84d0-023f8428b478\" (UID: \"fcc8e92d-9145-4558-84d0-023f8428b478\") "
Sep 11 05:11:24.231655 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:11:24.231630    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcc8e92d-9145-4558-84d0-023f8428b478-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fcc8e92d-9145-4558-84d0-023f8428b478" (UID: "fcc8e92d-9145-4558-84d0-023f8428b478"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 05:11:24.233553 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:11:24.233526    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcc8e92d-9145-4558-84d0-023f8428b478-kube-api-access-h2774" (OuterVolumeSpecName: "kube-api-access-h2774") pod "fcc8e92d-9145-4558-84d0-023f8428b478" (UID: "fcc8e92d-9145-4558-84d0-023f8428b478"). InnerVolumeSpecName "kube-api-access-h2774". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:11:24.332937 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:11:24.332906    2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcc8e92d-9145-4558-84d0-023f8428b478-extensions-socket-volume\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:11:24.332937 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:11:24.332932    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2774\" (UniqueName: \"kubernetes.io/projected/fcc8e92d-9145-4558-84d0-023f8428b478-kube-api-access-h2774\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:11:24.678783 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:11:24.678743    2567 generic.go:356] "Generic (PLEG): container finished" podID="fcc8e92d-9145-4558-84d0-023f8428b478" containerID="824e5df02a03ea05b47f045a0e7ec58b1a2144a760d0903f54ec7b16b5bbc793" exitCode=0
Sep 11 05:11:24.678961 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:11:24.678788    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7"
Sep 11 05:11:24.678961 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:11:24.678799    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7" event={"ID":"fcc8e92d-9145-4558-84d0-023f8428b478","Type":"ContainerDied","Data":"824e5df02a03ea05b47f045a0e7ec58b1a2144a760d0903f54ec7b16b5bbc793"}
Sep 11 05:11:24.678961 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:11:24.678875    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7" event={"ID":"fcc8e92d-9145-4558-84d0-023f8428b478","Type":"ContainerDied","Data":"b451f0e7eb51c580d6868633f893fb6552aa8840fae610748c883525cbadf04c"}
Sep 11 05:11:24.678961 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:11:24.678898    2567 scope.go:117] "RemoveContainer" containerID="824e5df02a03ea05b47f045a0e7ec58b1a2144a760d0903f54ec7b16b5bbc793"
Sep 11 05:11:24.688838 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:11:24.688818    2567 scope.go:117] "RemoveContainer" containerID="824e5df02a03ea05b47f045a0e7ec58b1a2144a760d0903f54ec7b16b5bbc793"
Sep 11 05:11:24.689153 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:11:24.689134    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"824e5df02a03ea05b47f045a0e7ec58b1a2144a760d0903f54ec7b16b5bbc793\": container with ID starting with 824e5df02a03ea05b47f045a0e7ec58b1a2144a760d0903f54ec7b16b5bbc793 not found: ID does not exist" containerID="824e5df02a03ea05b47f045a0e7ec58b1a2144a760d0903f54ec7b16b5bbc793"
Sep 11 05:11:24.689218 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:11:24.689164    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"824e5df02a03ea05b47f045a0e7ec58b1a2144a760d0903f54ec7b16b5bbc793"} err="failed to get container status \"824e5df02a03ea05b47f045a0e7ec58b1a2144a760d0903f54ec7b16b5bbc793\": rpc error: code = NotFound desc = could not find container \"824e5df02a03ea05b47f045a0e7ec58b1a2144a760d0903f54ec7b16b5bbc793\": container with ID starting with 824e5df02a03ea05b47f045a0e7ec58b1a2144a760d0903f54ec7b16b5bbc793 not found: ID does not exist"
Sep 11 05:11:24.700347 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:11:24.700327    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7"]
Sep 11 05:11:24.703934 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:11:24.703915    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lfdw7"]
Sep 11 05:11:26.017146 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:11:26.017118    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcc8e92d-9145-4558-84d0-023f8428b478" path="/var/lib/kubelet/pods/fcc8e92d-9145-4558-84d0-023f8428b478/volumes"
Sep 11 05:12:24.640841 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.640807    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz4fd"]
Sep 11 05:12:24.641326 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641239    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fb40efe-b37b-49c1-b7cb-6448b64d0ea9" containerName="maas-api"
Sep 11 05:12:24.641326 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641253    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fb40efe-b37b-49c1-b7cb-6448b64d0ea9" containerName="maas-api"
Sep 11 05:12:24.641326 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641263    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f568534-d42f-4808-86a0-f23f22861913" containerName="payload-pre-processing"
Sep 11 05:12:24.641326 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641269    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f568534-d42f-4808-86a0-f23f22861913" containerName="payload-pre-processing"
Sep 11 05:12:24.641326 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641281    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="866ad27d-598c-4156-b02c-0f842e988593" containerName="payload-processing"
Sep 11 05:12:24.641326 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641287    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="866ad27d-598c-4156-b02c-0f842e988593" containerName="payload-processing"
Sep 11 05:12:24.641326 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641294    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79875611-ced2-408c-89dc-461f0e2fa730" containerName="payload-pre-processing"
Sep 11 05:12:24.641326 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641299    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="79875611-ced2-408c-89dc-461f0e2fa730" containerName="payload-pre-processing"
Sep 11 05:12:24.641326 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641307    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcc8e92d-9145-4558-84d0-023f8428b478" containerName="manager"
Sep 11 05:12:24.641326 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641312    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcc8e92d-9145-4558-84d0-023f8428b478" containerName="manager"
Sep 11 05:12:24.641326 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641317    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dfc98bf-1183-4a35-93e7-d57204bed9d4" containerName="maas-api"
Sep 11 05:12:24.641326 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641322    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dfc98bf-1183-4a35-93e7-d57204bed9d4" containerName="maas-api"
Sep 11 05:12:24.641326 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641330    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb692b75-60df-413a-92c9-2aed7ba93b20" containerName="payload-processing"
Sep 11 05:12:24.641723 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641336    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb692b75-60df-413a-92c9-2aed7ba93b20" containerName="payload-processing"
Sep 11 05:12:24.641723 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641343    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="586458cb-fb1d-4851-9c6f-4a11d1223989" containerName="maas-api"
Sep 11 05:12:24.641723 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641348    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="586458cb-fb1d-4851-9c6f-4a11d1223989" containerName="maas-api"
Sep 11 05:12:24.641723 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641357    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12178127-9faf-4eda-b496-a8e4766592da" containerName="payload-processing"
Sep 11 05:12:24.641723 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641362    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="12178127-9faf-4eda-b496-a8e4766592da" containerName="payload-processing"
Sep 11 05:12:24.641723 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641369    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e71f04c8-1bc1-463e-a6ab-111adbca0b2d" containerName="maas-api"
Sep 11 05:12:24.641723 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641374    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e71f04c8-1bc1-463e-a6ab-111adbca0b2d" containerName="maas-api"
Sep 11 05:12:24.641723 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641429    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcc8e92d-9145-4558-84d0-023f8428b478" containerName="manager"
Sep 11 05:12:24.641723 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641436    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fb40efe-b37b-49c1-b7cb-6448b64d0ea9" containerName="maas-api"
Sep 11 05:12:24.641723 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641441    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dfc98bf-1183-4a35-93e7-d57204bed9d4" containerName="maas-api"
Sep 11 05:12:24.641723 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641447    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="79875611-ced2-408c-89dc-461f0e2fa730" containerName="payload-pre-processing"
Sep 11 05:12:24.641723 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641452    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="586458cb-fb1d-4851-9c6f-4a11d1223989" containerName="maas-api"
Sep 11 05:12:24.641723 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641459    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="12178127-9faf-4eda-b496-a8e4766592da" containerName="payload-processing"
Sep 11 05:12:24.641723 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641463    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e71f04c8-1bc1-463e-a6ab-111adbca0b2d" containerName="maas-api"
Sep 11 05:12:24.641723 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641468    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="866ad27d-598c-4156-b02c-0f842e988593" containerName="payload-processing"
Sep 11 05:12:24.641723 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641473    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb692b75-60df-413a-92c9-2aed7ba93b20" containerName="payload-processing"
Sep 11 05:12:24.641723 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.641478    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f568534-d42f-4808-86a0-f23f22861913" containerName="payload-pre-processing"
Sep 11 05:12:24.644565 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.644549    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz4fd"
Sep 11 05:12:24.648089 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.648053    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 05:12:24.648241 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.648220    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 05:12:24.648355 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.648269    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-wvcgs\""
Sep 11 05:12:24.658767 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.658746    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz4fd"]
Sep 11 05:12:24.768565 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.768527    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl4kx\" (UniqueName: \"kubernetes.io/projected/a9537d97-aca4-4dff-bfa7-940ee3406e69-kube-api-access-rl4kx\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hz4fd\" (UID: \"a9537d97-aca4-4dff-bfa7-940ee3406e69\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz4fd"
Sep 11 05:12:24.768729 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.768601    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9537d97-aca4-4dff-bfa7-940ee3406e69-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hz4fd\" (UID: \"a9537d97-aca4-4dff-bfa7-940ee3406e69\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz4fd"
Sep 11 05:12:24.869797 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.869754    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl4kx\" (UniqueName: \"kubernetes.io/projected/a9537d97-aca4-4dff-bfa7-940ee3406e69-kube-api-access-rl4kx\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hz4fd\" (UID: \"a9537d97-aca4-4dff-bfa7-940ee3406e69\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz4fd"
Sep 11 05:12:24.869924 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.869835    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9537d97-aca4-4dff-bfa7-940ee3406e69-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hz4fd\" (UID: \"a9537d97-aca4-4dff-bfa7-940ee3406e69\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz4fd"
Sep 11 05:12:24.870255 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.870234    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a9537d97-aca4-4dff-bfa7-940ee3406e69-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hz4fd\" (UID: \"a9537d97-aca4-4dff-bfa7-940ee3406e69\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz4fd"
Sep 11 05:12:24.884345 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.884322    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl4kx\" (UniqueName: \"kubernetes.io/projected/a9537d97-aca4-4dff-bfa7-940ee3406e69-kube-api-access-rl4kx\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hz4fd\" (UID: \"a9537d97-aca4-4dff-bfa7-940ee3406e69\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz4fd"
Sep 11 05:12:24.954644 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:24.954580    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz4fd"
Sep 11 05:12:25.237504 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:25.237480    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz4fd"]
Sep 11 05:12:25.239259 ip-10-0-128-18 kubenswrapper[2567]: W0911 05:12:25.239230    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9537d97_aca4_4dff_bfa7_940ee3406e69.slice/crio-83978e20887c18fac54589529cfb03de72ecdf81c6a971918ec37433c990b05e WatchSource:0}: Error finding container 83978e20887c18fac54589529cfb03de72ecdf81c6a971918ec37433c990b05e: Status 404 returned error can't find the container with id 83978e20887c18fac54589529cfb03de72ecdf81c6a971918ec37433c990b05e
Sep 11 05:12:25.975907 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:25.975872    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz4fd" event={"ID":"a9537d97-aca4-4dff-bfa7-940ee3406e69","Type":"ContainerStarted","Data":"6aa9fc0a4dbf4b3af46493172dbe7c336cc2b4eade6f9120cb35e20eb89878a9"}
Sep 11 05:12:25.976316 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:25.975924    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz4fd" event={"ID":"a9537d97-aca4-4dff-bfa7-940ee3406e69","Type":"ContainerStarted","Data":"83978e20887c18fac54589529cfb03de72ecdf81c6a971918ec37433c990b05e"}
Sep 11 05:12:25.976316 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:25.975963    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz4fd"
Sep 11 05:12:25.997292 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:25.997246    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz4fd" podStartSLOduration=1.997231059 podStartE2EDuration="1.997231059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:12:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:12:25.995894204 +0000 UTC m=+1960.467358093" watchObservedRunningTime="2026-09-11 05:12:25.997231059 +0000 UTC m=+1960.468694934"
Sep 11 05:12:36.984372 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:36.984288    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hz4fd"
Sep 11 05:12:42.841396 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:42.841361    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:12:44.075185 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:12:44.075150    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:13:04.689782 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:13:04.689742    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:13:18.482447 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:13:18.482410    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:13:44.275607 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:13:44.275570    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:13:47.417277 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:13:47.417245    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:13:48.596173 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:13:48.596141    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:14:05.871514 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:05.871476    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"]
Sep 11 05:14:05.875314 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:05.875296    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"
Sep 11 05:14:05.878081 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:05.878044    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 11 05:14:05.879111 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:05.879092    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-74gzc\""
Sep 11 05:14:05.879188 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:05.879133    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 05:14:05.889766 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:05.889743    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"]
Sep 11 05:14:05.984226 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:05.984192    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e49559d-c2a2-4ef5-b806-66ad21212125-maas-api-tls\") pod \"maas-api-646f6bcd4d-7vvzg\" (UID: \"7e49559d-c2a2-4ef5-b806-66ad21212125\") " pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"
Sep 11 05:14:05.984386 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:05.984233    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e49559d-c2a2-4ef5-b806-66ad21212125-openshift-service-ca\") pod \"maas-api-646f6bcd4d-7vvzg\" (UID: \"7e49559d-c2a2-4ef5-b806-66ad21212125\") " pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"
Sep 11 05:14:05.984386 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:05.984271    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e49559d-c2a2-4ef5-b806-66ad21212125-metrics-certs\") pod \"maas-api-646f6bcd4d-7vvzg\" (UID: \"7e49559d-c2a2-4ef5-b806-66ad21212125\") " pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"
Sep 11 05:14:05.984386 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:05.984295    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bv5z\" (UniqueName: \"kubernetes.io/projected/7e49559d-c2a2-4ef5-b806-66ad21212125-kube-api-access-7bv5z\") pod \"maas-api-646f6bcd4d-7vvzg\" (UID: \"7e49559d-c2a2-4ef5-b806-66ad21212125\") " pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"
Sep 11 05:14:06.084813 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:06.084783    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e49559d-c2a2-4ef5-b806-66ad21212125-maas-api-tls\") pod \"maas-api-646f6bcd4d-7vvzg\" (UID: \"7e49559d-c2a2-4ef5-b806-66ad21212125\") " pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"
Sep 11 05:14:06.085010 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:06.084839    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e49559d-c2a2-4ef5-b806-66ad21212125-openshift-service-ca\") pod \"maas-api-646f6bcd4d-7vvzg\" (UID: \"7e49559d-c2a2-4ef5-b806-66ad21212125\") " pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"
Sep 11 05:14:06.085010 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:06.084913    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e49559d-c2a2-4ef5-b806-66ad21212125-metrics-certs\") pod \"maas-api-646f6bcd4d-7vvzg\" (UID: \"7e49559d-c2a2-4ef5-b806-66ad21212125\") " pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"
Sep 11 05:14:06.085010 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:06.084950    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bv5z\" (UniqueName: \"kubernetes.io/projected/7e49559d-c2a2-4ef5-b806-66ad21212125-kube-api-access-7bv5z\") pod \"maas-api-646f6bcd4d-7vvzg\" (UID: \"7e49559d-c2a2-4ef5-b806-66ad21212125\") " pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"
Sep 11 05:14:06.085726 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:06.085700    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e49559d-c2a2-4ef5-b806-66ad21212125-openshift-service-ca\") pod \"maas-api-646f6bcd4d-7vvzg\" (UID: \"7e49559d-c2a2-4ef5-b806-66ad21212125\") " pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"
Sep 11 05:14:06.087412 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:06.087381    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e49559d-c2a2-4ef5-b806-66ad21212125-maas-api-tls\") pod \"maas-api-646f6bcd4d-7vvzg\" (UID: \"7e49559d-c2a2-4ef5-b806-66ad21212125\") " pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"
Sep 11 05:14:06.087504 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:06.087434    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e49559d-c2a2-4ef5-b806-66ad21212125-metrics-certs\") pod \"maas-api-646f6bcd4d-7vvzg\" (UID: \"7e49559d-c2a2-4ef5-b806-66ad21212125\") " pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"
Sep 11 05:14:06.098858 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:06.098838    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bv5z\" (UniqueName: \"kubernetes.io/projected/7e49559d-c2a2-4ef5-b806-66ad21212125-kube-api-access-7bv5z\") pod \"maas-api-646f6bcd4d-7vvzg\" (UID: \"7e49559d-c2a2-4ef5-b806-66ad21212125\") " pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"
Sep 11 05:14:06.185753 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:06.185675    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"
Sep 11 05:14:06.471563 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:06.471535    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"]
Sep 11 05:14:06.474265 ip-10-0-128-18 kubenswrapper[2567]: W0911 05:14:06.474239    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e49559d_c2a2_4ef5_b806_66ad21212125.slice/crio-9027f4ffc7b86b2e486a0c5a221a7a72357c90d1e6808ea3baee98229bdaf60d WatchSource:0}: Error finding container 9027f4ffc7b86b2e486a0c5a221a7a72357c90d1e6808ea3baee98229bdaf60d: Status 404 returned error can't find the container with id 9027f4ffc7b86b2e486a0c5a221a7a72357c90d1e6808ea3baee98229bdaf60d
Sep 11 05:14:06.475619 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:06.475597    2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 05:14:06.496449 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:06.496420    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg" event={"ID":"7e49559d-c2a2-4ef5-b806-66ad21212125","Type":"ContainerStarted","Data":"9027f4ffc7b86b2e486a0c5a221a7a72357c90d1e6808ea3baee98229bdaf60d"}
Sep 11 05:14:07.502577 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:07.502536    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg" event={"ID":"7e49559d-c2a2-4ef5-b806-66ad21212125","Type":"ContainerStarted","Data":"908a5207c936916f354d795b3ca7fb3d4e7457a786132862a9112a8c13cd0930"}
Sep 11 05:14:07.503044 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:07.502626    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"
Sep 11 05:14:07.525163 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:07.525115    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg" podStartSLOduration=2.1089003 podStartE2EDuration="2.525098227s" totalImagesPullingTime="416.197927ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:14:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:14:07.521986417 +0000 UTC m=+2061.993450292" watchObservedRunningTime="2026-09-11 05:14:07.525098227 +0000 UTC m=+2061.996562103"
Sep 11 05:14:13.514146 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:13.514116    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"
Sep 11 05:14:13.865830 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:13.865748    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-55659b49bd-d89hx"]
Sep 11 05:14:13.869538 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:13.869516    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-55659b49bd-d89hx"
Sep 11 05:14:13.890523 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:13.890499    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-55659b49bd-d89hx"]
Sep 11 05:14:13.956464 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:13.956436    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/921961e8-2710-4713-bef7-cd2de0d43cd5-maas-api-tls\") pod \"maas-api-55659b49bd-d89hx\" (UID: \"921961e8-2710-4713-bef7-cd2de0d43cd5\") " pod="odh-ai-gateway-infra/maas-api-55659b49bd-d89hx"
Sep 11 05:14:13.956464 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:13.956469    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/921961e8-2710-4713-bef7-cd2de0d43cd5-metrics-certs\") pod \"maas-api-55659b49bd-d89hx\" (UID: \"921961e8-2710-4713-bef7-cd2de0d43cd5\") " pod="odh-ai-gateway-infra/maas-api-55659b49bd-d89hx"
Sep 11 05:14:13.956672 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:13.956498    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk4zm\" (UniqueName: \"kubernetes.io/projected/921961e8-2710-4713-bef7-cd2de0d43cd5-kube-api-access-gk4zm\") pod \"maas-api-55659b49bd-d89hx\" (UID: \"921961e8-2710-4713-bef7-cd2de0d43cd5\") " pod="odh-ai-gateway-infra/maas-api-55659b49bd-d89hx"
Sep 11 05:14:13.956672 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:13.956560    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/921961e8-2710-4713-bef7-cd2de0d43cd5-openshift-service-ca\") pod \"maas-api-55659b49bd-d89hx\" (UID: \"921961e8-2710-4713-bef7-cd2de0d43cd5\") " pod="odh-ai-gateway-infra/maas-api-55659b49bd-d89hx"
Sep 11 05:14:14.057965 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:14.057931    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/921961e8-2710-4713-bef7-cd2de0d43cd5-maas-api-tls\") pod \"maas-api-55659b49bd-d89hx\" (UID: \"921961e8-2710-4713-bef7-cd2de0d43cd5\") " pod="odh-ai-gateway-infra/maas-api-55659b49bd-d89hx"
Sep 11 05:14:14.058183 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:14.057979    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/921961e8-2710-4713-bef7-cd2de0d43cd5-metrics-certs\") pod \"maas-api-55659b49bd-d89hx\" (UID: \"921961e8-2710-4713-bef7-cd2de0d43cd5\") " pod="odh-ai-gateway-infra/maas-api-55659b49bd-d89hx"
Sep 11 05:14:14.058183 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:14.058027    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk4zm\" (UniqueName: \"kubernetes.io/projected/921961e8-2710-4713-bef7-cd2de0d43cd5-kube-api-access-gk4zm\") pod \"maas-api-55659b49bd-d89hx\" (UID: \"921961e8-2710-4713-bef7-cd2de0d43cd5\") " pod="odh-ai-gateway-infra/maas-api-55659b49bd-d89hx"
Sep 11 05:14:14.058183 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:14.058061    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/921961e8-2710-4713-bef7-cd2de0d43cd5-openshift-service-ca\") pod \"maas-api-55659b49bd-d89hx\" (UID: \"921961e8-2710-4713-bef7-cd2de0d43cd5\") " pod="odh-ai-gateway-infra/maas-api-55659b49bd-d89hx"
Sep 11 05:14:14.058838 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:14.058811    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/921961e8-2710-4713-bef7-cd2de0d43cd5-openshift-service-ca\") pod \"maas-api-55659b49bd-d89hx\" (UID: \"921961e8-2710-4713-bef7-cd2de0d43cd5\") " pod="odh-ai-gateway-infra/maas-api-55659b49bd-d89hx"
Sep 11 05:14:14.060667 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:14.060644    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/921961e8-2710-4713-bef7-cd2de0d43cd5-maas-api-tls\") pod \"maas-api-55659b49bd-d89hx\" (UID: \"921961e8-2710-4713-bef7-cd2de0d43cd5\") " pod="odh-ai-gateway-infra/maas-api-55659b49bd-d89hx"
Sep 11 05:14:14.060787 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:14.060687    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/921961e8-2710-4713-bef7-cd2de0d43cd5-metrics-certs\") pod \"maas-api-55659b49bd-d89hx\" (UID: \"921961e8-2710-4713-bef7-cd2de0d43cd5\") " pod="odh-ai-gateway-infra/maas-api-55659b49bd-d89hx"
Sep 11 05:14:14.065765 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:14.065741    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk4zm\" (UniqueName: \"kubernetes.io/projected/921961e8-2710-4713-bef7-cd2de0d43cd5-kube-api-access-gk4zm\") pod \"maas-api-55659b49bd-d89hx\" (UID: \"921961e8-2710-4713-bef7-cd2de0d43cd5\") " pod="odh-ai-gateway-infra/maas-api-55659b49bd-d89hx"
Sep 11 05:14:14.181319 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:14.181245    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-55659b49bd-d89hx"
Sep 11 05:14:14.458335 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:14.458229    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-55659b49bd-d89hx"]
Sep 11 05:14:14.461250 ip-10-0-128-18 kubenswrapper[2567]: W0911 05:14:14.461209    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod921961e8_2710_4713_bef7_cd2de0d43cd5.slice/crio-4237407cf4acbdee7b53a2b152cf2354945533240b48d003d16e70d5faf3739b WatchSource:0}: Error finding container 4237407cf4acbdee7b53a2b152cf2354945533240b48d003d16e70d5faf3739b: Status 404 returned error can't find the container with id 4237407cf4acbdee7b53a2b152cf2354945533240b48d003d16e70d5faf3739b
Sep 11 05:14:14.544125 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:14.544089    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-55659b49bd-d89hx" event={"ID":"921961e8-2710-4713-bef7-cd2de0d43cd5","Type":"ContainerStarted","Data":"4237407cf4acbdee7b53a2b152cf2354945533240b48d003d16e70d5faf3739b"}
Sep 11 05:14:15.557930 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:15.557889    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-55659b49bd-d89hx" event={"ID":"921961e8-2710-4713-bef7-cd2de0d43cd5","Type":"ContainerStarted","Data":"9042ab58563d4bf17e2bf7a44b78a49edc6ad93794003d21796f4cc83602f48b"}
Sep 11 05:14:15.558375 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:15.558092    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-55659b49bd-d89hx"
Sep 11 05:14:15.585474 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:15.585420    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-55659b49bd-d89hx" podStartSLOduration=2.268982942 podStartE2EDuration="2.585405428s" totalImagesPullingTime="316.422486ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 05:14:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:14:15.582724912 +0000 UTC m=+2070.054188789" watchObservedRunningTime="2026-09-11 05:14:15.585405428 +0000 UTC m=+2070.056869303"
Sep 11 05:14:20.434380 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:20.434342    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:14:20.540958 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:20.540918    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:14:21.567986 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:21.567956    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-55659b49bd-d89hx"
Sep 11 05:14:21.649128 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:21.649089    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"]
Sep 11 05:14:21.649345 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:21.649301    2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg" podUID="7e49559d-c2a2-4ef5-b806-66ad21212125" containerName="maas-api" containerID="cri-o://908a5207c936916f354d795b3ca7fb3d4e7457a786132862a9112a8c13cd0930" gracePeriod=30
Sep 11 05:14:21.898413 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:21.898384    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"
Sep 11 05:14:22.034097 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:22.034050    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7bv5z\" (UniqueName: \"kubernetes.io/projected/7e49559d-c2a2-4ef5-b806-66ad21212125-kube-api-access-7bv5z\") pod \"7e49559d-c2a2-4ef5-b806-66ad21212125\" (UID: \"7e49559d-c2a2-4ef5-b806-66ad21212125\") "
Sep 11 05:14:22.034253 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:22.034139    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e49559d-c2a2-4ef5-b806-66ad21212125-openshift-service-ca\") pod \"7e49559d-c2a2-4ef5-b806-66ad21212125\" (UID: \"7e49559d-c2a2-4ef5-b806-66ad21212125\") "
Sep 11 05:14:22.034253 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:22.034186    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e49559d-c2a2-4ef5-b806-66ad21212125-maas-api-tls\") pod \"7e49559d-c2a2-4ef5-b806-66ad21212125\" (UID: \"7e49559d-c2a2-4ef5-b806-66ad21212125\") "
Sep 11 05:14:22.034253 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:22.034220    2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e49559d-c2a2-4ef5-b806-66ad21212125-metrics-certs\") pod \"7e49559d-c2a2-4ef5-b806-66ad21212125\" (UID: \"7e49559d-c2a2-4ef5-b806-66ad21212125\") "
Sep 11 05:14:22.034529 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:22.034497    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e49559d-c2a2-4ef5-b806-66ad21212125-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7e49559d-c2a2-4ef5-b806-66ad21212125" (UID: "7e49559d-c2a2-4ef5-b806-66ad21212125"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 05:14:22.036381 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:22.036358    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e49559d-c2a2-4ef5-b806-66ad21212125-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7e49559d-c2a2-4ef5-b806-66ad21212125" (UID: "7e49559d-c2a2-4ef5-b806-66ad21212125"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:14:22.036484 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:22.036381    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e49559d-c2a2-4ef5-b806-66ad21212125-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7e49559d-c2a2-4ef5-b806-66ad21212125" (UID: "7e49559d-c2a2-4ef5-b806-66ad21212125"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 05:14:22.036484 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:22.036445    2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e49559d-c2a2-4ef5-b806-66ad21212125-kube-api-access-7bv5z" (OuterVolumeSpecName: "kube-api-access-7bv5z") pod "7e49559d-c2a2-4ef5-b806-66ad21212125" (UID: "7e49559d-c2a2-4ef5-b806-66ad21212125"). InnerVolumeSpecName "kube-api-access-7bv5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 05:14:22.135406 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:22.135330    2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7bv5z\" (UniqueName: \"kubernetes.io/projected/7e49559d-c2a2-4ef5-b806-66ad21212125-kube-api-access-7bv5z\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:14:22.135406 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:22.135360    2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7e49559d-c2a2-4ef5-b806-66ad21212125-openshift-service-ca\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:14:22.135406 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:22.135370    2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7e49559d-c2a2-4ef5-b806-66ad21212125-maas-api-tls\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:14:22.135406 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:22.135378    2567 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e49559d-c2a2-4ef5-b806-66ad21212125-metrics-certs\") on node \"ip-10-0-128-18.ec2.internal\" DevicePath \"\""
Sep 11 05:14:22.592471 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:22.592432    2567 generic.go:356] "Generic (PLEG): container finished" podID="7e49559d-c2a2-4ef5-b806-66ad21212125" containerID="908a5207c936916f354d795b3ca7fb3d4e7457a786132862a9112a8c13cd0930" exitCode=0
Sep 11 05:14:22.592866 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:22.592475    2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"
Sep 11 05:14:22.592866 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:22.592516    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg" event={"ID":"7e49559d-c2a2-4ef5-b806-66ad21212125","Type":"ContainerDied","Data":"908a5207c936916f354d795b3ca7fb3d4e7457a786132862a9112a8c13cd0930"}
Sep 11 05:14:22.592866 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:22.592571    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg" event={"ID":"7e49559d-c2a2-4ef5-b806-66ad21212125","Type":"ContainerDied","Data":"9027f4ffc7b86b2e486a0c5a221a7a72357c90d1e6808ea3baee98229bdaf60d"}
Sep 11 05:14:22.592866 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:22.592596    2567 scope.go:117] "RemoveContainer" containerID="908a5207c936916f354d795b3ca7fb3d4e7457a786132862a9112a8c13cd0930"
Sep 11 05:14:22.602555 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:22.602541    2567 scope.go:117] "RemoveContainer" containerID="908a5207c936916f354d795b3ca7fb3d4e7457a786132862a9112a8c13cd0930"
Sep 11 05:14:22.602848 ip-10-0-128-18 kubenswrapper[2567]: E0911 05:14:22.602830    2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"908a5207c936916f354d795b3ca7fb3d4e7457a786132862a9112a8c13cd0930\": container with ID starting with 908a5207c936916f354d795b3ca7fb3d4e7457a786132862a9112a8c13cd0930 not found: ID does not exist" containerID="908a5207c936916f354d795b3ca7fb3d4e7457a786132862a9112a8c13cd0930"
Sep 11 05:14:22.602905 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:22.602856    2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"908a5207c936916f354d795b3ca7fb3d4e7457a786132862a9112a8c13cd0930"} err="failed to get container status \"908a5207c936916f354d795b3ca7fb3d4e7457a786132862a9112a8c13cd0930\": rpc error: code = NotFound desc = could not find container \"908a5207c936916f354d795b3ca7fb3d4e7457a786132862a9112a8c13cd0930\": container with ID starting with 908a5207c936916f354d795b3ca7fb3d4e7457a786132862a9112a8c13cd0930 not found: ID does not exist"
Sep 11 05:14:22.622223 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:22.622200    2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"]
Sep 11 05:14:22.627632 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:22.627609    2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-646f6bcd4d-7vvzg"]
Sep 11 05:14:24.017837 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:24.017799    2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e49559d-c2a2-4ef5-b806-66ad21212125" path="/var/lib/kubelet/pods/7e49559d-c2a2-4ef5-b806-66ad21212125/volumes"
Sep 11 05:14:41.875900 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:41.875866    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fbc9b44c4-bht4g_ece7b42e-bc90-4471-bb64-202955606c18/copy-manifests/0.log"
Sep 11 05:14:41.889448 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:41.889423    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fbc9b44c4-bht4g_ece7b42e-bc90-4471-bb64-202955606c18/manager/0.log"
Sep 11 05:14:42.609890 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:42.609864    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-55659b49bd-d89hx_921961e8-2710-4713-bef7-cd2de0d43cd5/maas-api/0.log"
Sep 11 05:14:42.829936 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:42.829908    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-hcp4g_e938b91c-8d6a-42ac-8803-8dd2de7002a2/postgres/0.log"
Sep 11 05:14:44.062602 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:44.062573    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-hz4fd_a9537d97-aca4-4dff-bfa7-940ee3406e69/manager/0.log"
Sep 11 05:14:44.792591 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:44.792564    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-k9cf4_4cbd8223-30c6-4ea7-b72f-017985855258/discovery/0.log"
Sep 11 05:14:45.402812 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:45.402786    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-4wnbl_9d36da1d-c48a-4391-b2eb-5ffc687f1164/payload-processing/0.log"
Sep 11 05:14:46.091411 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:46.091382    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-c8s6w_a91f2a51-ac21-46c4-ae8d-b4be35ff9c62/main/0.log"
Sep 11 05:14:52.427278 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:52.427241    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:14:52.518216 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:52.518187    2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log" limit=1024
Sep 11 05:14:55.675253 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:55.675217    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9zqb2_dbc886e3-0aab-4149-87f3-1d4ec91672f9/global-pull-secret-syncer/0.log"
Sep 11 05:14:55.766511 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:55.766485    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2lz5t_d5d3232f-f007-4c0d-976f-80397eb71a4b/konnectivity-agent/0.log"
Sep 11 05:14:55.860675 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:14:55.860649    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-18.ec2.internal_81eb68d3a2c532023a6d1abc2053e6b2/haproxy/0.log"
Sep 11 05:15:00.474645 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:00.474617    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-hz4fd_a9537d97-aca4-4dff-bfa7-940ee3406e69/manager/0.log"
Sep 11 05:15:02.117317 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:02.117280    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_45fd11dc-75ca-4420-a997-263d7f5f75d3/alertmanager/0.log"
Sep 11 05:15:02.150831 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:02.150794    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_45fd11dc-75ca-4420-a997-263d7f5f75d3/config-reloader/0.log"
Sep 11 05:15:02.175911 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:02.175889    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_45fd11dc-75ca-4420-a997-263d7f5f75d3/kube-rbac-proxy-web/0.log"
Sep 11 05:15:02.203438 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:02.203414    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_45fd11dc-75ca-4420-a997-263d7f5f75d3/kube-rbac-proxy/0.log"
Sep 11 05:15:02.229020 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:02.228949    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_45fd11dc-75ca-4420-a997-263d7f5f75d3/kube-rbac-proxy-metric/0.log"
Sep 11 05:15:02.254118 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:02.254100    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_45fd11dc-75ca-4420-a997-263d7f5f75d3/prom-label-proxy/0.log"
Sep 11 05:15:02.278233 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:02.278212    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_45fd11dc-75ca-4420-a997-263d7f5f75d3/init-config-reloader/0.log"
Sep 11 05:15:02.433203 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:02.433175    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-67fc898877-796tx_277ab579-b700-4770-ac93-8fe753be5622/metrics-server/0.log"
Sep 11 05:15:02.468141 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:02.468117    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5d85778b9d-69vlz_0b9dee47-fd70-4c08-ae4b-4aa8580e0614/monitoring-plugin/0.log"
Sep 11 05:15:02.830105 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:02.830082    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lxknm_1d396c1c-8868-4964-9a46-b02a0d5ffc42/node-exporter/0.log"
Sep 11 05:15:02.879492 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:02.879462    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lxknm_1d396c1c-8868-4964-9a46-b02a0d5ffc42/kube-rbac-proxy/0.log"
Sep 11 05:15:02.910767 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:02.910742    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lxknm_1d396c1c-8868-4964-9a46-b02a0d5ffc42/init-textfile/0.log"
Sep 11 05:15:02.946139 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:02.946044    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-7jp2s_205359f4-7e46-436d-8118-7fc925558ef5/kube-rbac-proxy-main/0.log"
Sep 11 05:15:02.973991 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:02.973959    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-7jp2s_205359f4-7e46-436d-8118-7fc925558ef5/kube-rbac-proxy-self/0.log"
Sep 11 05:15:03.000026 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:03.000003    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-67dcf8c979-7jp2s_205359f4-7e46-436d-8118-7fc925558ef5/openshift-state-metrics/0.log"
Sep 11 05:15:03.053380 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:03.053356    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2261ebde-fc80-4c2b-b741-6db6b80bbfb7/prometheus/0.log"
Sep 11 05:15:03.078371 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:03.078351    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2261ebde-fc80-4c2b-b741-6db6b80bbfb7/config-reloader/0.log"
Sep 11 05:15:03.104052 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:03.104034    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2261ebde-fc80-4c2b-b741-6db6b80bbfb7/thanos-sidecar/0.log"
Sep 11 05:15:03.130743 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:03.130724    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2261ebde-fc80-4c2b-b741-6db6b80bbfb7/kube-rbac-proxy-web/0.log"
Sep 11 05:15:03.175340 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:03.175317    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2261ebde-fc80-4c2b-b741-6db6b80bbfb7/kube-rbac-proxy/0.log"
Sep 11 05:15:03.203265 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:03.203203    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2261ebde-fc80-4c2b-b741-6db6b80bbfb7/kube-rbac-proxy-thanos/0.log"
Sep 11 05:15:03.227913 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:03.227893    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_2261ebde-fc80-4c2b-b741-6db6b80bbfb7/init-config-reloader/0.log"
Sep 11 05:15:03.330964 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:03.330939    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-pm2cn_543c405c-52a2-4808-a21a-ea6141fbeb1e/prometheus-operator-admission-webhook/0.log"
Sep 11 05:15:03.362660 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:03.362636    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-576f54fd8d-4gjnr_cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a/telemeter-client/0.log"
Sep 11 05:15:03.391566 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:03.391544    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-576f54fd8d-4gjnr_cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a/reload/0.log"
Sep 11 05:15:03.443096 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:03.443042    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-576f54fd8d-4gjnr_cacbdfef-e2a3-4c6a-aaf9-3ae5add87b6a/kube-rbac-proxy/0.log"
Sep 11 05:15:03.491987 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:03.491908    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6788cfb4b9-xvtz9_9b8b4da4-4461-4b19-a007-9031ae775cde/thanos-query/0.log"
Sep 11 05:15:03.515046 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:03.515021    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6788cfb4b9-xvtz9_9b8b4da4-4461-4b19-a007-9031ae775cde/kube-rbac-proxy-web/0.log"
Sep 11 05:15:03.542220 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:03.542197    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6788cfb4b9-xvtz9_9b8b4da4-4461-4b19-a007-9031ae775cde/kube-rbac-proxy/0.log"
Sep 11 05:15:03.565273 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:03.565252    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6788cfb4b9-xvtz9_9b8b4da4-4461-4b19-a007-9031ae775cde/prom-label-proxy/0.log"
Sep 11 05:15:03.591921 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:03.591890    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6788cfb4b9-xvtz9_9b8b4da4-4461-4b19-a007-9031ae775cde/kube-rbac-proxy-rules/0.log"
Sep 11 05:15:03.614861 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:03.614835    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6788cfb4b9-xvtz9_9b8b4da4-4461-4b19-a007-9031ae775cde/kube-rbac-proxy-metrics/0.log"
Sep 11 05:15:05.927525 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:05.927486    2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl"]
Sep 11 05:15:05.928124 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:05.928105    2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e49559d-c2a2-4ef5-b806-66ad21212125" containerName="maas-api"
Sep 11 05:15:05.928124 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:05.928124    2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e49559d-c2a2-4ef5-b806-66ad21212125" containerName="maas-api"
Sep 11 05:15:05.928239 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:05.928197    2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e49559d-c2a2-4ef5-b806-66ad21212125" containerName="maas-api"
Sep 11 05:15:05.931317 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:05.931302    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl"
Sep 11 05:15:05.934853 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:05.934830    2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hb6n7\"/\"default-dockercfg-vqpj6\""
Sep 11 05:15:05.935755 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:05.935736    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hb6n7\"/\"kube-root-ca.crt\""
Sep 11 05:15:05.935858 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:05.935766    2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hb6n7\"/\"openshift-service-ca.crt\""
Sep 11 05:15:05.944422 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:05.944398    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl"]
Sep 11 05:15:06.028490 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:06.028466    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb5nt\" (UniqueName: \"kubernetes.io/projected/aaf0d479-842e-4df0-a9bf-23d91609cc90-kube-api-access-zb5nt\") pod \"perf-node-gather-daemonset-tp7vl\" (UID: \"aaf0d479-842e-4df0-a9bf-23d91609cc90\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl"
Sep 11 05:15:06.028645 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:06.028507    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aaf0d479-842e-4df0-a9bf-23d91609cc90-lib-modules\") pod \"perf-node-gather-daemonset-tp7vl\" (UID: \"aaf0d479-842e-4df0-a9bf-23d91609cc90\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl"
Sep 11 05:15:06.028645 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:06.028563    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aaf0d479-842e-4df0-a9bf-23d91609cc90-proc\") pod \"perf-node-gather-daemonset-tp7vl\" (UID: \"aaf0d479-842e-4df0-a9bf-23d91609cc90\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl"
Sep 11 05:15:06.028645 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:06.028605    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aaf0d479-842e-4df0-a9bf-23d91609cc90-podres\") pod \"perf-node-gather-daemonset-tp7vl\" (UID: \"aaf0d479-842e-4df0-a9bf-23d91609cc90\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl"
Sep 11 05:15:06.028645 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:06.028623    2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aaf0d479-842e-4df0-a9bf-23d91609cc90-sys\") pod \"perf-node-gather-daemonset-tp7vl\" (UID: \"aaf0d479-842e-4df0-a9bf-23d91609cc90\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl"
Sep 11 05:15:06.129311 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:06.129281    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aaf0d479-842e-4df0-a9bf-23d91609cc90-podres\") pod \"perf-node-gather-daemonset-tp7vl\" (UID: \"aaf0d479-842e-4df0-a9bf-23d91609cc90\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl"
Sep 11 05:15:06.129311 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:06.129311    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aaf0d479-842e-4df0-a9bf-23d91609cc90-sys\") pod \"perf-node-gather-daemonset-tp7vl\" (UID: \"aaf0d479-842e-4df0-a9bf-23d91609cc90\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl"
Sep 11 05:15:06.129524 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:06.129344    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb5nt\" (UniqueName: \"kubernetes.io/projected/aaf0d479-842e-4df0-a9bf-23d91609cc90-kube-api-access-zb5nt\") pod \"perf-node-gather-daemonset-tp7vl\" (UID: \"aaf0d479-842e-4df0-a9bf-23d91609cc90\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl"
Sep 11 05:15:06.129524 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:06.129379    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aaf0d479-842e-4df0-a9bf-23d91609cc90-lib-modules\") pod \"perf-node-gather-daemonset-tp7vl\" (UID: \"aaf0d479-842e-4df0-a9bf-23d91609cc90\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl"
Sep 11 05:15:06.129524 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:06.129412    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aaf0d479-842e-4df0-a9bf-23d91609cc90-sys\") pod \"perf-node-gather-daemonset-tp7vl\" (UID: \"aaf0d479-842e-4df0-a9bf-23d91609cc90\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl"
Sep 11 05:15:06.129524 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:06.129419    2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aaf0d479-842e-4df0-a9bf-23d91609cc90-proc\") pod \"perf-node-gather-daemonset-tp7vl\" (UID: \"aaf0d479-842e-4df0-a9bf-23d91609cc90\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl"
Sep 11 05:15:06.129524 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:06.129459    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aaf0d479-842e-4df0-a9bf-23d91609cc90-podres\") pod \"perf-node-gather-daemonset-tp7vl\" (UID: \"aaf0d479-842e-4df0-a9bf-23d91609cc90\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl"
Sep 11 05:15:06.129524 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:06.129467    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aaf0d479-842e-4df0-a9bf-23d91609cc90-proc\") pod \"perf-node-gather-daemonset-tp7vl\" (UID: \"aaf0d479-842e-4df0-a9bf-23d91609cc90\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl"
Sep 11 05:15:06.129729 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:06.129534    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aaf0d479-842e-4df0-a9bf-23d91609cc90-lib-modules\") pod \"perf-node-gather-daemonset-tp7vl\" (UID: \"aaf0d479-842e-4df0-a9bf-23d91609cc90\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl"
Sep 11 05:15:06.139018 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:06.138997    2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb5nt\" (UniqueName: \"kubernetes.io/projected/aaf0d479-842e-4df0-a9bf-23d91609cc90-kube-api-access-zb5nt\") pod \"perf-node-gather-daemonset-tp7vl\" (UID: \"aaf0d479-842e-4df0-a9bf-23d91609cc90\") " pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl"
Sep 11 05:15:06.241147 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:06.241043    2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl"
Sep 11 05:15:06.594242 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:06.594211    2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl"]
Sep 11 05:15:06.605232 ip-10-0-128-18 kubenswrapper[2567]: W0911 05:15:06.605195    2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podaaf0d479_842e_4df0_a9bf_23d91609cc90.slice/crio-d4020c33152b96c50ef9ceb4d237147eb2ff5af3c8ca17dc8dbc6f01668a0fb6 WatchSource:0}: Error finding container d4020c33152b96c50ef9ceb4d237147eb2ff5af3c8ca17dc8dbc6f01668a0fb6: Status 404 returned error can't find the container with id d4020c33152b96c50ef9ceb4d237147eb2ff5af3c8ca17dc8dbc6f01668a0fb6
Sep 11 05:15:06.825153 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:06.825119    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl" event={"ID":"aaf0d479-842e-4df0-a9bf-23d91609cc90","Type":"ContainerStarted","Data":"5b73ae2197c20ba51fe76aa7524bb69dafc6e9e7d31400ee75cc307cd81c8199"}
Sep 11 05:15:06.825302 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:06.825172    2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl" event={"ID":"aaf0d479-842e-4df0-a9bf-23d91609cc90","Type":"ContainerStarted","Data":"d4020c33152b96c50ef9ceb4d237147eb2ff5af3c8ca17dc8dbc6f01668a0fb6"}
Sep 11 05:15:06.825302 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:06.825197    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl"
Sep 11 05:15:08.585229 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:08.585194    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dt7d2_b855f12e-f907-45d0-b777-5930f8a8af4c/dns/0.log"
Sep 11 05:15:08.632056 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:08.632002    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-dt7d2_b855f12e-f907-45d0-b777-5930f8a8af4c/kube-rbac-proxy/0.log"
Sep 11 05:15:08.780619 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:08.780595    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-clstx_53a003c6-2a55-4b7c-9242-766116f41b2a/dns-node-resolver/0.log"
Sep 11 05:15:09.639144 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:09.639111    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7thkg_57e4b5b7-b3cc-4052-be3c-07a138c81e91/node-ca/0.log"
Sep 11 05:15:10.978836 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:10.978801    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-k9cf4_4cbd8223-30c6-4ea7-b72f-017985855258/discovery/0.log"
Sep 11 05:15:11.186821 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:11.186786    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-4wnbl_9d36da1d-c48a-4391-b2eb-5ffc687f1164/payload-processing/0.log"
Sep 11 05:15:12.139592 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:12.139562    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2t5fm_9285e03b-0880-4ae3-b25d-3d4cd81ba474/serve-healthcheck-canary/0.log"
Sep 11 05:15:12.753454 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:12.753423    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q9lj5_d9c0d97b-b5e9-4996-9091-84bab1fb8529/kube-rbac-proxy/0.log"
Sep 11 05:15:12.778248 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:12.778224    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q9lj5_d9c0d97b-b5e9-4996-9091-84bab1fb8529/exporter/0.log"
Sep 11 05:15:12.800450 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:12.800426    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q9lj5_d9c0d97b-b5e9-4996-9091-84bab1fb8529/extractor/0.log"
Sep 11 05:15:12.840298 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:12.840281    2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl"
Sep 11 05:15:12.861706 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:12.861664    2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hb6n7/perf-node-gather-daemonset-tp7vl" podStartSLOduration=7.861651413 podStartE2EDuration="7.861651413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 05:15:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 05:15:06.854394798 +0000 UTC m=+2121.325858674" watchObservedRunningTime="2026-09-11 05:15:12.861651413 +0000 UTC m=+2127.333115288"
Sep 11 05:15:15.741506 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:15.741478    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-k9cf4_4cbd8223-30c6-4ea7-b72f-017985855258/discovery/0.log"
Sep 11 05:15:15.756251 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:15.756225    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fbc9b44c4-bht4g_ece7b42e-bc90-4471-bb64-202955606c18/manager/0.log"
Sep 11 05:15:15.781384 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:15.781360    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7fbc9b44c4-bht4g_ece7b42e-bc90-4471-bb64-202955606c18/copy-manifests/0.log"
Sep 11 05:15:15.873651 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:15.873626    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-4wnbl_9d36da1d-c48a-4391-b2eb-5ffc687f1164/payload-processing/0.log"
Sep 11 05:15:17.299254 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:17.299225    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6f84bb6f84-hlxct_84168371-431c-43d4-aa16-06e00e771aeb/manager/0.log"
Sep 11 05:15:17.595743 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:17.595653    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-k9cf4_4cbd8223-30c6-4ea7-b72f-017985855258/discovery/0.log"
Sep 11 05:15:17.739301 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:17.739270    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-4wnbl_9d36da1d-c48a-4391-b2eb-5ffc687f1164/payload-processing/0.log"
Sep 11 05:15:22.288910 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:22.288877    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-xpr6g_936d53ea-0a0d-4f20-9884-83c4a93f5550/migrator/0.log"
Sep 11 05:15:22.308465 ip-10-0-128-18 kubenswrapper[2567]: I0911 05:15:22.308441    2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-xpr6g_936d53ea-0a0d-4f20-9884-83c4a93f5550/graceful-termination/0.log"