Sep 04 21:51:41.620717 ip-10-0-134-141 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 04 21:51:41.620729 ip-10-0-134-141 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 04 21:51:41.620739 ip-10-0-134-141 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 04 21:51:41.621093 ip-10-0-134-141 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 04 21:51:51.687535 ip-10-0-134-141 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 04 21:51:51.687553 ip-10-0-134-141 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 747f7d8a33b04231b855fe9219907c55 --
Sep 04 21:53:44.332180 ip-10-0-134-141 systemd[1]: Starting Kubernetes Kubelet...
Sep 04 21:53:44.731400 ip-10-0-134-141 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 04 21:53:44.731400 ip-10-0-134-141 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 04 21:53:44.731400 ip-10-0-134-141 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 04 21:53:44.731400 ip-10-0-134-141 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 04 21:53:44.731400 ip-10-0-134-141 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 04 21:53:44.732397 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.732306    2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 04 21:53:44.738553 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738522    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 21:53:44.738553 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738547    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 21:53:44.738553 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738551    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 21:53:44.738553 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738554    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 21:53:44.738553 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738558    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 21:53:44.738553 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738561    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 21:53:44.738553 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738565    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 21:53:44.738553 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738569    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 21:53:44.738967 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738572    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 21:53:44.738967 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738575    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 21:53:44.738967 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738577    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 21:53:44.738967 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738580    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 21:53:44.738967 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738583    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 21:53:44.738967 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738586    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 21:53:44.738967 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738589    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 21:53:44.738967 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738592    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 21:53:44.738967 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738595    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 21:53:44.738967 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738599    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 21:53:44.738967 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738603    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 21:53:44.738967 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738606    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 21:53:44.738967 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738609    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 21:53:44.738967 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738612    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 21:53:44.738967 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738615    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 21:53:44.738967 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738618    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 21:53:44.738967 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738621    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 21:53:44.738967 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738624    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 21:53:44.738967 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738629    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 21:53:44.739641 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738632    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 21:53:44.739641 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738635    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 21:53:44.739641 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738639    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 21:53:44.739641 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738642    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 21:53:44.739641 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738645    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 21:53:44.739641 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738648    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 21:53:44.739641 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738650    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 21:53:44.739641 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738653    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 21:53:44.739641 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738656    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 21:53:44.739641 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738658    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 21:53:44.739641 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738661    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 21:53:44.739641 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738663    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 21:53:44.739641 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738666    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 21:53:44.739641 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738668    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 21:53:44.739641 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738673    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 21:53:44.739641 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738676    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 21:53:44.739641 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738678    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 21:53:44.739641 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738683    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 21:53:44.739641 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738687    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 21:53:44.739641 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738691    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 21:53:44.740397 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738693    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 21:53:44.740397 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738696    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 21:53:44.740397 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738700    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 21:53:44.740397 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738704    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 21:53:44.740397 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738707    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 21:53:44.740397 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738710    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 21:53:44.740397 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738713    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 21:53:44.740397 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738716    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 21:53:44.740397 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738718    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 21:53:44.740397 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738722    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 21:53:44.740397 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738724    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 21:53:44.740397 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738727    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 21:53:44.740397 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738729    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 21:53:44.740397 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738732    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 21:53:44.740397 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738734    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 21:53:44.740397 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738737    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 21:53:44.740397 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738740    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 21:53:44.740397 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738742    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 21:53:44.740397 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738745    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 21:53:44.740397 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738748    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 21:53:44.741135 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738750    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 21:53:44.741135 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738753    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 21:53:44.741135 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738756    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 21:53:44.741135 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738759    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 21:53:44.741135 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738762    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 21:53:44.741135 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738764    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 21:53:44.741135 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738767    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 21:53:44.741135 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738770    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 21:53:44.741135 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738773    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 21:53:44.741135 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738775    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 21:53:44.741135 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738779    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 21:53:44.741135 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738782    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 21:53:44.741135 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738784    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 21:53:44.741135 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738787    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 21:53:44.741135 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738790    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 21:53:44.741135 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738792    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 21:53:44.741135 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738796    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 21:53:44.741135 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738799    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 21:53:44.741135 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738802    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 21:53:44.741135 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738805    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 21:53:44.741847 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738807    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 21:53:44.741847 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738810    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 21:53:44.741847 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738812    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 21:53:44.741847 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738815    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 21:53:44.741847 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738818    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 21:53:44.741847 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738821    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 21:53:44.741847 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738823    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 21:53:44.741847 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738826    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 21:53:44.741847 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738829    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 21:53:44.741847 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738832    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 21:53:44.741847 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738834    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 21:53:44.741847 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.738837    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 21:53:44.741847 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740584    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 21:53:44.741847 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740593    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 21:53:44.741847 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740597    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 21:53:44.741847 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740601    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 21:53:44.741847 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740604    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 21:53:44.741847 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740607    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 21:53:44.741847 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740611    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 21:53:44.741847 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740615    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 21:53:44.742627 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740618    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 21:53:44.742627 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740621    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 21:53:44.742627 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740624    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 21:53:44.742627 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740627    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 21:53:44.742627 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740630    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 21:53:44.742627 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740633    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 21:53:44.742627 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740635    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 21:53:44.742627 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740638    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 21:53:44.742627 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740641    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 21:53:44.742627 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740645    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 21:53:44.742627 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740648    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 21:53:44.742627 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740651    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 21:53:44.742627 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740653    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 21:53:44.742627 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740656    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 21:53:44.742627 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740658    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 21:53:44.742627 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740661    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 21:53:44.742627 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740664    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 21:53:44.742627 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740666    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 21:53:44.742627 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740669    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 21:53:44.743472 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740673    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 21:53:44.743472 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740677    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 21:53:44.743472 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740680    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 21:53:44.743472 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740683    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 21:53:44.743472 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740686    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 21:53:44.743472 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740688    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 21:53:44.743472 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740691    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 21:53:44.743472 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740693    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 21:53:44.743472 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740696    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 21:53:44.743472 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740699    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 21:53:44.743472 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740702    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 21:53:44.743472 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740706    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 21:53:44.743472 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740710    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 21:53:44.743472 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740713    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 21:53:44.743472 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740716    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 21:53:44.743472 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740719    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 21:53:44.743472 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740722    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 21:53:44.743472 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740725    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 21:53:44.744142 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740729    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 21:53:44.744142 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740732    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 21:53:44.744142 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740735    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 21:53:44.744142 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740738    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 21:53:44.744142 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740741    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 21:53:44.744142 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740749    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 21:53:44.744142 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740753    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 21:53:44.744142 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740756    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 21:53:44.744142 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740759    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 21:53:44.744142 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740762    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 21:53:44.744142 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740764    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 21:53:44.744142 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740767    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 21:53:44.744142 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740770    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 21:53:44.744142 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740772    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 21:53:44.744142 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740778    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 21:53:44.744142 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740781    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 21:53:44.744142 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740783    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 21:53:44.744142 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740786    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 21:53:44.744142 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740789    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 21:53:44.744839 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740792    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 21:53:44.744839 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740795    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 21:53:44.744839 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740798    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 21:53:44.744839 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740801    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 21:53:44.744839 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740804    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 21:53:44.744839 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740807    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 21:53:44.744839 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740810    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 21:53:44.744839 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740812    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 21:53:44.744839 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740815    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 21:53:44.744839 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740818    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 21:53:44.744839 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740821    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 21:53:44.744839 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740824    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 21:53:44.744839 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740827    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 21:53:44.744839 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740829    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 21:53:44.744839 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740832    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 21:53:44.744839 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740834    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 21:53:44.744839 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740837    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 21:53:44.744839 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740840    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 21:53:44.744839 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740842    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 21:53:44.744839 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740845    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 21:53:44.745577 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740847    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 21:53:44.745577 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740850    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 21:53:44.745577 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740853    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 21:53:44.745577 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740871    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 21:53:44.745577 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740874    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 21:53:44.745577 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740877    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 21:53:44.745577 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740879    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 21:53:44.745577 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740884    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 21:53:44.745577 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740886    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 21:53:44.745577 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740889    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 21:53:44.745577 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740892    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 21:53:44.745577 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740895    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 21:53:44.745577 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740897    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 21:53:44.745577 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740900    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 21:53:44.745577 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.740903    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 21:53:44.745577 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741374    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 21:53:44.745577 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741380    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 21:53:44.745577 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741383    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 21:53:44.745577 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741387    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 21:53:44.745577 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741390    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 21:53:44.746312 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741393    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 21:53:44.746312 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741396    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 21:53:44.746312 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741398    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 21:53:44.746312 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741401    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 21:53:44.746312 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741404    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 21:53:44.746312 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741406    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 21:53:44.746312 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741409    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 21:53:44.746312 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741412    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 21:53:44.746312 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741415    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 21:53:44.746312 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741418    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 21:53:44.746312 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741421    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 21:53:44.746312 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741424    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 21:53:44.746312 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741426    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 21:53:44.746312 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741429    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 21:53:44.746312 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741432    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 21:53:44.746312 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741434    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 21:53:44.746312 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741437    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 21:53:44.746312 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741440    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 21:53:44.746312 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741442    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 21:53:44.747024 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741446    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 21:53:44.747024 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741448    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 21:53:44.747024 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741451    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 21:53:44.747024 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741453    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 21:53:44.747024 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741456    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 21:53:44.747024 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741458    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 21:53:44.747024 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741461    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 21:53:44.747024 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741463    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 21:53:44.747024 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741466    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 21:53:44.747024 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741469    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 21:53:44.747024 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741471    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 21:53:44.747024 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741474    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 21:53:44.747024 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741478    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 21:53:44.747024 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741482    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 21:53:44.747024 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741485    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 21:53:44.747024 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741488    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 21:53:44.747024 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741491    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 21:53:44.747024 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741494    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 21:53:44.747024 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741497    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 21:53:44.747709 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741499    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 21:53:44.747709 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741502    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 21:53:44.747709 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741505    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 21:53:44.747709 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741508    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 21:53:44.747709 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741511    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 21:53:44.747709 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741513    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 21:53:44.747709 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741516    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 21:53:44.747709 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741519    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 21:53:44.747709 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741521    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 21:53:44.747709 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741524    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 21:53:44.747709 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741527    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 21:53:44.747709 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741531    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 21:53:44.747709 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741534    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 21:53:44.747709 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741538    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 21:53:44.747709 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741540    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 21:53:44.747709 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741543    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 21:53:44.747709 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741546    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 21:53:44.747709 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741548    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 21:53:44.747709 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741551    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 21:53:44.747709 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741553    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 21:53:44.748507 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741556    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 21:53:44.748507 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741558    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 21:53:44.748507 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741561    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 21:53:44.748507 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741564    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 21:53:44.748507 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741567    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 21:53:44.748507 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741570    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 21:53:44.748507 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741572    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 21:53:44.748507 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741576    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 21:53:44.748507 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741580    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 21:53:44.748507 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741583    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 21:53:44.748507 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741586    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 21:53:44.748507 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741589    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 21:53:44.748507 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741592    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 21:53:44.748507 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741594    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 21:53:44.748507 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741597    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 21:53:44.748507 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741599    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 21:53:44.748507 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741602    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 21:53:44.748507 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741605    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 21:53:44.748507 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741607    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 21:53:44.748507 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741610    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 21:53:44.749254 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741613    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 21:53:44.749254 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741616    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 21:53:44.749254 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741618    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 21:53:44.749254 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741621    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 21:53:44.749254 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741624    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 21:53:44.749254 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741627    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 21:53:44.749254 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741629    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 21:53:44.749254 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741633    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 21:53:44.749254 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741636    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 21:53:44.749254 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741639    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 21:53:44.749254 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741641    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 21:53:44.749254 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741644    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 21:53:44.749254 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741646    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 21:53:44.749254 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741649    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 21:53:44.749254 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741651    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 21:53:44.749254 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.741654    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 21:53:44.749254 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741736    2557 flags.go:64] FLAG: --address="0.0.0.0"
Sep 04 21:53:44.749254 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741784    2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 04 21:53:44.749254 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741792    2557 flags.go:64] FLAG: --anonymous-auth="true"
Sep 04 21:53:44.749254 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741797    2557 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 04 21:53:44.749254 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741801    2557 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741804    2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741809    2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741813    2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741817    2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741820    2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741823    2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741826    2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741829    2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741832    2557 flags.go:64] FLAG: --cgroup-root=""
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741835    2557 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741838    2557 flags.go:64] FLAG: --client-ca-file=""
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741843    2557 flags.go:64] FLAG: --cloud-provider="external"
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741846    2557 flags.go:64] FLAG: --cluster-dns="[]"
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741851    2557 flags.go:64] FLAG: --cluster-domain=""
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741866    2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741869    2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741873    2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741876    2557 flags.go:64] FLAG: --container-log-max-files="5"
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741883    2557 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741886    2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741889    2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741893    2557 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741896    2557 flags.go:64] FLAG: --contention-profiling="false"
Sep 04 21:53:44.750087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741899    2557 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741902    2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741905    2557 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741908    2557 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741912    2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741915    2557 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741918    2557 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741921    2557 flags.go:64] FLAG: --enable-load-reader="false"
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741924    2557 flags.go:64] FLAG: --enable-server="true"
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741927    2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741932    2557 flags.go:64] FLAG: --event-burst="100"
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741935    2557 flags.go:64] FLAG: --event-qps="50"
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741938    2557 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741941    2557 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741944    2557 flags.go:64] FLAG: --eviction-hard=""
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741948    2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741951    2557 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741954    2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741957    2557 flags.go:64] FLAG: --eviction-soft=""
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741960    2557 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741963    2557 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741966    2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741969    2557 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741974    2557 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741977    2557 flags.go:64] FLAG: --fail-swap-on="true"
Sep 04 21:53:44.750979 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741979    2557 flags.go:64] FLAG: --feature-gates=""
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741983    2557 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741987    2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741990    2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741993    2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.741997    2557 flags.go:64] FLAG: --healthz-port="10248"
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742000    2557 flags.go:64] FLAG: --help="false"
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742004    2557 flags.go:64] FLAG: --hostname-override="ip-10-0-134-141.ec2.internal"
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742007    2557 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742010    2557 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742013    2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742017    2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742020    2557 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742023    2557 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742026    2557 flags.go:64] FLAG: --image-service-endpoint=""
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742029    2557 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742031    2557 flags.go:64] FLAG: --kube-api-burst="100"
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742035    2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742038    2557 flags.go:64] FLAG: --kube-api-qps="50"
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742041    2557 flags.go:64] FLAG: --kube-reserved=""
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742044    2557 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742047    2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742049    2557 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742052    2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 04 21:53:44.751918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742055    2557 flags.go:64] FLAG: --lock-file=""
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742058    2557 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742061    2557 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742064    2557 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742069    2557 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742073    2557 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742077    2557 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742080    2557 flags.go:64] FLAG: --logging-format="text"
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742083    2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742087    2557 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742091    2557 flags.go:64] FLAG: --manifest-url=""
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742094    2557 flags.go:64] FLAG: --manifest-url-header=""
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742099    2557 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742102    2557 flags.go:64] FLAG: --max-open-files="1000000"
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742106    2557 flags.go:64] FLAG: --max-pods="110"
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742110    2557 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742112    2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742115    2557 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742118    2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742121    2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742124    2557 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742127    2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742135    2557 flags.go:64] FLAG: --node-status-max-images="50"
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742138    2557 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742141    2557 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 04 21:53:44.752809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742144    2557 flags.go:64] FLAG: --pod-cidr=""
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742147    2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742153    2557 flags.go:64] FLAG: --pod-manifest-path=""
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742155    2557 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742159    2557 flags.go:64] FLAG: --pods-per-core="0"
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742161    2557 flags.go:64] FLAG: --port="10250"
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742165    2557 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742168    2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a824c65240f72189"
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742171    2557 flags.go:64] FLAG: --qos-reserved=""
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742174    2557 flags.go:64] FLAG: --read-only-port="10255"
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742177    2557 flags.go:64] FLAG: --register-node="true"
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742180    2557 flags.go:64] FLAG: --register-with-taints=""
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742183    2557 flags.go:64] FLAG: --registry-burst="10"
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742188    2557 flags.go:64] FLAG: --registry-qps="5"
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742192    2557 flags.go:64] FLAG: --reserved-cpus=""
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742195    2557 flags.go:64] FLAG: --reserved-memory=""
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742198    2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742202    2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742206    2557 flags.go:64] FLAG: --rotate-certificates="false"
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742209    2557 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742212    2557 flags.go:64] FLAG: --runonce="false"
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742215    2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742218    2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742221    2557 flags.go:64] FLAG: --seccomp-default="false"
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742223    2557 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 04 21:53:44.753774 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742226    2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742229    2557 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742233    2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742236    2557 flags.go:64] FLAG: --storage-driver-password="root"
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742239    2557 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742242    2557 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742245    2557 flags.go:64] FLAG: --storage-driver-user="root"
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742248    2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742250    2557 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742253    2557 flags.go:64] FLAG: --system-cgroups=""
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742256    2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742262    2557 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742264    2557 flags.go:64] FLAG: --tls-cert-file=""
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742267    2557 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742272    2557 flags.go:64] FLAG: --tls-min-version=""
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742275    2557 flags.go:64] FLAG: --tls-private-key-file=""
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742278    2557 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742280    2557 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742283    2557 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742286    2557 flags.go:64] FLAG: --v="2"
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742292    2557 flags.go:64] FLAG: --version="false"
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742297    2557 flags.go:64] FLAG: --vmodule=""
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742301    2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742304    2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742422    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 21:53:44.754751 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742427    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 21:53:44.755698 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742430    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 21:53:44.755698 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742433    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 21:53:44.755698 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742437    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 21:53:44.755698 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742441    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 21:53:44.755698 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742444    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 21:53:44.755698 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742447    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 21:53:44.755698 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742451    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 21:53:44.755698 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742454    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 21:53:44.755698 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742457    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 21:53:44.755698 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742460    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 21:53:44.755698 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742463    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 21:53:44.755698 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742466    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 21:53:44.755698 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742468    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 21:53:44.755698 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742471    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 21:53:44.755698 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742474    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 21:53:44.755698 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742476    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 21:53:44.755698 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742479    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 21:53:44.755698 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742481    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 21:53:44.755698 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742484    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 21:53:44.756409 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742486    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 21:53:44.756409 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742489    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 21:53:44.756409 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742492    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 21:53:44.756409 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742494    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 21:53:44.756409 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742497    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 21:53:44.756409 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742499    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 21:53:44.756409 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742502    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 21:53:44.756409 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742506    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 21:53:44.756409 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742508    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 21:53:44.756409 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742511    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 21:53:44.756409 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742514    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 21:53:44.756409 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742517    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 21:53:44.756409 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742521    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 21:53:44.756409 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742524    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 21:53:44.756409 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742527    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 21:53:44.756409 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742529    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 21:53:44.756409 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742532    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 21:53:44.756409 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742535    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 21:53:44.756409 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742537    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 21:53:44.756409 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742540    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 21:53:44.757173 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742543    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 21:53:44.757173 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742545    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 21:53:44.757173 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742548    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 21:53:44.757173 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742551    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 21:53:44.757173 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742554    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 21:53:44.757173 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742556    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 21:53:44.757173 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742559    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 21:53:44.757173 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742561    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 21:53:44.757173 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742564    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 21:53:44.757173 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742568    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 21:53:44.757173 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742571    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 21:53:44.757173 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742574    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 21:53:44.757173 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742577    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 21:53:44.757173 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742580    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 21:53:44.757173 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742583    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 21:53:44.757173 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742586    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 21:53:44.757173 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742589    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 21:53:44.757173 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742592    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 21:53:44.757173 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742595    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 21:53:44.757886 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742599    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 21:53:44.757886 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742602    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 21:53:44.757886 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742605    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 21:53:44.757886 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742609    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 21:53:44.757886 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742611    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 21:53:44.757886 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742615    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 21:53:44.757886 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742618    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 21:53:44.757886 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742620    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 21:53:44.757886 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742632    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 21:53:44.757886 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742635    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 21:53:44.757886 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742638    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 21:53:44.757886 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742641    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 21:53:44.757886 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742644    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 21:53:44.757886 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742646    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 21:53:44.757886 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742649    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 21:53:44.757886 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742652    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 21:53:44.757886 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742655    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 21:53:44.757886 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742658    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 21:53:44.757886 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742660    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 21:53:44.757886 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742663    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 21:53:44.758631 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742665    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 21:53:44.758631 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742668    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 21:53:44.758631 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742671    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 21:53:44.758631 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742674    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 21:53:44.758631 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742676    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 21:53:44.758631 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742679    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 21:53:44.758631 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742681    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 21:53:44.758631 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742684    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 21:53:44.758631 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742687    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 21:53:44.758631 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742690    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 21:53:44.758631 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742692    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 21:53:44.758631 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742695    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 21:53:44.758631 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742702    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 21:53:44.758631 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742705    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 21:53:44.758631 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742708    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 21:53:44.758631 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742710    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 21:53:44.758631 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742714    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 21:53:44.758631 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742718    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 21:53:44.758631 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.742721    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 21:53:44.759352 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.742726    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 04 21:53:44.759352 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.750748    2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 04 21:53:44.759352 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.750767    2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 04 21:53:44.759352 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.750789    2557 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 04 21:53:44.759352 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.750796    2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 04 21:53:44.759352 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750883    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 21:53:44.759352 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750890    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 21:53:44.759352 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750893    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 21:53:44.759352 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750896    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 21:53:44.759352 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750900    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 21:53:44.759352 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750903    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 21:53:44.759352 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750906    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 21:53:44.759352 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750910    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 21:53:44.759352 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750913    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 21:53:44.759352 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750916    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 21:53:44.760005 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750919    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 21:53:44.760005 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750922    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 21:53:44.760005 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750925    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 21:53:44.760005 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750928    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 21:53:44.760005 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750931    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 21:53:44.760005 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750933    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 21:53:44.760005 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750936    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 21:53:44.760005 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750939    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 21:53:44.760005 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750942    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 21:53:44.760005 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750945    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 21:53:44.760005 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750947    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 21:53:44.760005 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750950    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 21:53:44.760005 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750953    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 21:53:44.760005 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750955    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 21:53:44.760005 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750958    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 21:53:44.760005 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750961    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 21:53:44.760005 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750964    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 21:53:44.760005 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750966    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 21:53:44.760005 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750969    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 21:53:44.760704 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750973    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 21:53:44.760704 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750976    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 21:53:44.760704 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750979    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 21:53:44.760704 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750982    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 21:53:44.760704 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750984    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 21:53:44.760704 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750987    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 21:53:44.760704 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750990    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 21:53:44.760704 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750993    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 21:53:44.760704 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750995    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 21:53:44.760704 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.750998    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 21:53:44.760704 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751001    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 21:53:44.760704 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751003    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 21:53:44.760704 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751006    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 21:53:44.760704 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751010    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 21:53:44.760704 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751013    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 21:53:44.760704 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751016    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 21:53:44.760704 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751021    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 21:53:44.760704 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751025    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 21:53:44.761499 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751028    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 21:53:44.761499 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751031    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 21:53:44.761499 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751034    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 21:53:44.761499 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751037    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 21:53:44.761499 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751040    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 21:53:44.761499 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751043    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 21:53:44.761499 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751046    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 21:53:44.761499 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751048    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 21:53:44.761499 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751051    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 21:53:44.761499 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751053    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 21:53:44.761499 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751056    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 21:53:44.761499 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751059    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 21:53:44.761499 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751061    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 21:53:44.761499 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751064    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 21:53:44.761499 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751068    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 21:53:44.761499 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751071    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 21:53:44.761499 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751074    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 21:53:44.761499 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751076    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 21:53:44.761499 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751079    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 21:53:44.761499 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751082    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 21:53:44.762242 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751085    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 21:53:44.762242 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751087    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 21:53:44.762242 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751090    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 21:53:44.762242 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751093    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 21:53:44.762242 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751096    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 21:53:44.762242 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751098    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 21:53:44.762242 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751101    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 21:53:44.762242 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751103    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 21:53:44.762242 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751106    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 21:53:44.762242 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751108    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 21:53:44.762242 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751111    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 21:53:44.762242 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751114    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 21:53:44.762242 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751117    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 21:53:44.762242 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751119    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 21:53:44.762242 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751122    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 21:53:44.762242 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751124    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 21:53:44.762242 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751127    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 21:53:44.762242 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751130    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 21:53:44.762242 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751132    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 21:53:44.762242 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751135    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 21:53:44.763007 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751138    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 21:53:44.763007 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751142    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 21:53:44.763007 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751145    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 21:53:44.763007 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751148    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 21:53:44.763007 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751151    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 21:53:44.763007 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751154    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 21:53:44.763007 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751157    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 21:53:44.763007 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751160    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 21:53:44.763007 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751163    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 21:53:44.763007 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751166    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 21:53:44.763007 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751169    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 21:53:44.763007 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751172    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 21:53:44.763007 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.751177    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 04 21:53:44.763007 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751320    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 04 21:53:44.763007 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751326    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 04 21:53:44.763573 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751329    2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 04 21:53:44.763573 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751332    2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 04 21:53:44.763573 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751335    2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 04 21:53:44.763573 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751338    2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 04 21:53:44.763573 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751341    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 04 21:53:44.763573 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751344    2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 04 21:53:44.763573 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751347    2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 04 21:53:44.763573 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751350    2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 04 21:53:44.763573 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751352    2557 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 04 21:53:44.763573 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751355    2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 04 21:53:44.763573 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751358    2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 04 21:53:44.763573 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751360    2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 04 21:53:44.763573 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751363    2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 04 21:53:44.763573 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751366    2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 04 21:53:44.763573 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751368    2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 04 21:53:44.763573 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751371    2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 04 21:53:44.763573 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751373    2557 feature_gate.go:328] unrecognized feature gate: Example2
Sep 04 21:53:44.763573 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751376    2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 04 21:53:44.763573 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751378    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 04 21:53:44.763573 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751381    2557 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 04 21:53:44.764331 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751384    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 04 21:53:44.764331 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751387    2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 04 21:53:44.764331 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751389    2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 04 21:53:44.764331 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751392    2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 04 21:53:44.764331 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751394    2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 04 21:53:44.764331 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751397    2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 04 21:53:44.764331 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751401    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 04 21:53:44.764331 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751403    2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 04 21:53:44.764331 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751406    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 04 21:53:44.764331 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751409    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 04 21:53:44.764331 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751411    2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 04 21:53:44.764331 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751414    2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 04 21:53:44.764331 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751417    2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 04 21:53:44.764331 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751420    2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 04 21:53:44.764331 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751422    2557 feature_gate.go:328] unrecognized feature gate: Example
Sep 04 21:53:44.764331 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751425    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 04 21:53:44.764331 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751427    2557 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 04 21:53:44.764331 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751430    2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 04 21:53:44.764331 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751433    2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 04 21:53:44.764331 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751435    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 04 21:53:44.765145 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751438    2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 04 21:53:44.765145 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751440    2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 04 21:53:44.765145 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751443    2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 04 21:53:44.765145 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751445    2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 04 21:53:44.765145 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751448    2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 04 21:53:44.765145 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751450    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 04 21:53:44.765145 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751453    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 04 21:53:44.765145 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751455    2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 04 21:53:44.765145 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751458    2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 04 21:53:44.765145 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751461    2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 04 21:53:44.765145 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751463    2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 04 21:53:44.765145 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751466    2557 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 04 21:53:44.765145 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751468    2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 04 21:53:44.765145 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751471    2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 04 21:53:44.765145 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751474    2557 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 04 21:53:44.765145 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751476    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 04 21:53:44.765145 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751479    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 04 21:53:44.765145 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751481    2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 04 21:53:44.765145 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751486    2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 04 21:53:44.765145 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751490    2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 04 21:53:44.765884 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751494    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 04 21:53:44.765884 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751497    2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 04 21:53:44.765884 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751500    2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 04 21:53:44.765884 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751503    2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 04 21:53:44.765884 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751506    2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 04 21:53:44.765884 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751508    2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 04 21:53:44.765884 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751511    2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 04 21:53:44.765884 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751514    2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 04 21:53:44.765884 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751516    2557 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 04 21:53:44.765884 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751519    2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 04 21:53:44.765884 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751522    2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 04 21:53:44.765884 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751524    2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 04 21:53:44.765884 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751527    2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 04 21:53:44.765884 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751529    2557 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 04 21:53:44.765884 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751532    2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 04 21:53:44.765884 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751535    2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 04 21:53:44.765884 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751537    2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 04 21:53:44.765884 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751540    2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 04 21:53:44.765884 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751543    2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 04 21:53:44.766571 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751547    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 04 21:53:44.766571 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751550    2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 04 21:53:44.766571 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751552    2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 04 21:53:44.766571 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751556    2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 04 21:53:44.766571 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751559    2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 04 21:53:44.766571 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751561    2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 04 21:53:44.766571 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751564    2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 04 21:53:44.766571 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751566    2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 04 21:53:44.766571 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751569    2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 04 21:53:44.766571 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751572    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 04 21:53:44.766571 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751575    2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 04 21:53:44.766571 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751577    2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 04 21:53:44.766571 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751580    2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 04 21:53:44.766571 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751589    2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 04 21:53:44.766571 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751592    2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 04 21:53:44.766571 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751595    2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 04 21:53:44.766571 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751597    2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 04 21:53:44.766571 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:44.751600    2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 04 21:53:44.767244 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.751605    2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 04 21:53:44.767244 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.751753    2557 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 04 21:53:44.885885 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.885843    2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 04 21:53:44.887129 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.887115    2557 server.go:1019] "Starting client certificate rotation"
Sep 04 21:53:44.887246 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.887224    2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 04 21:53:44.887300 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.887269    2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 04 21:53:44.890147 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.890129    2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 04 21:53:44.891521 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.891492    2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 04 21:53:44.903193 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.903170    2557 log.go:25] "Validated CRI v1 runtime API"
Sep 04 21:53:44.907759 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.907732    2557 log.go:25] "Validated CRI v1 image API"
Sep 04 21:53:44.908744 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.908713    2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 04 21:53:44.915001 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.914972    2557 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 89bcec23-0c0c-46d9-866f-adf1e65c5d5e:/dev/nvme0n1p4 b7531fb8-cd5d-4edc-ae6c-60707b039b86:/dev/nvme0n1p3]
Sep 04 21:53:44.915116 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.914997    2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 04 21:53:44.921474 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.921322    2557 manager.go:216] Machine: {Timestamp:2026-09-04 21:53:44.919887399 +0000 UTC m=+0.488010493 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3072947 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d68af880042a76621dc74eda6ffdd SystemUUID:ec2d68af-8800-42a7-6621-dc74eda6ffdd BootID:747f7d8a-33b0-4231-b855-fe9219907c55 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a2:b2:b6:e8:95 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a2:b2:b6:e8:95 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d2:b9:6f:ce:3d:da Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 04 21:53:44.921474 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.921453    2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 04 21:53:44.921705 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.921593    2557 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 04 21:53:44.925353 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.925316    2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 04 21:53:44.925511 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.925354    2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-141.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 04 21:53:44.926532 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.926517    2557 topology_manager.go:138] "Creating topology manager with none policy"
Sep 04 21:53:44.926579 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.926534    2557 container_manager_linux.go:309] "Creating device plugin manager"
Sep 04 21:53:44.926579 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.926543    2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 04 21:53:44.927171 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.927159    2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 04 21:53:44.929028 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.929014    2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 04 21:53:44.929813 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.929793    2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 04 21:53:44.933723 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.933707    2557 state_mem.go:36] "Initialized new in-memory state store"
Sep 04 21:53:44.933849 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.933840    2557 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 04 21:53:44.933909 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.933882    2557 kubelet.go:486] "Attempting to sync node with API server"
Sep 04 21:53:44.933909 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.933892    2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 04 21:53:44.933983 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.933912    2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 04 21:53:44.933983 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.933923    2557 kubelet.go:398] "Adding apiserver pod source"
Sep 04 21:53:44.933983 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.933933    2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 04 21:53:44.934325 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.934313    2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 04 21:53:44.934357 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.934332    2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 04 21:53:44.936736 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.936716    2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 04 21:53:44.940037 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.940020    2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 04 21:53:44.940106 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.940051    2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 04 21:53:44.940242 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.940222    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 04 21:53:44.940281 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.940249    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 04 21:53:44.940281 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.940258    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 04 21:53:44.940281 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.940273    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 04 21:53:44.940281 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.940280    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 04 21:53:44.940388 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.940286    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 04 21:53:44.940918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.940908    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 04 21:53:44.940918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.940919    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 04 21:53:44.940983 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.940925    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 04 21:53:44.940983 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.940931    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 04 21:53:44.940983 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.940944    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 04 21:53:44.940983 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.940952    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 04 21:53:44.941773 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.941760    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 04 21:53:44.941811 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.941775    2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 04 21:53:44.943355 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.943320    2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-141.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 04 21:53:44.943355 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:44.943342    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-141.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 04 21:53:44.943483 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:44.943411    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 04 21:53:44.946386 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.946364    2557 server.go:1268] "Started kubelet"
Sep 04 21:53:44.946477 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.946414    2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 04 21:53:44.947093 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.947023    2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 04 21:53:44.947103 ip-10-0-134-141 systemd[1]: Started Kubernetes Kubelet.
Sep 04 21:53:44.951815 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.951782    2557 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 04 21:53:44.953489 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:44.953458    2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 04 21:53:44.953906 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.953889    2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 04 21:53:44.954569 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.954553    2557 server.go:310] "Adding debug handlers to kubelet server"
Sep 04 21:53:44.956338 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.956319    2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dhshr"
Sep 04 21:53:44.958145 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:44.957053    2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-141.ec2.internal.18d23cc66ccd2f67  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-141.ec2.internal,UID:ip-10-0-134-141.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-141.ec2.internal,},FirstTimestamp:2026-09-04 21:53:44.946085735 +0000 UTC m=+0.514208809,LastTimestamp:2026-09-04 21:53:44.946085735 +0000 UTC m=+0.514208809,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-141.ec2.internal,}"
Sep 04 21:53:44.959624 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.959607    2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 04 21:53:44.959697 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.959622    2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 04 21:53:44.959750 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:44.959710    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-141.ec2.internal\" not found"
Sep 04 21:53:44.960401 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.960383    2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 04 21:53:44.960494 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.960387    2557 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 04 21:53:44.960494 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.960421    2557 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 04 21:53:44.960599 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.960495    2557 reconstruct.go:97] "Volume reconstruction finished"
Sep 04 21:53:44.960599 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.960505    2557 reconciler.go:29] "Reconciler: start to sync state"
Sep 04 21:53:44.961700 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.961685    2557 factory.go:55] Registering systemd factory
Sep 04 21:53:44.961776 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.961705    2557 factory.go:223] Registration of the systemd container factory successfully
Sep 04 21:53:44.961989 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.961972    2557 factory.go:153] Registering CRI-O factory
Sep 04 21:53:44.961989 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.961988    2557 factory.go:223] Registration of the crio container factory successfully
Sep 04 21:53:44.962124 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.962035    2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 04 21:53:44.962124 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.962050    2557 factory.go:103] Registering Raw factory
Sep 04 21:53:44.962124 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.962060    2557 manager.go:1196] Started watching for new ooms in manager
Sep 04 21:53:44.963187 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.963168    2557 manager.go:349] Starting recovery of all containers
Sep 04 21:53:44.964556 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.964527    2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dhshr"
Sep 04 21:53:44.970436 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:44.970401    2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 04 21:53:44.970573 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:44.970453    2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-141.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 04 21:53:44.972692 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.972672    2557 manager.go:354] Recovery completed
Sep 04 21:53:44.976951 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.976933    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 21:53:44.978960 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.978943    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-141.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 21:53:44.979057 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.978972    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-141.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 21:53:44.979057 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.978982    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-141.ec2.internal" event="NodeHasSufficientPID"
Sep 04 21:53:44.980031 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.980018    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 21:53:44.980101 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.980018    2557 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 04 21:53:44.980101 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.980094    2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 04 21:53:44.980197 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.980117    2557 state_mem.go:36] "Initialized new in-memory state store"
Sep 04 21:53:44.980687 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.980672    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-141.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 21:53:44.980742 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.980700    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-141.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 21:53:44.980742 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.980711    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-141.ec2.internal" event="NodeHasSufficientPID"
Sep 04 21:53:44.982243 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.982196    2557 policy_none.go:49] "None policy: Start"
Sep 04 21:53:44.982243 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.982214    2557 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 04 21:53:44.982243 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.982227    2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 04 21:53:44.984231 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:44.984217    2557 policy_none.go:47] "Start"
Sep 04 21:53:45.040702 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:45.019538    2557 manager.go:337] "Starting Device Plugin manager"
Sep 04 21:53:45.040702 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:45.019587    2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 04 21:53:45.040702 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:45.019601    2557 server.go:87] "Starting device plugin registration server"
Sep 04 21:53:45.040702 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:45.019924    2557 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 04 21:53:45.040702 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:45.019937    2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 04 21:53:45.040702 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:45.020052    2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 04 21:53:45.040702 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:45.020156    2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 04 21:53:45.040702 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:45.020165    2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 04 21:53:45.040702 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:45.020833    2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 04 21:53:45.040702 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:45.020910    2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-141.ec2.internal\" not found"
Sep 04 21:53:45.093205 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:45.093159    2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 04 21:53:45.094460 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:45.094440    2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 04 21:53:45.094542 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:45.094514    2557 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 04 21:53:45.094542 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:45.094541    2557 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 04 21:53:45.094649 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:45.094590    2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 04 21:53:45.096601 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:45.096579    2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 21:53:45.120451 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:45.120415    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 21:53:45.121442 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:45.121422    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-141.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 21:53:45.121560 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:45.121461    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-141.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 21:53:45.121560 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:45.121476    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-141.ec2.internal" event="NodeHasSufficientPID"
Sep 04 21:53:45.121560 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:45.121506    2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-141.ec2.internal"
Sep 04 21:53:45.129295 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:45.129270    2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-141.ec2.internal"
Sep 04 21:53:45.129411 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:45.129301    2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-141.ec2.internal\": node \"ip-10-0-134-141.ec2.internal\" not found"
Sep 04 21:53:45.144003 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:45.143974    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-141.ec2.internal\" not found"
Sep 04 21:53:46.446615 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.446410    2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-141.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-141.ec2.internal"]
Sep 04 21:53:46.446726 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.446664    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 21:53:46.448391 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.448369    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-141.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 21:53:46.448494 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.448430    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-141.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 21:53:46.448494 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.448446    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-141.ec2.internal" event="NodeHasSufficientPID"
Sep 04 21:53:46.450677 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.450658    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 21:53:46.450791 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.450768    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-141.ec2.internal"
Sep 04 21:53:46.450843 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.450807    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 21:53:46.451615 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.451599    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-141.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 21:53:46.451682 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.451629    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-141.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 21:53:46.451682 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.451642    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-141.ec2.internal" event="NodeHasSufficientPID"
Sep 04 21:53:46.451682 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.451600    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-141.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 21:53:46.451682 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.451674    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-141.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 21:53:46.451838 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.451712    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-141.ec2.internal" event="NodeHasSufficientPID"
Sep 04 21:53:46.453920 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.453903    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-141.ec2.internal"
Sep 04 21:53:46.453997 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.453941    2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 04 21:53:46.454777 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.454762    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-141.ec2.internal" event="NodeHasSufficientMemory"
Sep 04 21:53:46.454877 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.454790    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-141.ec2.internal" event="NodeHasNoDiskPressure"
Sep 04 21:53:46.454877 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.454803    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-141.ec2.internal" event="NodeHasSufficientPID"
Sep 04 21:53:46.476366 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:46.476334    2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-141.ec2.internal\" not found" node="ip-10-0-134-141.ec2.internal"
Sep 04 21:53:46.480931 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:46.480908    2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-141.ec2.internal\" not found" node="ip-10-0-134-141.ec2.internal"
Sep 04 21:53:46.496439 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:46.496355    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-141.ec2.internal\" not found"
Sep 04 21:53:46.513020 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.512989    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9b84a1cc7bbade60d3a69ba540f3e588-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-141.ec2.internal\" (UID: \"9b84a1cc7bbade60d3a69ba540f3e588\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-141.ec2.internal"
Sep 04 21:53:46.513113 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.513032    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b84a1cc7bbade60d3a69ba540f3e588-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-141.ec2.internal\" (UID: \"9b84a1cc7bbade60d3a69ba540f3e588\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-141.ec2.internal"
Sep 04 21:53:46.513113 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.513058    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/79b44ba13a4b8fc8cb712cd9da0778c3-config\") pod \"kube-apiserver-proxy-ip-10-0-134-141.ec2.internal\" (UID: \"79b44ba13a4b8fc8cb712cd9da0778c3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-141.ec2.internal"
Sep 04 21:53:46.596807 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:46.596779    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-141.ec2.internal\" not found"
Sep 04 21:53:46.614175 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.614138    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9b84a1cc7bbade60d3a69ba540f3e588-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-141.ec2.internal\" (UID: \"9b84a1cc7bbade60d3a69ba540f3e588\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-141.ec2.internal"
Sep 04 21:53:46.614284 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.614183    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b84a1cc7bbade60d3a69ba540f3e588-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-141.ec2.internal\" (UID: \"9b84a1cc7bbade60d3a69ba540f3e588\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-141.ec2.internal"
Sep 04 21:53:46.614284 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.614207    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/79b44ba13a4b8fc8cb712cd9da0778c3-config\") pod \"kube-apiserver-proxy-ip-10-0-134-141.ec2.internal\" (UID: \"79b44ba13a4b8fc8cb712cd9da0778c3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-141.ec2.internal"
Sep 04 21:53:46.614284 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.614250    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/79b44ba13a4b8fc8cb712cd9da0778c3-config\") pod \"kube-apiserver-proxy-ip-10-0-134-141.ec2.internal\" (UID: \"79b44ba13a4b8fc8cb712cd9da0778c3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-141.ec2.internal"
Sep 04 21:53:46.614284 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.614250    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9b84a1cc7bbade60d3a69ba540f3e588-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-141.ec2.internal\" (UID: \"9b84a1cc7bbade60d3a69ba540f3e588\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-141.ec2.internal"
Sep 04 21:53:46.614446 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.614250    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9b84a1cc7bbade60d3a69ba540f3e588-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-141.ec2.internal\" (UID: \"9b84a1cc7bbade60d3a69ba540f3e588\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-141.ec2.internal"
Sep 04 21:53:46.697626 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:46.697590    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-141.ec2.internal\" not found"
Sep 04 21:53:46.779198 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.779130    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-141.ec2.internal"
Sep 04 21:53:46.784378 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:46.783929    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-141.ec2.internal"
Sep 04 21:53:46.798717 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:46.798686    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-141.ec2.internal\" not found"
Sep 04 21:53:46.899295 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:46.899257    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-141.ec2.internal\" not found"
Sep 04 21:53:46.999901 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:46.999871    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-141.ec2.internal\" not found"
Sep 04 21:53:47.100811 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:47.100720    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-141.ec2.internal\" not found"
Sep 04 21:53:47.139323 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:47.139287    2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 04 21:53:47.139524 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:47.139504    2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 04 21:53:47.201234 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:47.201203    2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-141.ec2.internal\" not found"
Sep 04 21:53:47.220993 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:47.220938    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-03 21:48:46 +0000 UTC" deadline="2028-02-07 12:27:51.779380792 +0000 UTC"
Sep 04 21:53:47.220993 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:47.220984    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12494h34m4.558400753s"
Sep 04 21:53:47.245659 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:47.245629    2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 21:53:47.285786 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:47.285748    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b84a1cc7bbade60d3a69ba540f3e588.slice/crio-34a8838bdf72580958373d96e59850eb56a978d26549249fe8e0cbc7f0fc4adf WatchSource:0}: Error finding container 34a8838bdf72580958373d96e59850eb56a978d26549249fe8e0cbc7f0fc4adf: Status 404 returned error can't find the container with id 34a8838bdf72580958373d96e59850eb56a978d26549249fe8e0cbc7f0fc4adf
Sep 04 21:53:47.286190 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:47.286177    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79b44ba13a4b8fc8cb712cd9da0778c3.slice/crio-701e0c8ed50382e8802ba2e8dc54c3a5f07f9ef4407137f45622435f726e0f81 WatchSource:0}: Error finding container 701e0c8ed50382e8802ba2e8dc54c3a5f07f9ef4407137f45622435f726e0f81: Status 404 returned error can't find the container with id 701e0c8ed50382e8802ba2e8dc54c3a5f07f9ef4407137f45622435f726e0f81
Sep 04 21:53:47.290981 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:47.290963    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 04 21:53:47.311790 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:47.311761    2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-141.ec2.internal"
Sep 04 21:53:47.324272 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:47.324250    2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 04 21:53:47.325693 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:47.325674    2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-141.ec2.internal"
Sep 04 21:53:47.333706 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:47.333675    2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 04 21:53:47.350140 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:47.350098    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-141.ec2.internal" event={"ID":"9b84a1cc7bbade60d3a69ba540f3e588","Type":"ContainerStarted","Data":"34a8838bdf72580958373d96e59850eb56a978d26549249fe8e0cbc7f0fc4adf"}
Sep 04 21:53:47.351200 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:47.351140    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-141.ec2.internal" event={"ID":"79b44ba13a4b8fc8cb712cd9da0778c3","Type":"ContainerStarted","Data":"701e0c8ed50382e8802ba2e8dc54c3a5f07f9ef4407137f45622435f726e0f81"}
Sep 04 21:53:47.364669 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:47.364648    2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 21:53:47.711804 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:47.711717    2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 21:53:48.187139 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.187045    2557 apiserver.go:52] "Watching apiserver"
Sep 04 21:53:48.194982 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.194953    2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 04 21:53:48.196874 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.196814    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-46fbr","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw","openshift-cluster-node-tuning-operator/tuned-ft99b","openshift-dns/node-resolver-tr8wn","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-141.ec2.internal","openshift-multus/multus-9nl4m","openshift-network-diagnostics/network-check-target-dflhd","openshift-network-operator/iptables-alerter-r22zb","openshift-ovn-kubernetes/ovnkube-node-ss72b","kube-system/global-pull-secret-syncer-p7b77","kube-system/konnectivity-agent-xcgjm","kube-system/kube-apiserver-proxy-ip-10-0-134-141.ec2.internal","openshift-image-registry/node-ca-c6lfc","openshift-multus/multus-additional-cni-plugins-dzjmv"]
Sep 04 21:53:48.199327 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.199303    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:53:48.199475 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:48.199402    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:53:48.201349 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.201320    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xcgjm"
Sep 04 21:53:48.203636 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.203612    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.203865 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.203762    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gkdzt\""
Sep 04 21:53:48.204212 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.204080    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 04 21:53:48.204212 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.204106    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 04 21:53:48.205944 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.205924    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tr8wn"
Sep 04 21:53:48.207187 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.207165    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 04 21:53:48.207271 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.207165    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 04 21:53:48.207271 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.207219    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vhvnl\""
Sep 04 21:53:48.208253 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.208233    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dzjmv"
Sep 04 21:53:48.209668 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.209504    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 04 21:53:48.209913 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.209894    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hl8jm\""
Sep 04 21:53:48.210528 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.210507    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.211693 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.211665    2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 04 21:53:48.213022 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.213005    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 04 21:53:48.213292 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.213276    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 04 21:53:48.213469 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.213442    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jgjll\""
Sep 04 21:53:48.213537 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.213520    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 04 21:53:48.213593 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.213575    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 04 21:53:48.213650 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.213633    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 04 21:53:48.213698 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.213657    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 04 21:53:48.213906 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.213880    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xjxsh\""
Sep 04 21:53:48.214081 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.214062    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 04 21:53:48.214247 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.214191    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.217019 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.216605    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-c6lfc"
Sep 04 21:53:48.217019 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.216820    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x75w5\""
Sep 04 21:53:48.218211 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.217702    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 04 21:53:48.218211 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.217804    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 04 21:53:48.218211 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.217970    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 04 21:53:48.218211 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.218021    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 04 21:53:48.218211 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.218098    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 04 21:53:48.218211 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.218134    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 04 21:53:48.219542 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.219526    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 04 21:53:48.219680 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.219660    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw"
Sep 04 21:53:48.220459 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.220436    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-slr9c\""
Sep 04 21:53:48.220563 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.220469    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 04 21:53:48.220563 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.220494    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 04 21:53:48.222134 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.222096    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:53:48.222231 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:48.222214    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:53:48.223570 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.222431    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90-cnibin\") pod \"multus-additional-cni-plugins-dzjmv\" (UID: \"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90\") " pod="openshift-multus/multus-additional-cni-plugins-dzjmv"
Sep 04 21:53:48.223570 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.222535    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 04 21:53:48.223570 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.222604    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-host-cni-bin\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.223570 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.222643    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.223570 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.222680    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-hostroot\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.223570 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.222710    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-multus-daemon-config\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.223570 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.222785    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-host-run-k8s-cni-cncf-io\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.223570 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.222818    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b36d1fd6-09dc-43b7-b179-051cd03a518f-hosts-file\") pod \"node-resolver-tr8wn\" (UID: \"b36d1fd6-09dc-43b7-b179-051cd03a518f\") " pod="openshift-dns/node-resolver-tr8wn"
Sep 04 21:53:48.223570 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.222874    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/db97b042-b391-4f31-8f0f-3deceacb5157-konnectivity-ca\") pod \"konnectivity-agent-xcgjm\" (UID: \"db97b042-b391-4f31-8f0f-3deceacb5157\") " pod="kube-system/konnectivity-agent-xcgjm"
Sep 04 21:53:48.223570 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.222933    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dzjmv\" (UID: \"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90\") " pod="openshift-multus/multus-additional-cni-plugins-dzjmv"
Sep 04 21:53:48.223570 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.222952    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 04 21:53:48.223570 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.223036    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dzjmv\" (UID: \"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90\") " pod="openshift-multus/multus-additional-cni-plugins-dzjmv"
Sep 04 21:53:48.223570 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.223067    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-log-socket\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.223570 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.223098    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-etc-sysctl-d\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.223570 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.223130    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-sys\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.223570 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.223166    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-j5kcv\""
Sep 04 21:53:48.223570 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.222818    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 04 21:53:48.223570 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.223159    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-tmp\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.224448 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.223322    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-system-cni-dir\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.224448 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.223344    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-systemd-units\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.224448 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.223372    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/db97b042-b391-4f31-8f0f-3deceacb5157-agent-certs\") pod \"konnectivity-agent-xcgjm\" (UID: \"db97b042-b391-4f31-8f0f-3deceacb5157\") " pod="kube-system/konnectivity-agent-xcgjm"
Sep 04 21:53:48.224448 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.223440    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b36d1fd6-09dc-43b7-b179-051cd03a518f-tmp-dir\") pod \"node-resolver-tr8wn\" (UID: \"b36d1fd6-09dc-43b7-b179-051cd03a518f\") " pod="openshift-dns/node-resolver-tr8wn"
Sep 04 21:53:48.224448 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.223483    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-cnibin\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.224448 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.223592    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5d2dc985-0ea0-47ff-88d2-600e4e46a436-ovnkube-script-lib\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.224448 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.223651    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-etc-kubernetes\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.224448 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.223705    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-host-run-netns\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.224448 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.223747    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5d2dc985-0ea0-47ff-88d2-600e4e46a436-env-overrides\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.224448 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.223783    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-etc-modprobe-d\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.224448 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.223833    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-etc-systemd\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.224448 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.223871    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-host-run-netns\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.224448 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.223904    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kxzt\" (UniqueName: \"kubernetes.io/projected/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-kube-api-access-9kxzt\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.224448 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.223936    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-node-log\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.224448 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.223991    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-lib-modules\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.224448 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.224023    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-var-lib-kubelet\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.224448 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.224056    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhjmp\" (UniqueName: \"kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp\") pod \"network-check-target-dflhd\" (UID: \"10da7e37-b753-4b5f-93f5-07e7a48ba85b\") " pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:53:48.225140 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.224091    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-host-run-multus-certs\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.225140 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.224126    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-etc-kubernetes\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.225140 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.224152    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-host-cni-netd\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.225140 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.224252    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrj6r\" (UniqueName: \"kubernetes.io/projected/5d2dc985-0ea0-47ff-88d2-600e4e46a436-kube-api-access-wrj6r\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.225140 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.224286    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-multus-cni-dir\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.225140 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.224354    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-host-var-lib-cni-multus\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.225140 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.224392    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-host-var-lib-kubelet\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.225140 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.224507    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-host-kubelet\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.225140 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.224596    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-host-run-ovn-kubernetes\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.225140 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.224719    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88wmk\" (UniqueName: \"kubernetes.io/projected/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-kube-api-access-88wmk\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.225140 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.224761    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-multus-conf-dir\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.225140 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.224814    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k9k5\" (UniqueName: \"kubernetes.io/projected/bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90-kube-api-access-2k9k5\") pod \"multus-additional-cni-plugins-dzjmv\" (UID: \"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90\") " pod="openshift-multus/multus-additional-cni-plugins-dzjmv"
Sep 04 21:53:48.225140 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.224869    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-run-openvswitch\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.225140 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.224900    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5d2dc985-0ea0-47ff-88d2-600e4e46a436-ovnkube-config\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.225140 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.224952    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-etc-sysctl-conf\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.225140 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.224988    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-cni-binary-copy\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.225933 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.225016    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90-cni-binary-copy\") pod \"multus-additional-cni-plugins-dzjmv\" (UID: \"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90\") " pod="openshift-multus/multus-additional-cni-plugins-dzjmv"
Sep 04 21:53:48.225933 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.225071    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-etc-sysconfig\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.225933 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.225117    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-etc-tuned\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.225933 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.225201    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7mdc\" (UniqueName: \"kubernetes.io/projected/b36d1fd6-09dc-43b7-b179-051cd03a518f-kube-api-access-s7mdc\") pod \"node-resolver-tr8wn\" (UID: \"b36d1fd6-09dc-43b7-b179-051cd03a518f\") " pod="openshift-dns/node-resolver-tr8wn"
Sep 04 21:53:48.225933 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.225239    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-run-systemd\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.225933 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.225284    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-etc-openvswitch\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.225933 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.225331    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-run-ovn\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.225933 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.225354    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5d2dc985-0ea0-47ff-88d2-600e4e46a436-ovn-node-metrics-cert\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.225933 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.225388    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-run\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.225933 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.225546    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:53:48.225933 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.225456    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-host\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.225933 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:48.225667    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:53:48.225933 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.225706    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-multus-socket-dir-parent\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.225933 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.225740    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-host-var-lib-cni-bin\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.225933 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.225772    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90-system-cni-dir\") pod \"multus-additional-cni-plugins-dzjmv\" (UID: \"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90\") " pod="openshift-multus/multus-additional-cni-plugins-dzjmv"
Sep 04 21:53:48.225933 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.225802    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dzjmv\" (UID: \"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90\") " pod="openshift-multus/multus-additional-cni-plugins-dzjmv"
Sep 04 21:53:48.225933 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.225828    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-host-slash\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.226455 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.225857    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-var-lib-openvswitch\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.228362 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.228290    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r22zb"
Sep 04 21:53:48.228790 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.228769    2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 04 21:53:48.231141 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.231125    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 04 21:53:48.231239 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.231210    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qr8nv\""
Sep 04 21:53:48.231328 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.231307    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 04 21:53:48.231388 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.231347    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 04 21:53:48.260687 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.260658    2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mlrh2"
Sep 04 21:53:48.274476 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.274449    2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mlrh2"
Sep 04 21:53:48.312916 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.312888    2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 04 21:53:48.326448 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326401    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1daa15e7-a0f6-483d-b923-d1ded7222d70-socket-dir\") pod \"aws-ebs-csi-driver-node-457zw\" (UID: \"1daa15e7-a0f6-483d-b923-d1ded7222d70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw"
Sep 04 21:53:48.326634 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326495    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-multus-socket-dir-parent\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.326634 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326532    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8fcc2326-c662-4e67-bdaf-60a54cce4235-kubelet-config\") pod \"global-pull-secret-syncer-p7b77\" (UID: \"8fcc2326-c662-4e67-bdaf-60a54cce4235\") " pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:53:48.326634 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326557    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/29eb21bd-c9b9-4206-8029-86c0d353d68d-serviceca\") pod \"node-ca-c6lfc\" (UID: \"29eb21bd-c9b9-4206-8029-86c0d353d68d\") " pod="openshift-image-registry/node-ca-c6lfc"
Sep 04 21:53:48.326634 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326580    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmn8x\" (UniqueName: \"kubernetes.io/projected/eadd93c2-4a94-4046-9924-6e342406d1f8-kube-api-access-kmn8x\") pod \"network-metrics-daemon-46fbr\" (UID: \"eadd93c2-4a94-4046-9924-6e342406d1f8\") " pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:53:48.326634 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326605    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-host-var-lib-cni-bin\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.326634 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326631    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90-system-cni-dir\") pod \"multus-additional-cni-plugins-dzjmv\" (UID: \"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90\") " pod="openshift-multus/multus-additional-cni-plugins-dzjmv"
Sep 04 21:53:48.326938 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326658    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dzjmv\" (UID: \"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90\") " pod="openshift-multus/multus-additional-cni-plugins-dzjmv"
Sep 04 21:53:48.326938 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326683    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-host-slash\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.326938 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326709    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-var-lib-openvswitch\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.326938 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326702    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-multus-socket-dir-parent\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.326938 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326706    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90-system-cni-dir\") pod \"multus-additional-cni-plugins-dzjmv\" (UID: \"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90\") " pod="openshift-multus/multus-additional-cni-plugins-dzjmv"
Sep 04 21:53:48.326938 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326738    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90-cnibin\") pod \"multus-additional-cni-plugins-dzjmv\" (UID: \"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90\") " pod="openshift-multus/multus-additional-cni-plugins-dzjmv"
Sep 04 21:53:48.326938 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326769    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-host-slash\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.326938 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326779    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-host-cni-bin\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.326938 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326784    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90-cnibin\") pod \"multus-additional-cni-plugins-dzjmv\" (UID: \"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90\") " pod="openshift-multus/multus-additional-cni-plugins-dzjmv"
Sep 04 21:53:48.326938 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326770    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-host-var-lib-cni-bin\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.326938 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326781    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-var-lib-openvswitch\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.326938 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326815    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.326938 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326831    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-host-cni-bin\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.326938 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326854    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-hostroot\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.326938 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326883    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-multus-daemon-config\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.326938 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326909    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-host-run-k8s-cni-cncf-io\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.326938 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326887    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-hostroot\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.327739 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326856    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.327739 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326937    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1daa15e7-a0f6-483d-b923-d1ded7222d70-device-dir\") pod \"aws-ebs-csi-driver-node-457zw\" (UID: \"1daa15e7-a0f6-483d-b923-d1ded7222d70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw"
Sep 04 21:53:48.327739 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326958    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-host-run-k8s-cni-cncf-io\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.327739 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326959    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1daa15e7-a0f6-483d-b923-d1ded7222d70-sys-fs\") pod \"aws-ebs-csi-driver-node-457zw\" (UID: \"1daa15e7-a0f6-483d-b923-d1ded7222d70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw"
Sep 04 21:53:48.327739 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.326987    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret\") pod \"global-pull-secret-syncer-p7b77\" (UID: \"8fcc2326-c662-4e67-bdaf-60a54cce4235\") " pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:53:48.327739 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327011    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8b91c97-3cb9-421e-8a7b-6da194236723-host-slash\") pod \"iptables-alerter-r22zb\" (UID: \"f8b91c97-3cb9-421e-8a7b-6da194236723\") " pod="openshift-network-operator/iptables-alerter-r22zb"
Sep 04 21:53:48.327739 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327057    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b36d1fd6-09dc-43b7-b179-051cd03a518f-hosts-file\") pod \"node-resolver-tr8wn\" (UID: \"b36d1fd6-09dc-43b7-b179-051cd03a518f\") " pod="openshift-dns/node-resolver-tr8wn"
Sep 04 21:53:48.327739 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327080    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/db97b042-b391-4f31-8f0f-3deceacb5157-konnectivity-ca\") pod \"konnectivity-agent-xcgjm\" (UID: \"db97b042-b391-4f31-8f0f-3deceacb5157\") " pod="kube-system/konnectivity-agent-xcgjm"
Sep 04 21:53:48.327739 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327101    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29eb21bd-c9b9-4206-8029-86c0d353d68d-host\") pod \"node-ca-c6lfc\" (UID: \"29eb21bd-c9b9-4206-8029-86c0d353d68d\") " pod="openshift-image-registry/node-ca-c6lfc"
Sep 04 21:53:48.327739 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327143    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dzjmv\" (UID: \"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90\") " pod="openshift-multus/multus-additional-cni-plugins-dzjmv"
Sep 04 21:53:48.327739 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327171    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dzjmv\" (UID: \"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90\") " pod="openshift-multus/multus-additional-cni-plugins-dzjmv"
Sep 04 21:53:48.327739 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327201    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b36d1fd6-09dc-43b7-b179-051cd03a518f-hosts-file\") pod \"node-resolver-tr8wn\" (UID: \"b36d1fd6-09dc-43b7-b179-051cd03a518f\") " pod="openshift-dns/node-resolver-tr8wn"
Sep 04 21:53:48.327739 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327198    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-log-socket\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.327739 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327239    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-etc-sysctl-d\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.327739 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327274    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-sys\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.327739 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327292    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-tmp\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.327739 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327309    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jls52\" (UniqueName: \"kubernetes.io/projected/1daa15e7-a0f6-483d-b923-d1ded7222d70-kube-api-access-jls52\") pod \"aws-ebs-csi-driver-node-457zw\" (UID: \"1daa15e7-a0f6-483d-b923-d1ded7222d70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw"
Sep 04 21:53:48.328487 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327328    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-system-cni-dir\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.328487 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327335    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dzjmv\" (UID: \"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90\") " pod="openshift-multus/multus-additional-cni-plugins-dzjmv"
Sep 04 21:53:48.328487 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327348    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-systemd-units\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.328487 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327368    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-systemd-units\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.328487 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327392    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-log-socket\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.328487 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327391    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7nz2\" (UniqueName: \"kubernetes.io/projected/f8b91c97-3cb9-421e-8a7b-6da194236723-kube-api-access-v7nz2\") pod \"iptables-alerter-r22zb\" (UID: \"f8b91c97-3cb9-421e-8a7b-6da194236723\") " pod="openshift-network-operator/iptables-alerter-r22zb"
Sep 04 21:53:48.328487 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327440    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/db97b042-b391-4f31-8f0f-3deceacb5157-agent-certs\") pod \"konnectivity-agent-xcgjm\" (UID: \"db97b042-b391-4f31-8f0f-3deceacb5157\") " pod="kube-system/konnectivity-agent-xcgjm"
Sep 04 21:53:48.328487 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327469    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b36d1fd6-09dc-43b7-b179-051cd03a518f-tmp-dir\") pod \"node-resolver-tr8wn\" (UID: \"b36d1fd6-09dc-43b7-b179-051cd03a518f\") " pod="openshift-dns/node-resolver-tr8wn"
Sep 04 21:53:48.328487 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327470    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dzjmv\" (UID: \"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90\") " pod="openshift-multus/multus-additional-cni-plugins-dzjmv"
Sep 04 21:53:48.328487 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327483    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-multus-daemon-config\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.328487 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327494    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-cnibin\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.328487 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327506    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-etc-sysctl-d\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.328487 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327520    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5d2dc985-0ea0-47ff-88d2-600e4e46a436-ovnkube-script-lib\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.328487 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327540    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-sys\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.328487 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327556    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-etc-kubernetes\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.328487 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327566    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-system-cni-dir\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.328487 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327587    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f8b91c97-3cb9-421e-8a7b-6da194236723-iptables-alerter-script\") pod \"iptables-alerter-r22zb\" (UID: \"f8b91c97-3cb9-421e-8a7b-6da194236723\") " pod="openshift-network-operator/iptables-alerter-r22zb"
Sep 04 21:53:48.329313 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327614    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-host-run-netns\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.329313 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327641    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5d2dc985-0ea0-47ff-88d2-600e4e46a436-env-overrides\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.329313 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327664    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-etc-modprobe-d\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.329313 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327689    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-etc-systemd\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.329313 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327730    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/db97b042-b391-4f31-8f0f-3deceacb5157-konnectivity-ca\") pod \"konnectivity-agent-xcgjm\" (UID: \"db97b042-b391-4f31-8f0f-3deceacb5157\") " pod="kube-system/konnectivity-agent-xcgjm"
Sep 04 21:53:48.329313 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327729    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-host-run-netns\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.329313 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327785    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-host-run-netns\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.329313 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327789    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dzjmv\" (UID: \"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90\") " pod="openshift-multus/multus-additional-cni-plugins-dzjmv"
Sep 04 21:53:48.329313 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327800    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-etc-kubernetes\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.329313 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327733    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-host-run-netns\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.329313 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327800    2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 04 21:53:48.329313 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327840    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kxzt\" (UniqueName: \"kubernetes.io/projected/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-kube-api-access-9kxzt\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.329313 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327839    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-etc-modprobe-d\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.329313 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327863    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-etc-systemd\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.329313 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327961    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-node-log\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.329313 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327987    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-lib-modules\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.329313 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.327975    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-cnibin\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.329313 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328040    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-node-log\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.330091 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328073    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-var-lib-kubelet\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.330091 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328079    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-lib-modules\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.330091 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328094    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b36d1fd6-09dc-43b7-b179-051cd03a518f-tmp-dir\") pod \"node-resolver-tr8wn\" (UID: \"b36d1fd6-09dc-43b7-b179-051cd03a518f\") " pod="openshift-dns/node-resolver-tr8wn"
Sep 04 21:53:48.330091 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328132    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5d2dc985-0ea0-47ff-88d2-600e4e46a436-ovnkube-script-lib\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.330091 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328115    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1daa15e7-a0f6-483d-b923-d1ded7222d70-kubelet-dir\") pod \"aws-ebs-csi-driver-node-457zw\" (UID: \"1daa15e7-a0f6-483d-b923-d1ded7222d70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw"
Sep 04 21:53:48.330091 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328130    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-var-lib-kubelet\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.330091 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328175    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8fcc2326-c662-4e67-bdaf-60a54cce4235-dbus\") pod \"global-pull-secret-syncer-p7b77\" (UID: \"8fcc2326-c662-4e67-bdaf-60a54cce4235\") " pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:53:48.330091 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328203    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhjmp\" (UniqueName: \"kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp\") pod \"network-check-target-dflhd\" (UID: \"10da7e37-b753-4b5f-93f5-07e7a48ba85b\") " pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:53:48.330091 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328265    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-host-run-multus-certs\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.330091 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328291    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-etc-kubernetes\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.330091 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328316    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-host-cni-netd\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.330091 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328353    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrj6r\" (UniqueName: \"kubernetes.io/projected/5d2dc985-0ea0-47ff-88d2-600e4e46a436-kube-api-access-wrj6r\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.330091 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328375    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-multus-cni-dir\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.330091 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328377    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-etc-kubernetes\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.330091 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328388    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-host-cni-netd\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.330091 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328339    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-host-run-multus-certs\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.330091 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328391    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-host-var-lib-cni-multus\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.330880 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328455    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-host-var-lib-cni-multus\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.330880 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328499    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-host-var-lib-kubelet\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.330880 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328541    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-host-kubelet\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.330880 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328546    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-host-var-lib-kubelet\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.330880 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328565    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-host-run-ovn-kubernetes\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.330880 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328591    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-multus-cni-dir\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.330880 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328605    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-host-kubelet\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.330880 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328611    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-host-run-ovn-kubernetes\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.330880 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328627    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88wmk\" (UniqueName: \"kubernetes.io/projected/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-kube-api-access-88wmk\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.330880 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328657    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1daa15e7-a0f6-483d-b923-d1ded7222d70-etc-selinux\") pod \"aws-ebs-csi-driver-node-457zw\" (UID: \"1daa15e7-a0f6-483d-b923-d1ded7222d70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw"
Sep 04 21:53:48.330880 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328707    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-multus-conf-dir\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.330880 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328737    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k9k5\" (UniqueName: \"kubernetes.io/projected/bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90-kube-api-access-2k9k5\") pod \"multus-additional-cni-plugins-dzjmv\" (UID: \"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90\") " pod="openshift-multus/multus-additional-cni-plugins-dzjmv"
Sep 04 21:53:48.330880 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328761    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-run-openvswitch\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.330880 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328787    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5d2dc985-0ea0-47ff-88d2-600e4e46a436-ovnkube-config\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.330880 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328789    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-multus-conf-dir\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.330880 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328811    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-etc-sysctl-conf\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.330880 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328836    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-cni-binary-copy\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.331530 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328839    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-run-openvswitch\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.331530 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328860    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90-cni-binary-copy\") pod \"multus-additional-cni-plugins-dzjmv\" (UID: \"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90\") " pod="openshift-multus/multus-additional-cni-plugins-dzjmv"
Sep 04 21:53:48.331530 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328886    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1daa15e7-a0f6-483d-b923-d1ded7222d70-registration-dir\") pod \"aws-ebs-csi-driver-node-457zw\" (UID: \"1daa15e7-a0f6-483d-b923-d1ded7222d70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw"
Sep 04 21:53:48.331530 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328902    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5d2dc985-0ea0-47ff-88d2-600e4e46a436-env-overrides\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.331530 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328909    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zm7nd\" (UniqueName: \"kubernetes.io/projected/29eb21bd-c9b9-4206-8029-86c0d353d68d-kube-api-access-zm7nd\") pod \"node-ca-c6lfc\" (UID: \"29eb21bd-c9b9-4206-8029-86c0d353d68d\") " pod="openshift-image-registry/node-ca-c6lfc"
Sep 04 21:53:48.331530 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328948    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs\") pod \"network-metrics-daemon-46fbr\" (UID: \"eadd93c2-4a94-4046-9924-6e342406d1f8\") " pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:53:48.331530 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.328980    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-etc-sysconfig\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.331530 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.329006    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-etc-tuned\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.331530 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.329031    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7mdc\" (UniqueName: \"kubernetes.io/projected/b36d1fd6-09dc-43b7-b179-051cd03a518f-kube-api-access-s7mdc\") pod \"node-resolver-tr8wn\" (UID: \"b36d1fd6-09dc-43b7-b179-051cd03a518f\") " pod="openshift-dns/node-resolver-tr8wn"
Sep 04 21:53:48.331530 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.329058    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-run-systemd\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.331530 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.329075    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-etc-sysconfig\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.331530 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.329083    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-etc-openvswitch\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.331530 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.329108    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-run-ovn\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.331530 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.329158    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5d2dc985-0ea0-47ff-88d2-600e4e46a436-ovn-node-metrics-cert\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.331530 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.329182    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-run\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.331530 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.329203    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-host\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.331530 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.329287    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-host\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.332352 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.329319    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-etc-openvswitch\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.332352 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.329332    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-run-ovn\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.332352 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.329408    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-cni-binary-copy\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.332352 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.329494    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90-cni-binary-copy\") pod \"multus-additional-cni-plugins-dzjmv\" (UID: \"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90\") " pod="openshift-multus/multus-additional-cni-plugins-dzjmv"
Sep 04 21:53:48.332352 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.329572    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-etc-sysctl-conf\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.332352 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.329579    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5d2dc985-0ea0-47ff-88d2-600e4e46a436-ovnkube-config\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.332352 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.329648    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5d2dc985-0ea0-47ff-88d2-600e4e46a436-run-systemd\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.332352 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.329689    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-run\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.332352 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.332072    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5d2dc985-0ea0-47ff-88d2-600e4e46a436-ovn-node-metrics-cert\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.332352 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.332195    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/db97b042-b391-4f31-8f0f-3deceacb5157-agent-certs\") pod \"konnectivity-agent-xcgjm\" (UID: \"db97b042-b391-4f31-8f0f-3deceacb5157\") " pod="kube-system/konnectivity-agent-xcgjm"
Sep 04 21:53:48.332864 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.332364    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-tmp\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.332920 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.332878    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-etc-tuned\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.339505 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:48.339474    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 21:53:48.339505 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:48.339503    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 21:53:48.339505 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:48.339517    2557 projected.go:196] Error preparing data for projected volume kube-api-access-fhjmp for pod openshift-network-diagnostics/network-check-target-dflhd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:53:48.339775 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:48.339605    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp podName:10da7e37-b753-4b5f-93f5-07e7a48ba85b nodeName:}" failed. No retries permitted until 2026-09-04 21:53:48.83957093 +0000 UTC m=+3.156137666 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fhjmp" (UniqueName: "kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp") pod "network-check-target-dflhd" (UID: "10da7e37-b753-4b5f-93f5-07e7a48ba85b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:53:48.342918 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.342890    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88wmk\" (UniqueName: \"kubernetes.io/projected/5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6-kube-api-access-88wmk\") pod \"tuned-ft99b\" (UID: \"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6\") " pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.343038 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.342907    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kxzt\" (UniqueName: \"kubernetes.io/projected/fb5aaba9-9cb0-4fa7-a342-344f1e40cad8-kube-api-access-9kxzt\") pod \"multus-9nl4m\" (UID: \"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8\") " pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.343038 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.342893    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrj6r\" (UniqueName: \"kubernetes.io/projected/5d2dc985-0ea0-47ff-88d2-600e4e46a436-kube-api-access-wrj6r\") pod \"ovnkube-node-ss72b\" (UID: \"5d2dc985-0ea0-47ff-88d2-600e4e46a436\") " pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.343351 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.343323    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7mdc\" (UniqueName: \"kubernetes.io/projected/b36d1fd6-09dc-43b7-b179-051cd03a518f-kube-api-access-s7mdc\") pod \"node-resolver-tr8wn\" (UID: \"b36d1fd6-09dc-43b7-b179-051cd03a518f\") " pod="openshift-dns/node-resolver-tr8wn"
Sep 04 21:53:48.343474 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.343456    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k9k5\" (UniqueName: \"kubernetes.io/projected/bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90-kube-api-access-2k9k5\") pod \"multus-additional-cni-plugins-dzjmv\" (UID: \"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90\") " pod="openshift-multus/multus-additional-cni-plugins-dzjmv"
Sep 04 21:53:48.410766 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.410727    2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 04 21:53:48.430173 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430137    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1daa15e7-a0f6-483d-b923-d1ded7222d70-device-dir\") pod \"aws-ebs-csi-driver-node-457zw\" (UID: \"1daa15e7-a0f6-483d-b923-d1ded7222d70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw"
Sep 04 21:53:48.430335 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430183    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1daa15e7-a0f6-483d-b923-d1ded7222d70-sys-fs\") pod \"aws-ebs-csi-driver-node-457zw\" (UID: \"1daa15e7-a0f6-483d-b923-d1ded7222d70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw"
Sep 04 21:53:48.430335 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430203    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret\") pod \"global-pull-secret-syncer-p7b77\" (UID: \"8fcc2326-c662-4e67-bdaf-60a54cce4235\") " pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:53:48.430335 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430218    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8b91c97-3cb9-421e-8a7b-6da194236723-host-slash\") pod \"iptables-alerter-r22zb\" (UID: \"f8b91c97-3cb9-421e-8a7b-6da194236723\") " pod="openshift-network-operator/iptables-alerter-r22zb"
Sep 04 21:53:48.430335 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430235    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29eb21bd-c9b9-4206-8029-86c0d353d68d-host\") pod \"node-ca-c6lfc\" (UID: \"29eb21bd-c9b9-4206-8029-86c0d353d68d\") " pod="openshift-image-registry/node-ca-c6lfc"
Sep 04 21:53:48.430335 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430244    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1daa15e7-a0f6-483d-b923-d1ded7222d70-device-dir\") pod \"aws-ebs-csi-driver-node-457zw\" (UID: \"1daa15e7-a0f6-483d-b923-d1ded7222d70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw"
Sep 04 21:53:48.430335 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430279    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29eb21bd-c9b9-4206-8029-86c0d353d68d-host\") pod \"node-ca-c6lfc\" (UID: \"29eb21bd-c9b9-4206-8029-86c0d353d68d\") " pod="openshift-image-registry/node-ca-c6lfc"
Sep 04 21:53:48.430335 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430316    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f8b91c97-3cb9-421e-8a7b-6da194236723-host-slash\") pod \"iptables-alerter-r22zb\" (UID: \"f8b91c97-3cb9-421e-8a7b-6da194236723\") " pod="openshift-network-operator/iptables-alerter-r22zb"
Sep 04 21:53:48.430335 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430316    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1daa15e7-a0f6-483d-b923-d1ded7222d70-sys-fs\") pod \"aws-ebs-csi-driver-node-457zw\" (UID: \"1daa15e7-a0f6-483d-b923-d1ded7222d70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw"
Sep 04 21:53:48.430701 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:48.430352    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 21:53:48.430701 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430366    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jls52\" (UniqueName: \"kubernetes.io/projected/1daa15e7-a0f6-483d-b923-d1ded7222d70-kube-api-access-jls52\") pod \"aws-ebs-csi-driver-node-457zw\" (UID: \"1daa15e7-a0f6-483d-b923-d1ded7222d70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw"
Sep 04 21:53:48.430701 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430389    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7nz2\" (UniqueName: \"kubernetes.io/projected/f8b91c97-3cb9-421e-8a7b-6da194236723-kube-api-access-v7nz2\") pod \"iptables-alerter-r22zb\" (UID: \"f8b91c97-3cb9-421e-8a7b-6da194236723\") " pod="openshift-network-operator/iptables-alerter-r22zb"
Sep 04 21:53:48.430701 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430409    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f8b91c97-3cb9-421e-8a7b-6da194236723-iptables-alerter-script\") pod \"iptables-alerter-r22zb\" (UID: \"f8b91c97-3cb9-421e-8a7b-6da194236723\") " pod="openshift-network-operator/iptables-alerter-r22zb"
Sep 04 21:53:48.430701 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:48.430444    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret podName:8fcc2326-c662-4e67-bdaf-60a54cce4235 nodeName:}" failed. No retries permitted until 2026-09-04 21:53:48.930410441 +0000 UTC m=+3.246977188 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret") pod "global-pull-secret-syncer-p7b77" (UID: "8fcc2326-c662-4e67-bdaf-60a54cce4235") : object "kube-system"/"original-pull-secret" not registered
Sep 04 21:53:48.430701 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430476    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1daa15e7-a0f6-483d-b923-d1ded7222d70-kubelet-dir\") pod \"aws-ebs-csi-driver-node-457zw\" (UID: \"1daa15e7-a0f6-483d-b923-d1ded7222d70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw"
Sep 04 21:53:48.430701 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430494    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8fcc2326-c662-4e67-bdaf-60a54cce4235-dbus\") pod \"global-pull-secret-syncer-p7b77\" (UID: \"8fcc2326-c662-4e67-bdaf-60a54cce4235\") " pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:53:48.430701 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430525    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1daa15e7-a0f6-483d-b923-d1ded7222d70-etc-selinux\") pod \"aws-ebs-csi-driver-node-457zw\" (UID: \"1daa15e7-a0f6-483d-b923-d1ded7222d70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw"
Sep 04 21:53:48.430701 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430545    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1daa15e7-a0f6-483d-b923-d1ded7222d70-registration-dir\") pod \"aws-ebs-csi-driver-node-457zw\" (UID: \"1daa15e7-a0f6-483d-b923-d1ded7222d70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw"
Sep 04 21:53:48.430701 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430561    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zm7nd\" (UniqueName: \"kubernetes.io/projected/29eb21bd-c9b9-4206-8029-86c0d353d68d-kube-api-access-zm7nd\") pod \"node-ca-c6lfc\" (UID: \"29eb21bd-c9b9-4206-8029-86c0d353d68d\") " pod="openshift-image-registry/node-ca-c6lfc"
Sep 04 21:53:48.430701 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430576    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs\") pod \"network-metrics-daemon-46fbr\" (UID: \"eadd93c2-4a94-4046-9924-6e342406d1f8\") " pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:53:48.430701 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430595    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1daa15e7-a0f6-483d-b923-d1ded7222d70-socket-dir\") pod \"aws-ebs-csi-driver-node-457zw\" (UID: \"1daa15e7-a0f6-483d-b923-d1ded7222d70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw"
Sep 04 21:53:48.430701 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430626    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8fcc2326-c662-4e67-bdaf-60a54cce4235-kubelet-config\") pod \"global-pull-secret-syncer-p7b77\" (UID: \"8fcc2326-c662-4e67-bdaf-60a54cce4235\") " pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:53:48.430701 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430640    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/29eb21bd-c9b9-4206-8029-86c0d353d68d-serviceca\") pod \"node-ca-c6lfc\" (UID: \"29eb21bd-c9b9-4206-8029-86c0d353d68d\") " pod="openshift-image-registry/node-ca-c6lfc"
Sep 04 21:53:48.430701 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430654    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmn8x\" (UniqueName: \"kubernetes.io/projected/eadd93c2-4a94-4046-9924-6e342406d1f8-kube-api-access-kmn8x\") pod \"network-metrics-daemon-46fbr\" (UID: \"eadd93c2-4a94-4046-9924-6e342406d1f8\") " pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:53:48.430701 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430653    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1daa15e7-a0f6-483d-b923-d1ded7222d70-etc-selinux\") pod \"aws-ebs-csi-driver-node-457zw\" (UID: \"1daa15e7-a0f6-483d-b923-d1ded7222d70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw"
Sep 04 21:53:48.431309 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430725    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1daa15e7-a0f6-483d-b923-d1ded7222d70-registration-dir\") pod \"aws-ebs-csi-driver-node-457zw\" (UID: \"1daa15e7-a0f6-483d-b923-d1ded7222d70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw"
Sep 04 21:53:48.431309 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430810    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8fcc2326-c662-4e67-bdaf-60a54cce4235-dbus\") pod \"global-pull-secret-syncer-p7b77\" (UID: \"8fcc2326-c662-4e67-bdaf-60a54cce4235\") " pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:53:48.431309 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430857    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1daa15e7-a0f6-483d-b923-d1ded7222d70-kubelet-dir\") pod \"aws-ebs-csi-driver-node-457zw\" (UID: \"1daa15e7-a0f6-483d-b923-d1ded7222d70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw"
Sep 04 21:53:48.431309 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430868    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f8b91c97-3cb9-421e-8a7b-6da194236723-iptables-alerter-script\") pod \"iptables-alerter-r22zb\" (UID: \"f8b91c97-3cb9-421e-8a7b-6da194236723\") " pod="openshift-network-operator/iptables-alerter-r22zb"
Sep 04 21:53:48.431309 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:48.430881    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:53:48.431309 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:48.430945    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs podName:eadd93c2-4a94-4046-9924-6e342406d1f8 nodeName:}" failed. No retries permitted until 2026-09-04 21:53:48.930927723 +0000 UTC m=+3.247494454 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs") pod "network-metrics-daemon-46fbr" (UID: "eadd93c2-4a94-4046-9924-6e342406d1f8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:53:48.431309 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430948    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1daa15e7-a0f6-483d-b923-d1ded7222d70-socket-dir\") pod \"aws-ebs-csi-driver-node-457zw\" (UID: \"1daa15e7-a0f6-483d-b923-d1ded7222d70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw"
Sep 04 21:53:48.431309 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.430946    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8fcc2326-c662-4e67-bdaf-60a54cce4235-kubelet-config\") pod \"global-pull-secret-syncer-p7b77\" (UID: \"8fcc2326-c662-4e67-bdaf-60a54cce4235\") " pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:53:48.431309 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.431170    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/29eb21bd-c9b9-4206-8029-86c0d353d68d-serviceca\") pod \"node-ca-c6lfc\" (UID: \"29eb21bd-c9b9-4206-8029-86c0d353d68d\") " pod="openshift-image-registry/node-ca-c6lfc"
Sep 04 21:53:48.440472 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.440384    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zm7nd\" (UniqueName: \"kubernetes.io/projected/29eb21bd-c9b9-4206-8029-86c0d353d68d-kube-api-access-zm7nd\") pod \"node-ca-c6lfc\" (UID: \"29eb21bd-c9b9-4206-8029-86c0d353d68d\") " pod="openshift-image-registry/node-ca-c6lfc"
Sep 04 21:53:48.440608 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.440544    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jls52\" (UniqueName: \"kubernetes.io/projected/1daa15e7-a0f6-483d-b923-d1ded7222d70-kube-api-access-jls52\") pod \"aws-ebs-csi-driver-node-457zw\" (UID: \"1daa15e7-a0f6-483d-b923-d1ded7222d70\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw"
Sep 04 21:53:48.440948 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.440926    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7nz2\" (UniqueName: \"kubernetes.io/projected/f8b91c97-3cb9-421e-8a7b-6da194236723-kube-api-access-v7nz2\") pod \"iptables-alerter-r22zb\" (UID: \"f8b91c97-3cb9-421e-8a7b-6da194236723\") " pod="openshift-network-operator/iptables-alerter-r22zb"
Sep 04 21:53:48.441619 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.441600    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmn8x\" (UniqueName: \"kubernetes.io/projected/eadd93c2-4a94-4046-9924-6e342406d1f8-kube-api-access-kmn8x\") pod \"network-metrics-daemon-46fbr\" (UID: \"eadd93c2-4a94-4046-9924-6e342406d1f8\") " pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:53:48.513208 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.513164    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xcgjm"
Sep 04 21:53:48.521107 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.521079    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ft99b"
Sep 04 21:53:48.529744 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.529713    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-tr8wn"
Sep 04 21:53:48.536547 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.536518    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dzjmv"
Sep 04 21:53:48.544558 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.544533    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9nl4m"
Sep 04 21:53:48.552517 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.552490    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:53:48.560540 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.560506    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-c6lfc"
Sep 04 21:53:48.566448 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.566406    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw"
Sep 04 21:53:48.572291 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.572270    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r22zb"
Sep 04 21:53:48.933481 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.933383    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhjmp\" (UniqueName: \"kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp\") pod \"network-check-target-dflhd\" (UID: \"10da7e37-b753-4b5f-93f5-07e7a48ba85b\") " pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:53:48.933481 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.933447    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs\") pod \"network-metrics-daemon-46fbr\" (UID: \"eadd93c2-4a94-4046-9924-6e342406d1f8\") " pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:53:48.933481 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:48.933480    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret\") pod \"global-pull-secret-syncer-p7b77\" (UID: \"8fcc2326-c662-4e67-bdaf-60a54cce4235\") " pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:53:48.933718 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:48.933553    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 21:53:48.933718 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:48.933572    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 21:53:48.933718 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:48.933623    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret podName:8fcc2326-c662-4e67-bdaf-60a54cce4235 nodeName:}" failed. No retries permitted until 2026-09-04 21:53:49.933609924 +0000 UTC m=+4.250176643 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret") pod "global-pull-secret-syncer-p7b77" (UID: "8fcc2326-c662-4e67-bdaf-60a54cce4235") : object "kube-system"/"original-pull-secret" not registered
Sep 04 21:53:48.933718 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:48.933574    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:53:48.933718 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:48.933703    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs podName:eadd93c2-4a94-4046-9924-6e342406d1f8 nodeName:}" failed. No retries permitted until 2026-09-04 21:53:49.933684322 +0000 UTC m=+4.250251038 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs") pod "network-metrics-daemon-46fbr" (UID: "eadd93c2-4a94-4046-9924-6e342406d1f8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:53:48.933718 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:48.933576    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 21:53:48.933915 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:48.933732    2557 projected.go:196] Error preparing data for projected volume kube-api-access-fhjmp for pod openshift-network-diagnostics/network-check-target-dflhd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:53:48.933915 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:48.933778    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp podName:10da7e37-b753-4b5f-93f5-07e7a48ba85b nodeName:}" failed. No retries permitted until 2026-09-04 21:53:49.933765433 +0000 UTC m=+4.250332149 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fhjmp" (UniqueName: "kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp") pod "network-check-target-dflhd" (UID: "10da7e37-b753-4b5f-93f5-07e7a48ba85b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:53:49.082858 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:49.082832    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb36d1fd6_09dc_43b7_b179_051cd03a518f.slice/crio-1aabe6eec90dc557aebdf75b5871ad72feaf6307cfc3ebe30d1149386ae37dfa WatchSource:0}: Error finding container 1aabe6eec90dc557aebdf75b5871ad72feaf6307cfc3ebe30d1149386ae37dfa: Status 404 returned error can't find the container with id 1aabe6eec90dc557aebdf75b5871ad72feaf6307cfc3ebe30d1149386ae37dfa
Sep 04 21:53:49.083882 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:49.083859    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1daa15e7_a0f6_483d_b923_d1ded7222d70.slice/crio-150f7144b8222c21c05cd9ad5b43fac13ab767281020fa168e830c8ccce76f9e WatchSource:0}: Error finding container 150f7144b8222c21c05cd9ad5b43fac13ab767281020fa168e830c8ccce76f9e: Status 404 returned error can't find the container with id 150f7144b8222c21c05cd9ad5b43fac13ab767281020fa168e830c8ccce76f9e
Sep 04 21:53:49.084467 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:49.084449    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb97b042_b391_4f31_8f0f_3deceacb5157.slice/crio-de1d028f471d12c995f44107774a31ba53e8bd853fabc78117c3c00f8f2b5e26 WatchSource:0}: Error finding container de1d028f471d12c995f44107774a31ba53e8bd853fabc78117c3c00f8f2b5e26: Status 404 returned error can't find the container with id de1d028f471d12c995f44107774a31ba53e8bd853fabc78117c3c00f8f2b5e26
Sep 04 21:53:49.085721 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:49.085590    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb5aaba9_9cb0_4fa7_a342_344f1e40cad8.slice/crio-747f1dc60efcc69879747914ecd314977e79c098a076f73a434ddc4a6adc5004 WatchSource:0}: Error finding container 747f1dc60efcc69879747914ecd314977e79c098a076f73a434ddc4a6adc5004: Status 404 returned error can't find the container with id 747f1dc60efcc69879747914ecd314977e79c098a076f73a434ddc4a6adc5004
Sep 04 21:53:49.087002 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:49.086947    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcdb4989_2fe2_44ce_b20a_8d3e1dc05b90.slice/crio-165b43ee05ffadec3ece430ca899a9d9a0654d2d4cdca0637003b5329e2a11ba WatchSource:0}: Error finding container 165b43ee05ffadec3ece430ca899a9d9a0654d2d4cdca0637003b5329e2a11ba: Status 404 returned error can't find the container with id 165b43ee05ffadec3ece430ca899a9d9a0654d2d4cdca0637003b5329e2a11ba
Sep 04 21:53:49.087950 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:49.087928    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5416a22e_ce9e_4fd1_a2a2_6b6f82b59ff6.slice/crio-365b71de817b8c91501ffb2f132118b9437bac6cc7c5fb906f0a6ae1c018ac4b WatchSource:0}: Error finding container 365b71de817b8c91501ffb2f132118b9437bac6cc7c5fb906f0a6ae1c018ac4b: Status 404 returned error can't find the container with id 365b71de817b8c91501ffb2f132118b9437bac6cc7c5fb906f0a6ae1c018ac4b
Sep 04 21:53:49.089116 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:49.088987    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29eb21bd_c9b9_4206_8029_86c0d353d68d.slice/crio-57c65abdaf1e0834de612b092a8948807b7d1d54715fca366faf0c25032995da WatchSource:0}: Error finding container 57c65abdaf1e0834de612b092a8948807b7d1d54715fca366faf0c25032995da: Status 404 returned error can't find the container with id 57c65abdaf1e0834de612b092a8948807b7d1d54715fca366faf0c25032995da
Sep 04 21:53:49.089962 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:49.089840    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d2dc985_0ea0_47ff_88d2_600e4e46a436.slice/crio-ff58a00ff7209d13bafa2dd7fd60d559ab2db6202b85855c5f0d439c68539c74 WatchSource:0}: Error finding container ff58a00ff7209d13bafa2dd7fd60d559ab2db6202b85855c5f0d439c68539c74: Status 404 returned error can't find the container with id ff58a00ff7209d13bafa2dd7fd60d559ab2db6202b85855c5f0d439c68539c74
Sep 04 21:53:49.092674 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:53:49.092299    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8b91c97_3cb9_421e_8a7b_6da194236723.slice/crio-5efe576441f92eb19951118bf08c07e109e5af53aa85bcb14ab6e80474724a70 WatchSource:0}: Error finding container 5efe576441f92eb19951118bf08c07e109e5af53aa85bcb14ab6e80474724a70: Status 404 returned error can't find the container with id 5efe576441f92eb19951118bf08c07e109e5af53aa85bcb14ab6e80474724a70
Sep 04 21:53:49.278005 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:49.277958    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-03 21:48:48 +0000 UTC" deadline="2028-03-31 06:24:40.259953342 +0000 UTC"
Sep 04 21:53:49.278005 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:49.277992    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13760h30m50.9819644s"
Sep 04 21:53:49.346889 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:49.346860    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:53:49.347014 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:49.346960    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:53:49.358483 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:49.358445    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tr8wn" event={"ID":"b36d1fd6-09dc-43b7-b179-051cd03a518f","Type":"ContainerStarted","Data":"1aabe6eec90dc557aebdf75b5871ad72feaf6307cfc3ebe30d1149386ae37dfa"}
Sep 04 21:53:49.360516 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:49.360481    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-141.ec2.internal" event={"ID":"79b44ba13a4b8fc8cb712cd9da0778c3","Type":"ContainerStarted","Data":"72ce96241b7a2cc276450eeb58078caac02d8ba70c7912e3a07c4a5a6479296e"}
Sep 04 21:53:49.361860 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:49.361828    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw" event={"ID":"1daa15e7-a0f6-483d-b923-d1ded7222d70","Type":"ContainerStarted","Data":"150f7144b8222c21c05cd9ad5b43fac13ab767281020fa168e830c8ccce76f9e"}
Sep 04 21:53:49.362974 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:49.362954    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ss72b" event={"ID":"5d2dc985-0ea0-47ff-88d2-600e4e46a436","Type":"ContainerStarted","Data":"ff58a00ff7209d13bafa2dd7fd60d559ab2db6202b85855c5f0d439c68539c74"}
Sep 04 21:53:49.364095 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:49.364070    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-c6lfc" event={"ID":"29eb21bd-c9b9-4206-8029-86c0d353d68d","Type":"ContainerStarted","Data":"57c65abdaf1e0834de612b092a8948807b7d1d54715fca366faf0c25032995da"}
Sep 04 21:53:49.365158 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:49.365140    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ft99b" event={"ID":"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6","Type":"ContainerStarted","Data":"365b71de817b8c91501ffb2f132118b9437bac6cc7c5fb906f0a6ae1c018ac4b"}
Sep 04 21:53:49.366133 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:49.366107    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xcgjm" event={"ID":"db97b042-b391-4f31-8f0f-3deceacb5157","Type":"ContainerStarted","Data":"de1d028f471d12c995f44107774a31ba53e8bd853fabc78117c3c00f8f2b5e26"}
Sep 04 21:53:49.367197 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:49.367177    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r22zb" event={"ID":"f8b91c97-3cb9-421e-8a7b-6da194236723","Type":"ContainerStarted","Data":"5efe576441f92eb19951118bf08c07e109e5af53aa85bcb14ab6e80474724a70"}
Sep 04 21:53:49.368210 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:49.368193    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzjmv" event={"ID":"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90","Type":"ContainerStarted","Data":"165b43ee05ffadec3ece430ca899a9d9a0654d2d4cdca0637003b5329e2a11ba"}
Sep 04 21:53:49.369097 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:49.369079    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9nl4m" event={"ID":"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8","Type":"ContainerStarted","Data":"747f1dc60efcc69879747914ecd314977e79c098a076f73a434ddc4a6adc5004"}
Sep 04 21:53:49.375389 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:49.375345    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-141.ec2.internal" podStartSLOduration=2.375333587 podStartE2EDuration="2.375333587s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:53:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:53:49.374910143 +0000 UTC m=+3.691476880" watchObservedRunningTime="2026-09-04 21:53:49.375333587 +0000 UTC m=+3.691900325"
Sep 04 21:53:49.940946 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:49.940589    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret\") pod \"global-pull-secret-syncer-p7b77\" (UID: \"8fcc2326-c662-4e67-bdaf-60a54cce4235\") " pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:53:49.940946 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:49.940865    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 21:53:49.940946 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:49.940903    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhjmp\" (UniqueName: \"kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp\") pod \"network-check-target-dflhd\" (UID: \"10da7e37-b753-4b5f-93f5-07e7a48ba85b\") " pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:53:49.940946 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:49.940948    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret podName:8fcc2326-c662-4e67-bdaf-60a54cce4235 nodeName:}" failed. No retries permitted until 2026-09-04 21:53:51.940928413 +0000 UTC m=+6.257495144 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret") pod "global-pull-secret-syncer-p7b77" (UID: "8fcc2326-c662-4e67-bdaf-60a54cce4235") : object "kube-system"/"original-pull-secret" not registered
Sep 04 21:53:49.941294 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:49.940987    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs\") pod \"network-metrics-daemon-46fbr\" (UID: \"eadd93c2-4a94-4046-9924-6e342406d1f8\") " pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:53:49.941294 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:49.941091    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:53:49.941294 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:49.941137    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs podName:eadd93c2-4a94-4046-9924-6e342406d1f8 nodeName:}" failed. No retries permitted until 2026-09-04 21:53:51.941123185 +0000 UTC m=+6.257689906 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs") pod "network-metrics-daemon-46fbr" (UID: "eadd93c2-4a94-4046-9924-6e342406d1f8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:53:49.941294 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:49.941218    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 21:53:49.941294 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:49.941235    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 21:53:49.941294 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:49.941249    2557 projected.go:196] Error preparing data for projected volume kube-api-access-fhjmp for pod openshift-network-diagnostics/network-check-target-dflhd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:53:49.941294 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:49.941288    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp podName:10da7e37-b753-4b5f-93f5-07e7a48ba85b nodeName:}" failed. No retries permitted until 2026-09-04 21:53:51.941276069 +0000 UTC m=+6.257842785 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fhjmp" (UniqueName: "kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp") pod "network-check-target-dflhd" (UID: "10da7e37-b753-4b5f-93f5-07e7a48ba85b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:53:50.278965 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:50.278899    2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-03 21:48:48 +0000 UTC" deadline="2028-04-27 18:14:51.513179685 +0000 UTC"
Sep 04 21:53:50.278965 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:50.278962    2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14420h21m1.23422182s"
Sep 04 21:53:50.348065 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:50.347548    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:53:50.348065 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:50.347677    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:53:50.348767 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:50.348512    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:53:50.348767 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:50.348643    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:53:50.392441 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:50.390022    2557 generic.go:356] "Generic (PLEG): container finished" podID="9b84a1cc7bbade60d3a69ba540f3e588" containerID="4611cb41411c2ccbc55b4a183b8a109626724be2eb285c34019973ad87733e1f" exitCode=0
Sep 04 21:53:50.392441 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:50.391029    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-141.ec2.internal" event={"ID":"9b84a1cc7bbade60d3a69ba540f3e588","Type":"ContainerDied","Data":"4611cb41411c2ccbc55b4a183b8a109626724be2eb285c34019973ad87733e1f"}
Sep 04 21:53:51.347351 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:51.347317    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:53:51.347826 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:51.347484    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:53:51.404526 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:51.404485    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-141.ec2.internal" event={"ID":"9b84a1cc7bbade60d3a69ba540f3e588","Type":"ContainerStarted","Data":"cfdca3f734f7eb8a535b0f70ae0f4facd3b485e5c1b148f48a9185e4f8e75093"}
Sep 04 21:53:51.425347 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:51.425057    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-141.ec2.internal" podStartSLOduration=4.425035911 podStartE2EDuration="4.425035911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:53:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:53:51.424431907 +0000 UTC m=+5.740998638" watchObservedRunningTime="2026-09-04 21:53:51.425035911 +0000 UTC m=+5.741602650"
Sep 04 21:53:51.963468 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:51.962656    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret\") pod \"global-pull-secret-syncer-p7b77\" (UID: \"8fcc2326-c662-4e67-bdaf-60a54cce4235\") " pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:53:51.963468 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:51.962727    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhjmp\" (UniqueName: \"kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp\") pod \"network-check-target-dflhd\" (UID: \"10da7e37-b753-4b5f-93f5-07e7a48ba85b\") " pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:53:51.963468 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:51.962769    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs\") pod \"network-metrics-daemon-46fbr\" (UID: \"eadd93c2-4a94-4046-9924-6e342406d1f8\") " pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:53:51.963468 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:51.962899    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:53:51.963468 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:51.962962    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs podName:eadd93c2-4a94-4046-9924-6e342406d1f8 nodeName:}" failed. No retries permitted until 2026-09-04 21:53:55.96294316 +0000 UTC m=+10.279509879 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs") pod "network-metrics-daemon-46fbr" (UID: "eadd93c2-4a94-4046-9924-6e342406d1f8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:53:51.963468 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:51.963368    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 21:53:51.963468 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:51.963440    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret podName:8fcc2326-c662-4e67-bdaf-60a54cce4235 nodeName:}" failed. No retries permitted until 2026-09-04 21:53:55.963407187 +0000 UTC m=+10.279973903 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret") pod "global-pull-secret-syncer-p7b77" (UID: "8fcc2326-c662-4e67-bdaf-60a54cce4235") : object "kube-system"/"original-pull-secret" not registered
Sep 04 21:53:51.963468 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:51.963449    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 21:53:51.963468 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:51.963464    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 21:53:51.963468 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:51.963476    2557 projected.go:196] Error preparing data for projected volume kube-api-access-fhjmp for pod openshift-network-diagnostics/network-check-target-dflhd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:53:51.964131 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:51.963509    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp podName:10da7e37-b753-4b5f-93f5-07e7a48ba85b nodeName:}" failed. No retries permitted until 2026-09-04 21:53:55.963499587 +0000 UTC m=+10.280066316 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fhjmp" (UniqueName: "kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp") pod "network-check-target-dflhd" (UID: "10da7e37-b753-4b5f-93f5-07e7a48ba85b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:53:52.348784 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:52.348690    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:53:52.348784 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:52.348726    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:53:52.349270 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:52.348828    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:53:52.349270 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:52.349030    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:53:53.346920 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:53.346882    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:53:53.347103 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:53.347017    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:53:54.346662 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:54.346629    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:53:54.347166 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:54.346760    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:53:54.347166 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:54.346841    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:53:54.347166 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:54.346944    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:53:55.346530 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:55.346498    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:53:55.346730 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:55.346630    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:53:55.998348 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:55.998305    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhjmp\" (UniqueName: \"kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp\") pod \"network-check-target-dflhd\" (UID: \"10da7e37-b753-4b5f-93f5-07e7a48ba85b\") " pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:53:55.998547 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:55.998374    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs\") pod \"network-metrics-daemon-46fbr\" (UID: \"eadd93c2-4a94-4046-9924-6e342406d1f8\") " pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:53:55.998547 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:55.998437    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret\") pod \"global-pull-secret-syncer-p7b77\" (UID: \"8fcc2326-c662-4e67-bdaf-60a54cce4235\") " pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:53:55.998547 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:55.998521    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:53:55.998547 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:55.998540    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 21:53:55.998727 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:55.998566    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 21:53:55.998727 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:55.998581    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs podName:eadd93c2-4a94-4046-9924-6e342406d1f8 nodeName:}" failed. No retries permitted until 2026-09-04 21:54:03.99856568 +0000 UTC m=+18.315132415 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs") pod "network-metrics-daemon-46fbr" (UID: "eadd93c2-4a94-4046-9924-6e342406d1f8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:53:55.998727 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:55.998581    2557 projected.go:196] Error preparing data for projected volume kube-api-access-fhjmp for pod openshift-network-diagnostics/network-check-target-dflhd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:53:55.998727 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:55.998611    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp podName:10da7e37-b753-4b5f-93f5-07e7a48ba85b nodeName:}" failed. No retries permitted until 2026-09-04 21:54:03.99860492 +0000 UTC m=+18.315171635 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fhjmp" (UniqueName: "kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp") pod "network-check-target-dflhd" (UID: "10da7e37-b753-4b5f-93f5-07e7a48ba85b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:53:55.998727 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:55.998546    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 21:53:55.998727 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:55.998632    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret podName:8fcc2326-c662-4e67-bdaf-60a54cce4235 nodeName:}" failed. No retries permitted until 2026-09-04 21:54:03.99862788 +0000 UTC m=+18.315194596 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret") pod "global-pull-secret-syncer-p7b77" (UID: "8fcc2326-c662-4e67-bdaf-60a54cce4235") : object "kube-system"/"original-pull-secret" not registered
Sep 04 21:53:56.348525 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:56.347933    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:53:56.348525 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:56.348049    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:53:56.349021 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:56.348598    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:53:56.349021 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:56.348740    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:53:57.346358 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:57.346322    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:53:57.346566 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:57.346485    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:53:58.347453 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:58.347276    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:53:58.347453 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:58.347312    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:53:58.347453 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:58.347451    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:53:58.348081 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:58.347540    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:53:59.346977 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:53:59.346938    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:53:59.347131 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:53:59.347044    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:54:00.346565 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:00.346529    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:00.347016 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:00.346657    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:54:00.347016 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:00.346710    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:00.347016 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:00.346825    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:54:01.347093 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:01.347054    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:01.347566 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:01.347256    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:54:02.346839 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:02.346808    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:02.347019 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:02.346808    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:02.347019 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:02.346953    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:54:02.347146 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:02.347027    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:54:03.347346 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:03.347308    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:03.347887 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:03.347454    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:54:04.054490 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:04.054447    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret\") pod \"global-pull-secret-syncer-p7b77\" (UID: \"8fcc2326-c662-4e67-bdaf-60a54cce4235\") " pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:04.054685 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:04.054520    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhjmp\" (UniqueName: \"kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp\") pod \"network-check-target-dflhd\" (UID: \"10da7e37-b753-4b5f-93f5-07e7a48ba85b\") " pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:04.054685 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:04.054556    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs\") pod \"network-metrics-daemon-46fbr\" (UID: \"eadd93c2-4a94-4046-9924-6e342406d1f8\") " pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:04.054685 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:04.054605    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 21:54:04.054685 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:04.054641    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:54:04.054685 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:04.054686    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret podName:8fcc2326-c662-4e67-bdaf-60a54cce4235 nodeName:}" failed. No retries permitted until 2026-09-04 21:54:20.054663769 +0000 UTC m=+34.371230573 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret") pod "global-pull-secret-syncer-p7b77" (UID: "8fcc2326-c662-4e67-bdaf-60a54cce4235") : object "kube-system"/"original-pull-secret" not registered
Sep 04 21:54:04.054891 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:04.054708    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs podName:eadd93c2-4a94-4046-9924-6e342406d1f8 nodeName:}" failed. No retries permitted until 2026-09-04 21:54:20.054698908 +0000 UTC m=+34.371265624 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs") pod "network-metrics-daemon-46fbr" (UID: "eadd93c2-4a94-4046-9924-6e342406d1f8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:54:04.054891 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:04.054687    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 21:54:04.054891 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:04.054761    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 21:54:04.054891 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:04.054777    2557 projected.go:196] Error preparing data for projected volume kube-api-access-fhjmp for pod openshift-network-diagnostics/network-check-target-dflhd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:54:04.054891 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:04.054833    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp podName:10da7e37-b753-4b5f-93f5-07e7a48ba85b nodeName:}" failed. No retries permitted until 2026-09-04 21:54:20.054814922 +0000 UTC m=+34.371381651 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fhjmp" (UniqueName: "kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp") pod "network-check-target-dflhd" (UID: "10da7e37-b753-4b5f-93f5-07e7a48ba85b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:54:04.347500 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:04.347395    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:04.347944 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:04.347400    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:04.347944 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:04.347555    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:54:04.347944 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:04.347625    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:54:05.347092 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:05.347057    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:05.347276 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:05.347176    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:54:06.347179 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:06.347141    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:06.347631 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:06.347250    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:54:06.347631 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:06.347333    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:06.347631 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:06.347473    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:54:07.347525 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:07.347306    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:07.348248 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:07.347605    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:54:07.437043 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:07.437007    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw" event={"ID":"1daa15e7-a0f6-483d-b923-d1ded7222d70","Type":"ContainerStarted","Data":"b51d062b9aab0bb25b76d4a3174ffd945072e78628646a7d593f7c5a17d45a81"}
Sep 04 21:54:07.439313 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:07.439286    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ss72b" event={"ID":"5d2dc985-0ea0-47ff-88d2-600e4e46a436","Type":"ContainerStarted","Data":"37f82503e4db9ca2c0aa7d289acc66e6263d364a4ff211cfd25190b787ff4942"}
Sep 04 21:54:07.439483 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:07.439357    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ss72b" event={"ID":"5d2dc985-0ea0-47ff-88d2-600e4e46a436","Type":"ContainerStarted","Data":"968ec480014ff9571716446b9929294936503f606b7edf1a75d17b4acd96c0eb"}
Sep 04 21:54:07.439483 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:07.439429    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ss72b" event={"ID":"5d2dc985-0ea0-47ff-88d2-600e4e46a436","Type":"ContainerStarted","Data":"f51c5f2f31b5e66d07f867b4fed537bc747dc068c86ea4f116bb0228f2dbf3cc"}
Sep 04 21:54:07.439483 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:07.439464    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ss72b" event={"ID":"5d2dc985-0ea0-47ff-88d2-600e4e46a436","Type":"ContainerStarted","Data":"540f45bc8c32b4b88a5d1ae87631d64f499dd7a6607f253524470ec1afc66309"}
Sep 04 21:54:07.440632 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:07.440608    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-c6lfc" event={"ID":"29eb21bd-c9b9-4206-8029-86c0d353d68d","Type":"ContainerStarted","Data":"7c77e045acfcca161293ec2e21a1603db4a6c69c2d0b65012faf4ec0116ed475"}
Sep 04 21:54:07.441785 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:07.441763    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ft99b" event={"ID":"5416a22e-ce9e-4fd1-a2a2-6b6f82b59ff6","Type":"ContainerStarted","Data":"6de5f0aeede94c92dd31e450eca0b011091e54ce1065cd058f5bc33502df7b8a"}
Sep 04 21:54:07.443113 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:07.443090    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xcgjm" event={"ID":"db97b042-b391-4f31-8f0f-3deceacb5157","Type":"ContainerStarted","Data":"3dfde8398d7c242d9c69354271274507d2a02ceef5fd5d1000815c5723227411"}
Sep 04 21:54:07.444460 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:07.444437    2557 generic.go:356] "Generic (PLEG): container finished" podID="bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90" containerID="0b4f9fd5640f529f448229ed90a9393a14d61c7b0f1c0dde99631414d8fcb2cd" exitCode=0
Sep 04 21:54:07.444548 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:07.444502    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzjmv" event={"ID":"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90","Type":"ContainerDied","Data":"0b4f9fd5640f529f448229ed90a9393a14d61c7b0f1c0dde99631414d8fcb2cd"}
Sep 04 21:54:07.445959 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:07.445938    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9nl4m" event={"ID":"fb5aaba9-9cb0-4fa7-a342-344f1e40cad8","Type":"ContainerStarted","Data":"f92ab163e35f1dc0597db330844a375c9a6486d9c50e798359fc8fa6ac9279c4"}
Sep 04 21:54:07.447213 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:07.447196    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-tr8wn" event={"ID":"b36d1fd6-09dc-43b7-b179-051cd03a518f","Type":"ContainerStarted","Data":"0fe583c78a987746f44924088fcb01251afe223ac19c89546fd89e8e04dbeb35"}
Sep 04 21:54:07.496909 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:07.496860    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9nl4m" podStartSLOduration=3.53750999 podStartE2EDuration="21.496843876s" totalImagesPullingTime="17.959333886s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:07.494674163 +0000 UTC m=+21.811240901" watchObservedRunningTime="2026-09-04 21:54:07.496843876 +0000 UTC m=+21.813410614"
Sep 04 21:54:07.514807 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:07.514751    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xcgjm" podStartSLOduration=7.451317198 podStartE2EDuration="21.51473619s" totalImagesPullingTime="14.063418992s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:07.513850038 +0000 UTC m=+21.830416775" watchObservedRunningTime="2026-09-04 21:54:07.51473619 +0000 UTC m=+21.831302927"
Sep 04 21:54:07.555837 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:07.555793    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ft99b" podStartSLOduration=3.649833763 podStartE2EDuration="21.555776162s" totalImagesPullingTime="17.905942399s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:07.537687471 +0000 UTC m=+21.854254205" watchObservedRunningTime="2026-09-04 21:54:07.555776162 +0000 UTC m=+21.872342899"
Sep 04 21:54:07.555984 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:07.555900    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-tr8wn" podStartSLOduration=3.646175094 podStartE2EDuration="21.555893724s" totalImagesPullingTime="17.90971863s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:07.555477848 +0000 UTC m=+21.872044586" watchObservedRunningTime="2026-09-04 21:54:07.555893724 +0000 UTC m=+21.872460461"
Sep 04 21:54:08.347147 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:08.347107    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:08.347308 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:08.347107    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:08.347308 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:08.347234    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:54:08.347308 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:08.347288    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:54:08.453297 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:08.453261    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ss72b" event={"ID":"5d2dc985-0ea0-47ff-88d2-600e4e46a436","Type":"ContainerStarted","Data":"e650bbf11e9bbf1ef348fc574dffc9897e9c7e9ce9ace92d61a673b23c9be336"}
Sep 04 21:54:08.453746 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:08.453352    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ss72b" event={"ID":"5d2dc985-0ea0-47ff-88d2-600e4e46a436","Type":"ContainerStarted","Data":"30dc7d1be7fc0a8d11a0cf0cba4a70e5d4925b3336f11d3fd365d2ca7820b035"}
Sep 04 21:54:08.455059 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:08.455025    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r22zb" event={"ID":"f8b91c97-3cb9-421e-8a7b-6da194236723","Type":"ContainerStarted","Data":"bb945caaea5f92baf92d09b96b7fd3397c4e772ac8ad252ac46588bb922fdd96"}
Sep 04 21:54:08.471268 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:08.471210    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-c6lfc" podStartSLOduration=4.568751552 podStartE2EDuration="22.471194226s" totalImagesPullingTime="17.902442674s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:07.578559939 +0000 UTC m=+21.895126676" watchObservedRunningTime="2026-09-04 21:54:08.471194226 +0000 UTC m=+22.787760964"
Sep 04 21:54:08.471878 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:08.471848    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-r22zb" podStartSLOduration=4.5720130260000005 podStartE2EDuration="22.471836008s" totalImagesPullingTime="17.899822982s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:08.470942422 +0000 UTC m=+22.787509161" watchObservedRunningTime="2026-09-04 21:54:08.471836008 +0000 UTC m=+22.788402745"
Sep 04 21:54:08.495219 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:08.495189    2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 04 21:54:09.284942 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:09.284823    2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-04T21:54:08.495209929Z","UUID":"9ae96731-5647-4167-aa3c-38c08ae61031","Handler":null,"Name":"","Endpoint":""}
Sep 04 21:54:09.286654 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:09.286628    2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 04 21:54:09.286789 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:09.286667    2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 04 21:54:09.346680 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:09.346645    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:09.346847 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:09.346749    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:54:09.460389 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:09.460356    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw" event={"ID":"1daa15e7-a0f6-483d-b923-d1ded7222d70","Type":"ContainerStarted","Data":"7c5eff006c08acb1931ed4b2e980d76d630be5cec7d8b754c29ef779cdd4c2a8"}
Sep 04 21:54:10.347212 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:10.347177    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:10.347429 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:10.347322    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:54:10.347429 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:10.347371    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:10.347561 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:10.347500    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:54:10.464227 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:10.464184    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw" event={"ID":"1daa15e7-a0f6-483d-b923-d1ded7222d70","Type":"ContainerStarted","Data":"e3340eb313e985bf933e7d20c327ce273951efdaa606f6a5ccef561a3c7fe3cc"}
Sep 04 21:54:10.467541 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:10.467506    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ss72b" event={"ID":"5d2dc985-0ea0-47ff-88d2-600e4e46a436","Type":"ContainerStarted","Data":"97cdc096eded6bcb55971e0464d4c94a22fec33f3cf9c2e4219dd0013b73725e"}
Sep 04 21:54:10.489166 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:10.489115    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-457zw" podStartSLOduration=4.526178516 podStartE2EDuration="24.489097756s" totalImagesPullingTime="19.96291924s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:10.488287405 +0000 UTC m=+24.804854140" watchObservedRunningTime="2026-09-04 21:54:10.489097756 +0000 UTC m=+24.805664485"
Sep 04 21:54:10.565789 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:10.565746    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xcgjm"
Sep 04 21:54:10.566491 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:10.566464    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xcgjm"
Sep 04 21:54:10.790587 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:10.790517    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tr8wn_b36d1fd6-09dc-43b7-b179-051cd03a518f/dns-node-resolver/0.log"
Sep 04 21:54:11.346739 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:11.346707    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:11.346902 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:11.346840    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:54:11.353703 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:11.353673    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-c6lfc_29eb21bd-c9b9-4206-8029-86c0d353d68d/node-ca/0.log"
Sep 04 21:54:11.470331 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:11.470272    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xcgjm"
Sep 04 21:54:11.470803 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:11.470707    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xcgjm"
Sep 04 21:54:12.350270 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:12.350239    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:12.350457 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:12.350341    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:54:12.350457 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:12.350392    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:12.350560 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:12.350488    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:54:12.476452 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:12.476355    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ss72b" event={"ID":"5d2dc985-0ea0-47ff-88d2-600e4e46a436","Type":"ContainerStarted","Data":"47bbf5583c88baeeb0226acf89e92954fe054caf1e9f6d70f3408ecc05d116d0"}
Sep 04 21:54:12.477335 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:12.476588    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:54:12.478553 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:12.478527    2557 generic.go:356] "Generic (PLEG): container finished" podID="bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90" containerID="5cb8ba7c506e87018d9085268bf4bf67c67d494d4fb5616af5e9a614df337e49" exitCode=0
Sep 04 21:54:12.478671 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:12.478620    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzjmv" event={"ID":"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90","Type":"ContainerDied","Data":"5cb8ba7c506e87018d9085268bf4bf67c67d494d4fb5616af5e9a614df337e49"}
Sep 04 21:54:12.493082 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:12.493052    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:54:12.514730 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:12.514676    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ss72b" podStartSLOduration=8.580136934 podStartE2EDuration="26.514659603s" totalImagesPullingTime="17.934522669s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:12.514482523 +0000 UTC m=+26.831049263" watchObservedRunningTime="2026-09-04 21:54:12.514659603 +0000 UTC m=+26.831226341"
Sep 04 21:54:13.346686 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:13.346644    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:13.346824 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:13.346757    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:54:13.485503 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:13.484376    2557 generic.go:356] "Generic (PLEG): container finished" podID="bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90" containerID="caf0d1daad771f55b67506ffb4cb86c8eac750a535c4e1fe0e53f94c08440920" exitCode=0
Sep 04 21:54:13.487364 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:13.485981    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzjmv" event={"ID":"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90","Type":"ContainerDied","Data":"caf0d1daad771f55b67506ffb4cb86c8eac750a535c4e1fe0e53f94c08440920"}
Sep 04 21:54:13.487364 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:13.487266    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:54:13.488042 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:13.487822    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:54:13.505789 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:13.505762    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:54:14.315881 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:14.315843    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p7b77"]
Sep 04 21:54:14.316045 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:14.315962    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:14.316127 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:14.316042    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:54:14.318185 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:14.318151    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-46fbr"]
Sep 04 21:54:14.318360 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:14.318279    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:14.318464 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:14.318372    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:54:14.318731 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:14.318709    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dflhd"]
Sep 04 21:54:14.318810 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:14.318797    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:14.318907 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:14.318891    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:54:14.488482 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:14.488385    2557 generic.go:356] "Generic (PLEG): container finished" podID="bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90" containerID="1bc794743415d2966906ae97816571bc358beed744d3a9b5f586eabb3381dad6" exitCode=0
Sep 04 21:54:14.488482 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:14.488456    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzjmv" event={"ID":"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90","Type":"ContainerDied","Data":"1bc794743415d2966906ae97816571bc358beed744d3a9b5f586eabb3381dad6"}
Sep 04 21:54:16.348474 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:16.348438    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:16.348985 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:16.348488    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:16.348985 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:16.348605    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:16.348985 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:16.348685    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:54:16.349193 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:16.349170    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:54:16.349281 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:16.349247    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:54:18.347151 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:18.347109    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:18.347631 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:18.347111    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:18.347631 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:18.347242    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:18.347631 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:18.347237    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:54:18.347631 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:18.347318    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:54:18.347631 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:18.347392    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:54:20.088273 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:20.088045    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs\") pod \"network-metrics-daemon-46fbr\" (UID: \"eadd93c2-4a94-4046-9924-6e342406d1f8\") " pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:20.088643 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:20.088314    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret\") pod \"global-pull-secret-syncer-p7b77\" (UID: \"8fcc2326-c662-4e67-bdaf-60a54cce4235\") " pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:20.088643 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:20.088200    2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:54:20.088643 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:20.088357    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhjmp\" (UniqueName: \"kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp\") pod \"network-check-target-dflhd\" (UID: \"10da7e37-b753-4b5f-93f5-07e7a48ba85b\") " pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:20.088643 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:20.088440    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs podName:eadd93c2-4a94-4046-9924-6e342406d1f8 nodeName:}" failed. No retries permitted until 2026-09-04 21:54:52.088403369 +0000 UTC m=+66.404970105 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs") pod "network-metrics-daemon-46fbr" (UID: "eadd93c2-4a94-4046-9924-6e342406d1f8") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 04 21:54:20.088643 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:20.088499    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 04 21:54:20.088643 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:20.088505    2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 04 21:54:20.088643 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:20.088517    2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 04 21:54:20.088643 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:20.088529    2557 projected.go:196] Error preparing data for projected volume kube-api-access-fhjmp for pod openshift-network-diagnostics/network-check-target-dflhd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:54:20.088643 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:20.088566    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret podName:8fcc2326-c662-4e67-bdaf-60a54cce4235 nodeName:}" failed. No retries permitted until 2026-09-04 21:54:52.088551564 +0000 UTC m=+66.405118280 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret") pod "global-pull-secret-syncer-p7b77" (UID: "8fcc2326-c662-4e67-bdaf-60a54cce4235") : object "kube-system"/"original-pull-secret" not registered
Sep 04 21:54:20.088643 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:20.088582    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp podName:10da7e37-b753-4b5f-93f5-07e7a48ba85b nodeName:}" failed. No retries permitted until 2026-09-04 21:54:52.088574826 +0000 UTC m=+66.405141543 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fhjmp" (UniqueName: "kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp") pod "network-check-target-dflhd" (UID: "10da7e37-b753-4b5f-93f5-07e7a48ba85b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 04 21:54:20.346816 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:20.346776    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:20.347043 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:20.346920    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:20.347043 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:20.346931    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:54:20.347154 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:20.347042    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:54:20.347154 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:20.347091    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:20.347249 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:20.347166    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:54:20.507111 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:20.507073    2557 generic.go:356] "Generic (PLEG): container finished" podID="bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90" containerID="c7093a2a84f632d52919c04bc802c6ed7a58baabd824a2e5dd98a87eaf08d082" exitCode=0
Sep 04 21:54:20.507111 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:20.507117    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzjmv" event={"ID":"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90","Type":"ContainerDied","Data":"c7093a2a84f632d52919c04bc802c6ed7a58baabd824a2e5dd98a87eaf08d082"}
Sep 04 21:54:21.512363 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:21.512324    2557 generic.go:356] "Generic (PLEG): container finished" podID="bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90" containerID="e709f5f86d2700f27b59c5a5892f7cbf031b0f15a096ed3acd95704b1a9aeb04" exitCode=0
Sep 04 21:54:21.512919 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:21.512397    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzjmv" event={"ID":"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90","Type":"ContainerDied","Data":"e709f5f86d2700f27b59c5a5892f7cbf031b0f15a096ed3acd95704b1a9aeb04"}
Sep 04 21:54:22.347039 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:22.347004    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:22.347227 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:22.347004    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:22.347227 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:22.347127    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:22.347227 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:22.347128    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:54:22.347227 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:22.347188    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:54:22.347390 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:22.347271    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:54:22.517842 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:22.517804    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzjmv" event={"ID":"bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90","Type":"ContainerStarted","Data":"d39d87720219b0df06db685227fadae6c829b2a77576b70ee2291d971af17b4c"}
Sep 04 21:54:22.541964 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:22.541906    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dzjmv" podStartSLOduration=7.320814302 podStartE2EDuration="36.541888948s" totalImagesPullingTime="29.221074646s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:22.540713985 +0000 UTC m=+36.857280723" watchObservedRunningTime="2026-09-04 21:54:22.541888948 +0000 UTC m=+36.858455686"
Sep 04 21:54:24.346609 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:24.346572    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:24.347105 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:24.346645    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:24.347105 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:24.346793    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:54:24.347105 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:24.346572    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:24.347105 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:24.346856    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:54:24.347105 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:24.346890    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:54:26.348044 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:26.348002    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:26.348593 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:26.348096    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:54:26.348593 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:26.348131    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:26.348593 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:26.348223    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:54:26.348593 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:26.348242    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:26.348593 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:26.348289    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:54:28.346987 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:28.346948    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:28.347370 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:28.346950    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:28.347370 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:28.347061    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:54:28.347370 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:28.347076    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:28.347370 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:28.347157    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:54:28.347370 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:28.347225    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:54:30.349655 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:30.349624    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:30.350050 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:30.349665    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:30.350050 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:30.349716    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:30.350050 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:30.349731    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:54:30.350050 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:30.349783    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:54:30.350050 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:30.349865    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:54:32.346532 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:32.346497    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:32.346532 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:32.346498    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:32.347048 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:32.346508    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:32.347048 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:32.346608    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:54:32.347048 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:32.346688    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:54:32.347048 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:32.346758    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:54:34.346716 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:34.346679    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:34.346716 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:34.346712    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:34.347135 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:34.346679    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:34.347135 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:34.346794    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-46fbr" podUID="eadd93c2-4a94-4046-9924-6e342406d1f8"
Sep 04 21:54:34.347135 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:34.346860    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p7b77" podUID="8fcc2326-c662-4e67-bdaf-60a54cce4235"
Sep 04 21:54:34.347135 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:34.346949    2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dflhd" podUID="10da7e37-b753-4b5f-93f5-07e7a48ba85b"
Sep 04 21:54:35.939682 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:35.939599    2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-141.ec2.internal" event="NodeReady"
Sep 04 21:54:35.940096 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:35.939740    2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 04 21:54:35.981951 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:35.981915    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-86f655765c-vdjrp"]
Sep 04 21:54:36.007934 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.007906    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86f655765c-vdjrp"]
Sep 04 21:54:36.007934 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.007940    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bc7b4"]
Sep 04 21:54:36.008159 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.008024    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.014464 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.014338    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 04 21:54:36.014464 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.014350    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 04 21:54:36.014607 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.014550    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vfckr\""
Sep 04 21:54:36.015078 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.015048    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 04 21:54:36.023094 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.023069    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 04 21:54:36.032060 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.032024    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-66fwd"]
Sep 04 21:54:36.032244 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.032156    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bc7b4"
Sep 04 21:54:36.034523 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.034501    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 04 21:54:36.034783 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.034768    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5hzgq\""
Sep 04 21:54:36.035252 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.035238    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 04 21:54:36.052013 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.051985    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bc7b4"]
Sep 04 21:54:36.052167 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.052022    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jsnbb"]
Sep 04 21:54:36.052167 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.052099    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-66fwd"
Sep 04 21:54:36.054562 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.054527    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 04 21:54:36.054686 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.054579    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 04 21:54:36.054686 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.054582    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 04 21:54:36.054942 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.054925    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dmz5v\""
Sep 04 21:54:36.078179 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.078145    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-66fwd"]
Sep 04 21:54:36.078179 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.078179    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jsnbb"]
Sep 04 21:54:36.078374 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.078250    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jsnbb"
Sep 04 21:54:36.080777 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.080756    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 04 21:54:36.080905 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.080792    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 04 21:54:36.081036 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.081020    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 04 21:54:36.081294 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.081280    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 04 21:54:36.081494 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.081481    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9gp7g\""
Sep 04 21:54:36.197293 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.197202    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-image-registry-private-configuration\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.197293 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.197242    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv69w\" (UniqueName: \"kubernetes.io/projected/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-kube-api-access-tv69w\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.197293 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.197272    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/07654df4-9b51-4323-ad80-879b19f73dfa-crio-socket\") pod \"insights-runtime-extractor-jsnbb\" (UID: \"07654df4-9b51-4323-ad80-879b19f73dfa\") " pod="openshift-insights/insights-runtime-extractor-jsnbb"
Sep 04 21:54:36.197540 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.197333    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhk4l\" (UniqueName: \"kubernetes.io/projected/07654df4-9b51-4323-ad80-879b19f73dfa-kube-api-access-dhk4l\") pod \"insights-runtime-extractor-jsnbb\" (UID: \"07654df4-9b51-4323-ad80-879b19f73dfa\") " pod="openshift-insights/insights-runtime-extractor-jsnbb"
Sep 04 21:54:36.197540 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.197374    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h5rc\" (UniqueName: \"kubernetes.io/projected/3c09d3a4-17fe-4fda-b7e7-723ad50bbe14-kube-api-access-6h5rc\") pod \"dns-default-bc7b4\" (UID: \"3c09d3a4-17fe-4fda-b7e7-723ad50bbe14\") " pod="openshift-dns/dns-default-bc7b4"
Sep 04 21:54:36.197540 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.197433    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbm9r\" (UniqueName: \"kubernetes.io/projected/59f90e70-25a7-4c3d-9536-03a9bb65e69b-kube-api-access-lbm9r\") pod \"ingress-canary-66fwd\" (UID: \"59f90e70-25a7-4c3d-9536-03a9bb65e69b\") " pod="openshift-ingress-canary/ingress-canary-66fwd"
Sep 04 21:54:36.197540 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.197459    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-trusted-ca\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.197540 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.197478    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-installation-pull-secrets\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.197540 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.197493    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c09d3a4-17fe-4fda-b7e7-723ad50bbe14-tmp-dir\") pod \"dns-default-bc7b4\" (UID: \"3c09d3a4-17fe-4fda-b7e7-723ad50bbe14\") " pod="openshift-dns/dns-default-bc7b4"
Sep 04 21:54:36.197767 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.197546    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-bound-sa-token\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.197767 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.197590    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-registry-certificates\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.197767 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.197607    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-registry-tls\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.197767 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.197627    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3c09d3a4-17fe-4fda-b7e7-723ad50bbe14-config-volume\") pod \"dns-default-bc7b4\" (UID: \"3c09d3a4-17fe-4fda-b7e7-723ad50bbe14\") " pod="openshift-dns/dns-default-bc7b4"
Sep 04 21:54:36.197767 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.197646    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59f90e70-25a7-4c3d-9536-03a9bb65e69b-cert\") pod \"ingress-canary-66fwd\" (UID: \"59f90e70-25a7-4c3d-9536-03a9bb65e69b\") " pod="openshift-ingress-canary/ingress-canary-66fwd"
Sep 04 21:54:36.197767 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.197674    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/07654df4-9b51-4323-ad80-879b19f73dfa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jsnbb\" (UID: \"07654df4-9b51-4323-ad80-879b19f73dfa\") " pod="openshift-insights/insights-runtime-extractor-jsnbb"
Sep 04 21:54:36.197767 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.197691    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c09d3a4-17fe-4fda-b7e7-723ad50bbe14-metrics-tls\") pod \"dns-default-bc7b4\" (UID: \"3c09d3a4-17fe-4fda-b7e7-723ad50bbe14\") " pod="openshift-dns/dns-default-bc7b4"
Sep 04 21:54:36.197767 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.197730    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/07654df4-9b51-4323-ad80-879b19f73dfa-data-volume\") pod \"insights-runtime-extractor-jsnbb\" (UID: \"07654df4-9b51-4323-ad80-879b19f73dfa\") " pod="openshift-insights/insights-runtime-extractor-jsnbb"
Sep 04 21:54:36.197767 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.197748    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-ca-trust-extracted\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.197767 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.197767    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/07654df4-9b51-4323-ad80-879b19f73dfa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jsnbb\" (UID: \"07654df4-9b51-4323-ad80-879b19f73dfa\") " pod="openshift-insights/insights-runtime-extractor-jsnbb"
Sep 04 21:54:36.298988 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.298951    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/07654df4-9b51-4323-ad80-879b19f73dfa-data-volume\") pod \"insights-runtime-extractor-jsnbb\" (UID: \"07654df4-9b51-4323-ad80-879b19f73dfa\") " pod="openshift-insights/insights-runtime-extractor-jsnbb"
Sep 04 21:54:36.298988 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.298989    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-ca-trust-extracted\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.299252 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.299009    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/07654df4-9b51-4323-ad80-879b19f73dfa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jsnbb\" (UID: \"07654df4-9b51-4323-ad80-879b19f73dfa\") " pod="openshift-insights/insights-runtime-extractor-jsnbb"
Sep 04 21:54:36.299252 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.299041    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-image-registry-private-configuration\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.299252 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.299058    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv69w\" (UniqueName: \"kubernetes.io/projected/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-kube-api-access-tv69w\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.299252 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.299080    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/07654df4-9b51-4323-ad80-879b19f73dfa-crio-socket\") pod \"insights-runtime-extractor-jsnbb\" (UID: \"07654df4-9b51-4323-ad80-879b19f73dfa\") " pod="openshift-insights/insights-runtime-extractor-jsnbb"
Sep 04 21:54:36.299252 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.299109    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhk4l\" (UniqueName: \"kubernetes.io/projected/07654df4-9b51-4323-ad80-879b19f73dfa-kube-api-access-dhk4l\") pod \"insights-runtime-extractor-jsnbb\" (UID: \"07654df4-9b51-4323-ad80-879b19f73dfa\") " pod="openshift-insights/insights-runtime-extractor-jsnbb"
Sep 04 21:54:36.299252 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.299132    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h5rc\" (UniqueName: \"kubernetes.io/projected/3c09d3a4-17fe-4fda-b7e7-723ad50bbe14-kube-api-access-6h5rc\") pod \"dns-default-bc7b4\" (UID: \"3c09d3a4-17fe-4fda-b7e7-723ad50bbe14\") " pod="openshift-dns/dns-default-bc7b4"
Sep 04 21:54:36.299252 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.299171    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbm9r\" (UniqueName: \"kubernetes.io/projected/59f90e70-25a7-4c3d-9536-03a9bb65e69b-kube-api-access-lbm9r\") pod \"ingress-canary-66fwd\" (UID: \"59f90e70-25a7-4c3d-9536-03a9bb65e69b\") " pod="openshift-ingress-canary/ingress-canary-66fwd"
Sep 04 21:54:36.299252 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.299202    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-trusted-ca\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.299252 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.299227    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-installation-pull-secrets\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.299252 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.299251    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c09d3a4-17fe-4fda-b7e7-723ad50bbe14-tmp-dir\") pod \"dns-default-bc7b4\" (UID: \"3c09d3a4-17fe-4fda-b7e7-723ad50bbe14\") " pod="openshift-dns/dns-default-bc7b4"
Sep 04 21:54:36.299757 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.299277    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-bound-sa-token\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.299757 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.299316    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-registry-certificates\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.299757 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.299314    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/07654df4-9b51-4323-ad80-879b19f73dfa-crio-socket\") pod \"insights-runtime-extractor-jsnbb\" (UID: \"07654df4-9b51-4323-ad80-879b19f73dfa\") " pod="openshift-insights/insights-runtime-extractor-jsnbb"
Sep 04 21:54:36.299757 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.299340    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-registry-tls\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.299757 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.299383    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/07654df4-9b51-4323-ad80-879b19f73dfa-data-volume\") pod \"insights-runtime-extractor-jsnbb\" (UID: \"07654df4-9b51-4323-ad80-879b19f73dfa\") " pod="openshift-insights/insights-runtime-extractor-jsnbb"
Sep 04 21:54:36.299757 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.299388    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3c09d3a4-17fe-4fda-b7e7-723ad50bbe14-config-volume\") pod \"dns-default-bc7b4\" (UID: \"3c09d3a4-17fe-4fda-b7e7-723ad50bbe14\") " pod="openshift-dns/dns-default-bc7b4"
Sep 04 21:54:36.299757 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.299457    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-ca-trust-extracted\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.299757 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.299485    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59f90e70-25a7-4c3d-9536-03a9bb65e69b-cert\") pod \"ingress-canary-66fwd\" (UID: \"59f90e70-25a7-4c3d-9536-03a9bb65e69b\") " pod="openshift-ingress-canary/ingress-canary-66fwd"
Sep 04 21:54:36.299757 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.299514    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/07654df4-9b51-4323-ad80-879b19f73dfa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jsnbb\" (UID: \"07654df4-9b51-4323-ad80-879b19f73dfa\") " pod="openshift-insights/insights-runtime-extractor-jsnbb"
Sep 04 21:54:36.299757 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.299542    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c09d3a4-17fe-4fda-b7e7-723ad50bbe14-metrics-tls\") pod \"dns-default-bc7b4\" (UID: \"3c09d3a4-17fe-4fda-b7e7-723ad50bbe14\") " pod="openshift-dns/dns-default-bc7b4"
Sep 04 21:54:36.300263 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.299766    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c09d3a4-17fe-4fda-b7e7-723ad50bbe14-tmp-dir\") pod \"dns-default-bc7b4\" (UID: \"3c09d3a4-17fe-4fda-b7e7-723ad50bbe14\") " pod="openshift-dns/dns-default-bc7b4"
Sep 04 21:54:36.300263 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.300018    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3c09d3a4-17fe-4fda-b7e7-723ad50bbe14-config-volume\") pod \"dns-default-bc7b4\" (UID: \"3c09d3a4-17fe-4fda-b7e7-723ad50bbe14\") " pod="openshift-dns/dns-default-bc7b4"
Sep 04 21:54:36.300372 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.300346    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-trusted-ca\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.301050 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.301025    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-registry-certificates\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.304403 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.304376    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3c09d3a4-17fe-4fda-b7e7-723ad50bbe14-metrics-tls\") pod \"dns-default-bc7b4\" (UID: \"3c09d3a4-17fe-4fda-b7e7-723ad50bbe14\") " pod="openshift-dns/dns-default-bc7b4"
Sep 04 21:54:36.304536 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.304432    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-installation-pull-secrets\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.304536 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.304384    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/07654df4-9b51-4323-ad80-879b19f73dfa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jsnbb\" (UID: \"07654df4-9b51-4323-ad80-879b19f73dfa\") " pod="openshift-insights/insights-runtime-extractor-jsnbb"
Sep 04 21:54:36.304536 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.304448    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/59f90e70-25a7-4c3d-9536-03a9bb65e69b-cert\") pod \"ingress-canary-66fwd\" (UID: \"59f90e70-25a7-4c3d-9536-03a9bb65e69b\") " pod="openshift-ingress-canary/ingress-canary-66fwd"
Sep 04 21:54:36.304536 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.304474    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-registry-tls\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.304748 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.304730    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-image-registry-private-configuration\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.312001 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.311974    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/07654df4-9b51-4323-ad80-879b19f73dfa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jsnbb\" (UID: \"07654df4-9b51-4323-ad80-879b19f73dfa\") " pod="openshift-insights/insights-runtime-extractor-jsnbb"
Sep 04 21:54:36.312156 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.312131    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h5rc\" (UniqueName: \"kubernetes.io/projected/3c09d3a4-17fe-4fda-b7e7-723ad50bbe14-kube-api-access-6h5rc\") pod \"dns-default-bc7b4\" (UID: \"3c09d3a4-17fe-4fda-b7e7-723ad50bbe14\") " pod="openshift-dns/dns-default-bc7b4"
Sep 04 21:54:36.313053 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.313027    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbm9r\" (UniqueName: \"kubernetes.io/projected/59f90e70-25a7-4c3d-9536-03a9bb65e69b-kube-api-access-lbm9r\") pod \"ingress-canary-66fwd\" (UID: \"59f90e70-25a7-4c3d-9536-03a9bb65e69b\") " pod="openshift-ingress-canary/ingress-canary-66fwd"
Sep 04 21:54:36.314677 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.314657    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv69w\" (UniqueName: \"kubernetes.io/projected/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-kube-api-access-tv69w\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.314760 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.314746    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d-bound-sa-token\") pod \"image-registry-86f655765c-vdjrp\" (UID: \"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d\") " pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.314798 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.314758    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhk4l\" (UniqueName: \"kubernetes.io/projected/07654df4-9b51-4323-ad80-879b19f73dfa-kube-api-access-dhk4l\") pod \"insights-runtime-extractor-jsnbb\" (UID: \"07654df4-9b51-4323-ad80-879b19f73dfa\") " pod="openshift-insights/insights-runtime-extractor-jsnbb"
Sep 04 21:54:36.317712 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.317692    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:36.340401 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.340374    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bc7b4"
Sep 04 21:54:36.347811 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.347779    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:36.348003 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.347868    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:36.348003 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.347912    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:36.351995 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.351971    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 04 21:54:36.352375 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.352354    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 04 21:54:36.352780 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.352763    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 04 21:54:36.352884 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.352830    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 04 21:54:36.352884 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.352871    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-268hq\""
Sep 04 21:54:36.352986 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.352896    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xzsq2\""
Sep 04 21:54:36.362549 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.362517    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-66fwd"
Sep 04 21:54:36.392840 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.392790    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jsnbb"
Sep 04 21:54:36.651530 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.651464    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bc7b4"]
Sep 04 21:54:36.655955 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:54:36.655923    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c09d3a4_17fe_4fda_b7e7_723ad50bbe14.slice/crio-a50dcf1d7e4df5ec69a2607ebef4a54734725fc34d40abff89518921452ef947 WatchSource:0}: Error finding container a50dcf1d7e4df5ec69a2607ebef4a54734725fc34d40abff89518921452ef947: Status 404 returned error can't find the container with id a50dcf1d7e4df5ec69a2607ebef4a54734725fc34d40abff89518921452ef947
Sep 04 21:54:36.657807 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.657776    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-86f655765c-vdjrp"]
Sep 04 21:54:36.661517 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:54:36.661483    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfb97231_da9f_4aec_b5c5_e77b3a0c0c8d.slice/crio-8651a4fca5eab938245c40fd7925f8768b45ec17e96443ee56ade377153d167d WatchSource:0}: Error finding container 8651a4fca5eab938245c40fd7925f8768b45ec17e96443ee56ade377153d167d: Status 404 returned error can't find the container with id 8651a4fca5eab938245c40fd7925f8768b45ec17e96443ee56ade377153d167d
Sep 04 21:54:36.671705 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.671666    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-66fwd"]
Sep 04 21:54:36.674903 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:54:36.674878    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59f90e70_25a7_4c3d_9536_03a9bb65e69b.slice/crio-b28e0977102668317413cb3b1f62aa2d4899e910159f15e028f793d4e80c68af WatchSource:0}: Error finding container b28e0977102668317413cb3b1f62aa2d4899e910159f15e028f793d4e80c68af: Status 404 returned error can't find the container with id b28e0977102668317413cb3b1f62aa2d4899e910159f15e028f793d4e80c68af
Sep 04 21:54:36.700871 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:36.700838    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jsnbb"]
Sep 04 21:54:36.704311 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:54:36.704286    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07654df4_9b51_4323_ad80_879b19f73dfa.slice/crio-9cf0ee1d153e9286780b39d4a6d015af6f5c8d639a46fbb837d9c7de655facf2 WatchSource:0}: Error finding container 9cf0ee1d153e9286780b39d4a6d015af6f5c8d639a46fbb837d9c7de655facf2: Status 404 returned error can't find the container with id 9cf0ee1d153e9286780b39d4a6d015af6f5c8d639a46fbb837d9c7de655facf2
Sep 04 21:54:37.556737 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:37.556695    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jsnbb" event={"ID":"07654df4-9b51-4323-ad80-879b19f73dfa","Type":"ContainerStarted","Data":"4fc540a497f776ed1658a39a7ca91ca87a55e64afecaf29abb35cbcb4fb32bb1"}
Sep 04 21:54:37.557306 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:37.556813    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jsnbb" event={"ID":"07654df4-9b51-4323-ad80-879b19f73dfa","Type":"ContainerStarted","Data":"9cf0ee1d153e9286780b39d4a6d015af6f5c8d639a46fbb837d9c7de655facf2"}
Sep 04 21:54:37.558458 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:37.558405    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-66fwd" event={"ID":"59f90e70-25a7-4c3d-9536-03a9bb65e69b","Type":"ContainerStarted","Data":"b28e0977102668317413cb3b1f62aa2d4899e910159f15e028f793d4e80c68af"}
Sep 04 21:54:37.560001 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:37.559973    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bc7b4" event={"ID":"3c09d3a4-17fe-4fda-b7e7-723ad50bbe14","Type":"ContainerStarted","Data":"a50dcf1d7e4df5ec69a2607ebef4a54734725fc34d40abff89518921452ef947"}
Sep 04 21:54:37.562026 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:37.561993    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86f655765c-vdjrp" event={"ID":"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d","Type":"ContainerStarted","Data":"878805a615ceee72b56f9b67dc3b8b716600d14034192fa66e237e8bfc4ac8bc"}
Sep 04 21:54:37.562146 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:37.562080    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-86f655765c-vdjrp" event={"ID":"cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d","Type":"ContainerStarted","Data":"8651a4fca5eab938245c40fd7925f8768b45ec17e96443ee56ade377153d167d"}
Sep 04 21:54:37.562146 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:37.562138    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:37.583940 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:37.583892    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-86f655765c-vdjrp" podStartSLOduration=3.5838758779999997 podStartE2EDuration="3.583875878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:54:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:37.583192535 +0000 UTC m=+51.899759274" watchObservedRunningTime="2026-09-04 21:54:37.583875878 +0000 UTC m=+51.900442610"
Sep 04 21:54:38.568090 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:38.567998    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jsnbb" event={"ID":"07654df4-9b51-4323-ad80-879b19f73dfa","Type":"ContainerStarted","Data":"98ba42bc3c48604e4ee74bb409e8ea71b912844986ce29f4f667199bebf14ea7"}
Sep 04 21:54:40.577101 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:40.577063    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jsnbb" event={"ID":"07654df4-9b51-4323-ad80-879b19f73dfa","Type":"ContainerStarted","Data":"981ccb5969c14589b51f419d0a3ab7f2dd5a2d401b9e6329d8f8b79ab7f3213c"}
Sep 04 21:54:40.578620 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:40.578597    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-66fwd" event={"ID":"59f90e70-25a7-4c3d-9536-03a9bb65e69b","Type":"ContainerStarted","Data":"4c003bbb7b4c8887f487becbb43450dfb83f0f70ab1efa09112dbeb541ba4f1f"}
Sep 04 21:54:40.580361 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:40.580332    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bc7b4" event={"ID":"3c09d3a4-17fe-4fda-b7e7-723ad50bbe14","Type":"ContainerStarted","Data":"de1c57424db71c632f1266df56b17923a38c2c2ef48c942b60b5f3cf2724426e"}
Sep 04 21:54:40.580493 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:40.580381    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bc7b4"
Sep 04 21:54:40.580493 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:40.580398    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bc7b4" event={"ID":"3c09d3a4-17fe-4fda-b7e7-723ad50bbe14","Type":"ContainerStarted","Data":"e61066017946ec12c2fb319477226c636cac0062f80111b4ccd089b34275b7c7"}
Sep 04 21:54:40.598481 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:40.596385    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jsnbb" podStartSLOduration=1.863181105 podStartE2EDuration="4.596366488s" totalImagesPullingTime="2.733185383s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:54:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:40.595632618 +0000 UTC m=+54.912199361" watchObservedRunningTime="2026-09-04 21:54:40.596366488 +0000 UTC m=+54.912933227"
Sep 04 21:54:40.613158 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:40.613076    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-66fwd" podStartSLOduration=2.683870957 podStartE2EDuration="5.613056533s" totalImagesPullingTime="2.929185576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:54:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:40.612740561 +0000 UTC m=+54.929307299" watchObservedRunningTime="2026-09-04 21:54:40.613056533 +0000 UTC m=+54.929623272"
Sep 04 21:54:40.630541 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:40.630483    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bc7b4" podStartSLOduration=2.703120633 podStartE2EDuration="5.630465382s" totalImagesPullingTime="2.927344749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:54:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:40.629498915 +0000 UTC m=+54.946065655" watchObservedRunningTime="2026-09-04 21:54:40.630465382 +0000 UTC m=+54.947032163"
Sep 04 21:54:45.502817 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:45.502783    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ss72b"
Sep 04 21:54:48.191240 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.191196    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-fsbsn"]
Sep 04 21:54:48.194229 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.194209    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fsbsn"
Sep 04 21:54:48.198179 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.198154    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 04 21:54:48.198776 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.198756    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 04 21:54:48.198885 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.198757    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 04 21:54:48.199142 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.199125    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 04 21:54:48.199142 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.199134    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-bpst8\""
Sep 04 21:54:48.199262 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.199128    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 04 21:54:48.218993 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.218961    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-fsbsn"]
Sep 04 21:54:48.220492 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.220462    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"]
Sep 04 21:54:48.222489 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.222469    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"
Sep 04 21:54:48.225151 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.225130    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 04 21:54:48.225661 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.225639    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-qhd72\""
Sep 04 21:54:48.226043 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.226022    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 04 21:54:48.226183 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.226025    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 04 21:54:48.243560 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.243523    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"]
Sep 04 21:54:48.253406 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.253376    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-m25wx"]
Sep 04 21:54:48.255532 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.255511    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.258041 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.258017    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 04 21:54:48.258200 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.258182    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 04 21:54:48.258270 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.258213    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-2cnfl\""
Sep 04 21:54:48.258355 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.258337    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 04 21:54:48.289193 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.289151    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd9483eb-f008-4171-9233-2724cb1ee215-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-qwtkd\" (UID: \"dd9483eb-f008-4171-9233-2724cb1ee215\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"
Sep 04 21:54:48.289193 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.289186    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dd9483eb-f008-4171-9233-2724cb1ee215-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-qwtkd\" (UID: \"dd9483eb-f008-4171-9233-2724cb1ee215\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"
Sep 04 21:54:48.289404 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.289215    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6420905f-c55a-4e5a-aa93-b36775b74d93-sys\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.289404 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.289266    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f435c58-0d37-40d1-8bf3-4cad62f0944d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-fsbsn\" (UID: \"1f435c58-0d37-40d1-8bf3-4cad62f0944d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fsbsn"
Sep 04 21:54:48.289404 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.289319    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6420905f-c55a-4e5a-aa93-b36775b74d93-metrics-client-ca\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.289404 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.289369    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6420905f-c55a-4e5a-aa93-b36775b74d93-node-exporter-accelerators-collector-config\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.289404 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.289390    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f435c58-0d37-40d1-8bf3-4cad62f0944d-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-fsbsn\" (UID: \"1f435c58-0d37-40d1-8bf3-4cad62f0944d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fsbsn"
Sep 04 21:54:48.289585 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.289410    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwlpv\" (UniqueName: \"kubernetes.io/projected/6420905f-c55a-4e5a-aa93-b36775b74d93-kube-api-access-xwlpv\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.289585 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.289480    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6420905f-c55a-4e5a-aa93-b36775b74d93-node-exporter-textfile\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.289585 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.289522    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvzbj\" (UniqueName: \"kubernetes.io/projected/dd9483eb-f008-4171-9233-2724cb1ee215-kube-api-access-pvzbj\") pod \"kube-state-metrics-74cb57df7-qwtkd\" (UID: \"dd9483eb-f008-4171-9233-2724cb1ee215\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"
Sep 04 21:54:48.289585 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.289540    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6420905f-c55a-4e5a-aa93-b36775b74d93-node-exporter-tls\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.289585 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.289563    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/dd9483eb-f008-4171-9233-2724cb1ee215-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-qwtkd\" (UID: \"dd9483eb-f008-4171-9233-2724cb1ee215\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"
Sep 04 21:54:48.289585 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.289581    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6420905f-c55a-4e5a-aa93-b36775b74d93-node-exporter-wtmp\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.289767 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.289626    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6420905f-c55a-4e5a-aa93-b36775b74d93-root\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.289767 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.289669    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f435c58-0d37-40d1-8bf3-4cad62f0944d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-fsbsn\" (UID: \"1f435c58-0d37-40d1-8bf3-4cad62f0944d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fsbsn"
Sep 04 21:54:48.289767 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.289684    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cgtj\" (UniqueName: \"kubernetes.io/projected/1f435c58-0d37-40d1-8bf3-4cad62f0944d-kube-api-access-7cgtj\") pod \"openshift-state-metrics-657fd6c54-fsbsn\" (UID: \"1f435c58-0d37-40d1-8bf3-4cad62f0944d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fsbsn"
Sep 04 21:54:48.289767 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.289710    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd9483eb-f008-4171-9233-2724cb1ee215-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-qwtkd\" (UID: \"dd9483eb-f008-4171-9233-2724cb1ee215\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"
Sep 04 21:54:48.289767 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.289730    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6420905f-c55a-4e5a-aa93-b36775b74d93-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.289767 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.289752    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/dd9483eb-f008-4171-9233-2724cb1ee215-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-qwtkd\" (UID: \"dd9483eb-f008-4171-9233-2724cb1ee215\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"
Sep 04 21:54:48.390556 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.390516    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/dd9483eb-f008-4171-9233-2724cb1ee215-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-qwtkd\" (UID: \"dd9483eb-f008-4171-9233-2724cb1ee215\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"
Sep 04 21:54:48.390556 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.390558    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd9483eb-f008-4171-9233-2724cb1ee215-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-qwtkd\" (UID: \"dd9483eb-f008-4171-9233-2724cb1ee215\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"
Sep 04 21:54:48.390819 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.390579    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dd9483eb-f008-4171-9233-2724cb1ee215-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-qwtkd\" (UID: \"dd9483eb-f008-4171-9233-2724cb1ee215\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"
Sep 04 21:54:48.390819 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:48.390681    2557 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 04 21:54:48.390819 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.390724    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6420905f-c55a-4e5a-aa93-b36775b74d93-sys\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.390819 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.390689    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6420905f-c55a-4e5a-aa93-b36775b74d93-sys\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.390819 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:48.390768    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dd9483eb-f008-4171-9233-2724cb1ee215-kube-state-metrics-tls podName:dd9483eb-f008-4171-9233-2724cb1ee215 nodeName:}" failed. No retries permitted until 2026-09-04 21:54:48.890745503 +0000 UTC m=+63.207312233 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/dd9483eb-f008-4171-9233-2724cb1ee215-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-qwtkd" (UID: "dd9483eb-f008-4171-9233-2724cb1ee215") : secret "kube-state-metrics-tls" not found
Sep 04 21:54:48.391083 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.390817    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f435c58-0d37-40d1-8bf3-4cad62f0944d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-fsbsn\" (UID: \"1f435c58-0d37-40d1-8bf3-4cad62f0944d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fsbsn"
Sep 04 21:54:48.391083 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.390849    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6420905f-c55a-4e5a-aa93-b36775b74d93-metrics-client-ca\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.391083 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.390885    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6420905f-c55a-4e5a-aa93-b36775b74d93-node-exporter-accelerators-collector-config\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.391083 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.390916    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f435c58-0d37-40d1-8bf3-4cad62f0944d-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-fsbsn\" (UID: \"1f435c58-0d37-40d1-8bf3-4cad62f0944d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fsbsn"
Sep 04 21:54:48.391083 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.390950    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwlpv\" (UniqueName: \"kubernetes.io/projected/6420905f-c55a-4e5a-aa93-b36775b74d93-kube-api-access-xwlpv\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.391083 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.390987    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6420905f-c55a-4e5a-aa93-b36775b74d93-node-exporter-textfile\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.391083 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.391037    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvzbj\" (UniqueName: \"kubernetes.io/projected/dd9483eb-f008-4171-9233-2724cb1ee215-kube-api-access-pvzbj\") pod \"kube-state-metrics-74cb57df7-qwtkd\" (UID: \"dd9483eb-f008-4171-9233-2724cb1ee215\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"
Sep 04 21:54:48.391083 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.391062    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6420905f-c55a-4e5a-aa93-b36775b74d93-node-exporter-tls\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.391496 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.391098    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/dd9483eb-f008-4171-9233-2724cb1ee215-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-qwtkd\" (UID: \"dd9483eb-f008-4171-9233-2724cb1ee215\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"
Sep 04 21:54:48.391496 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.391122    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6420905f-c55a-4e5a-aa93-b36775b74d93-node-exporter-wtmp\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.391496 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.391152    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6420905f-c55a-4e5a-aa93-b36775b74d93-root\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.391496 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.391206    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f435c58-0d37-40d1-8bf3-4cad62f0944d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-fsbsn\" (UID: \"1f435c58-0d37-40d1-8bf3-4cad62f0944d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fsbsn"
Sep 04 21:54:48.391496 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.391232    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cgtj\" (UniqueName: \"kubernetes.io/projected/1f435c58-0d37-40d1-8bf3-4cad62f0944d-kube-api-access-7cgtj\") pod \"openshift-state-metrics-657fd6c54-fsbsn\" (UID: \"1f435c58-0d37-40d1-8bf3-4cad62f0944d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fsbsn"
Sep 04 21:54:48.391496 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.391274    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd9483eb-f008-4171-9233-2724cb1ee215-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-qwtkd\" (UID: \"dd9483eb-f008-4171-9233-2724cb1ee215\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"
Sep 04 21:54:48.391496 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.391312    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6420905f-c55a-4e5a-aa93-b36775b74d93-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.391496 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:48.391329    2557 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 04 21:54:48.391496 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:48.391399    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6420905f-c55a-4e5a-aa93-b36775b74d93-node-exporter-tls podName:6420905f-c55a-4e5a-aa93-b36775b74d93 nodeName:}" failed. No retries permitted until 2026-09-04 21:54:48.891380228 +0000 UTC m=+63.207946945 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/6420905f-c55a-4e5a-aa93-b36775b74d93-node-exporter-tls") pod "node-exporter-m25wx" (UID: "6420905f-c55a-4e5a-aa93-b36775b74d93") : secret "node-exporter-tls" not found
Sep 04 21:54:48.391496 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.390983    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/dd9483eb-f008-4171-9233-2724cb1ee215-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-qwtkd\" (UID: \"dd9483eb-f008-4171-9233-2724cb1ee215\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"
Sep 04 21:54:48.391986 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.391531    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6420905f-c55a-4e5a-aa93-b36775b74d93-node-exporter-accelerators-collector-config\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.391986 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.391565    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6420905f-c55a-4e5a-aa93-b36775b74d93-metrics-client-ca\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.391986 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.391602    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6420905f-c55a-4e5a-aa93-b36775b74d93-root\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.391986 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.391643    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f435c58-0d37-40d1-8bf3-4cad62f0944d-metrics-client-ca\") pod \"openshift-state-metrics-657fd6c54-fsbsn\" (UID: \"1f435c58-0d37-40d1-8bf3-4cad62f0944d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fsbsn"
Sep 04 21:54:48.392185 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.392055    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6420905f-c55a-4e5a-aa93-b36775b74d93-node-exporter-wtmp\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.392239 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.392213    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6420905f-c55a-4e5a-aa93-b36775b74d93-node-exporter-textfile\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.392369 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.392344    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/dd9483eb-f008-4171-9233-2724cb1ee215-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-qwtkd\" (UID: \"dd9483eb-f008-4171-9233-2724cb1ee215\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"
Sep 04 21:54:48.392678 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.392659    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd9483eb-f008-4171-9233-2724cb1ee215-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-qwtkd\" (UID: \"dd9483eb-f008-4171-9233-2724cb1ee215\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"
Sep 04 21:54:48.393823 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.393791    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/dd9483eb-f008-4171-9233-2724cb1ee215-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-qwtkd\" (UID: \"dd9483eb-f008-4171-9233-2724cb1ee215\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"
Sep 04 21:54:48.394059 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.394039    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6420905f-c55a-4e5a-aa93-b36775b74d93-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.394187 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.394166    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f435c58-0d37-40d1-8bf3-4cad62f0944d-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-657fd6c54-fsbsn\" (UID: \"1f435c58-0d37-40d1-8bf3-4cad62f0944d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fsbsn"
Sep 04 21:54:48.394434 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.394401    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1f435c58-0d37-40d1-8bf3-4cad62f0944d-openshift-state-metrics-tls\") pod \"openshift-state-metrics-657fd6c54-fsbsn\" (UID: \"1f435c58-0d37-40d1-8bf3-4cad62f0944d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fsbsn"
Sep 04 21:54:48.403371 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.403345    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cgtj\" (UniqueName: \"kubernetes.io/projected/1f435c58-0d37-40d1-8bf3-4cad62f0944d-kube-api-access-7cgtj\") pod \"openshift-state-metrics-657fd6c54-fsbsn\" (UID: \"1f435c58-0d37-40d1-8bf3-4cad62f0944d\") " pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fsbsn"
Sep 04 21:54:48.403512 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.403357    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvzbj\" (UniqueName: \"kubernetes.io/projected/dd9483eb-f008-4171-9233-2724cb1ee215-kube-api-access-pvzbj\") pod \"kube-state-metrics-74cb57df7-qwtkd\" (UID: \"dd9483eb-f008-4171-9233-2724cb1ee215\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"
Sep 04 21:54:48.404396 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.404372    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwlpv\" (UniqueName: \"kubernetes.io/projected/6420905f-c55a-4e5a-aa93-b36775b74d93-kube-api-access-xwlpv\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.504011 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.503964    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fsbsn"
Sep 04 21:54:48.777148 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.777047    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-657fd6c54-fsbsn"]
Sep 04 21:54:48.780869 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:54:48.780841    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f435c58_0d37_40d1_8bf3_4cad62f0944d.slice/crio-cd69c42b80cb5ac3ca0f539c2492504c5a9e2d27df8328733a641b7c97888587 WatchSource:0}: Error finding container cd69c42b80cb5ac3ca0f539c2492504c5a9e2d27df8328733a641b7c97888587: Status 404 returned error can't find the container with id cd69c42b80cb5ac3ca0f539c2492504c5a9e2d27df8328733a641b7c97888587
Sep 04 21:54:48.895056 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.895032    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd9483eb-f008-4171-9233-2724cb1ee215-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-qwtkd\" (UID: \"dd9483eb-f008-4171-9233-2724cb1ee215\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"
Sep 04 21:54:48.895153 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.895102    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6420905f-c55a-4e5a-aa93-b36775b74d93-node-exporter-tls\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:48.895211 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:48.895199    2557 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 04 21:54:48.895257 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:48.895248    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6420905f-c55a-4e5a-aa93-b36775b74d93-node-exporter-tls podName:6420905f-c55a-4e5a-aa93-b36775b74d93 nodeName:}" failed. No retries permitted until 2026-09-04 21:54:49.89523359 +0000 UTC m=+64.211800306 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/6420905f-c55a-4e5a-aa93-b36775b74d93-node-exporter-tls") pod "node-exporter-m25wx" (UID: "6420905f-c55a-4e5a-aa93-b36775b74d93") : secret "node-exporter-tls" not found
Sep 04 21:54:48.897388 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:48.897360    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/dd9483eb-f008-4171-9233-2724cb1ee215-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-qwtkd\" (UID: \"dd9483eb-f008-4171-9233-2724cb1ee215\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"
Sep 04 21:54:49.131827 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.131740    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"
Sep 04 21:54:49.279341 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.279309    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 04 21:54:49.282077 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.282056    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.286768 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.286740    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 04 21:54:49.286900 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.286778    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 04 21:54:49.286900 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.286796    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 04 21:54:49.286900 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.286813    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 04 21:54:49.286900 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.286860    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 04 21:54:49.286900 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.286866    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 04 21:54:49.286900 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.286812    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 04 21:54:49.286900 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.286813    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 04 21:54:49.287216 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.287003    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 04 21:54:49.287216 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.287040    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-fdmks\""
Sep 04 21:54:49.308487 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.308457    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 04 21:54:49.399550 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.399455    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.399550 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.399507    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f720a6df-15a0-41b0-8367-58de99e72753-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.399550 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.399529    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.399815 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.399618    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-web-config\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.399815 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.399657    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-config-volume\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.399815 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.399687    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.399815 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.399718    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f720a6df-15a0-41b0-8367-58de99e72753-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.399815 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.399748    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f720a6df-15a0-41b0-8367-58de99e72753-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.399815 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.399807    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f720a6df-15a0-41b0-8367-58de99e72753-config-out\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.400044 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.399831    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.400044 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.399870    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f720a6df-15a0-41b0-8367-58de99e72753-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.400044 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.399951    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhwkl\" (UniqueName: \"kubernetes.io/projected/f720a6df-15a0-41b0-8367-58de99e72753-kube-api-access-zhwkl\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.400044 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.399984    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.406087 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.405930    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd"]
Sep 04 21:54:49.407954 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:54:49.407931    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd9483eb_f008_4171_9233_2724cb1ee215.slice/crio-6bd0750d6ce78011063a84fe8b537f46385c9632fd59531a48a93c3bdb7d091a WatchSource:0}: Error finding container 6bd0750d6ce78011063a84fe8b537f46385c9632fd59531a48a93c3bdb7d091a: Status 404 returned error can't find the container with id 6bd0750d6ce78011063a84fe8b537f46385c9632fd59531a48a93c3bdb7d091a
Sep 04 21:54:49.500717 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.500682    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f720a6df-15a0-41b0-8367-58de99e72753-config-out\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.500717 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.500720    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.500961 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.500746    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f720a6df-15a0-41b0-8367-58de99e72753-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.500961 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.500799    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhwkl\" (UniqueName: \"kubernetes.io/projected/f720a6df-15a0-41b0-8367-58de99e72753-kube-api-access-zhwkl\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.500961 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.500830    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.500961 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:49.500862    2557 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 04 21:54:49.500961 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.500875    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.500961 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.500909    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f720a6df-15a0-41b0-8367-58de99e72753-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.500961 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:49.500926    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-secret-alertmanager-main-tls podName:f720a6df-15a0-41b0-8367-58de99e72753 nodeName:}" failed. No retries permitted until 2026-09-04 21:54:50.000908084 +0000 UTC m=+64.317474825 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "f720a6df-15a0-41b0-8367-58de99e72753") : secret "alertmanager-main-tls" not found
Sep 04 21:54:49.501259 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:54:49.500967    2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f720a6df-15a0-41b0-8367-58de99e72753-alertmanager-trusted-ca-bundle podName:f720a6df-15a0-41b0-8367-58de99e72753 nodeName:}" failed. No retries permitted until 2026-09-04 21:54:50.000946281 +0000 UTC m=+64.317513016 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/f720a6df-15a0-41b0-8367-58de99e72753-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "f720a6df-15a0-41b0-8367-58de99e72753") : configmap references non-existent config key: ca-bundle.crt
Sep 04 21:54:49.501259 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.501022    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.501259 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.501148    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-web-config\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.501259 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.501181    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-config-volume\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.501259 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.501213    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.501259 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.501230    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f720a6df-15a0-41b0-8367-58de99e72753-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.501259 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.501247    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f720a6df-15a0-41b0-8367-58de99e72753-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.501640 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.501275    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f720a6df-15a0-41b0-8367-58de99e72753-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.502379 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.502354    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f720a6df-15a0-41b0-8367-58de99e72753-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.503514 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.503489    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f720a6df-15a0-41b0-8367-58de99e72753-config-out\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.503806 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.503783    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.503944 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.503922    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-config-volume\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.504013 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.503930    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f720a6df-15a0-41b0-8367-58de99e72753-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.504162 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.504141    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.504320 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.504298    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.504634 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.504619    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-web-config\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.505237 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.505220    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.509656 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.509632    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhwkl\" (UniqueName: \"kubernetes.io/projected/f720a6df-15a0-41b0-8367-58de99e72753-kube-api-access-zhwkl\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:49.612795 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.612754    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fsbsn" event={"ID":"1f435c58-0d37-40d1-8bf3-4cad62f0944d","Type":"ContainerStarted","Data":"97f69ce6169a9962f9a540e40a1ce0e017294d40ca44ee9cc785f61b2090d2e7"}
Sep 04 21:54:49.612969 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.612824    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fsbsn" event={"ID":"1f435c58-0d37-40d1-8bf3-4cad62f0944d","Type":"ContainerStarted","Data":"311f4179502cb144d3712ba667347045b90b7322b2058d781691658774db99e0"}
Sep 04 21:54:49.612969 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.612850    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fsbsn" event={"ID":"1f435c58-0d37-40d1-8bf3-4cad62f0944d","Type":"ContainerStarted","Data":"cd69c42b80cb5ac3ca0f539c2492504c5a9e2d27df8328733a641b7c97888587"}
Sep 04 21:54:49.613894 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.613872    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd" event={"ID":"dd9483eb-f008-4171-9233-2724cb1ee215","Type":"ContainerStarted","Data":"6bd0750d6ce78011063a84fe8b537f46385c9632fd59531a48a93c3bdb7d091a"}
Sep 04 21:54:49.904604 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.904563    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6420905f-c55a-4e5a-aa93-b36775b74d93-node-exporter-tls\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:49.907365 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:49.907335    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6420905f-c55a-4e5a-aa93-b36775b74d93-node-exporter-tls\") pod \"node-exporter-m25wx\" (UID: \"6420905f-c55a-4e5a-aa93-b36775b74d93\") " pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:50.005053 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:50.005013    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:50.005053 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:50.005062    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f720a6df-15a0-41b0-8367-58de99e72753-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:50.005970 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:50.005941    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f720a6df-15a0-41b0-8367-58de99e72753-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:50.008072 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:50.008048    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f720a6df-15a0-41b0-8367-58de99e72753-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f720a6df-15a0-41b0-8367-58de99e72753\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:50.063975 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:50.063933    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m25wx"
Sep 04 21:54:50.190887 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:50.190810    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 04 21:54:50.227753 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:54:50.227720    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6420905f_c55a_4e5a_aa93_b36775b74d93.slice/crio-eccb8baf648b9da2269af9862483b00e5d518e2a4cb0908a24e2eb52a68aef44 WatchSource:0}: Error finding container eccb8baf648b9da2269af9862483b00e5d518e2a4cb0908a24e2eb52a68aef44: Status 404 returned error can't find the container with id eccb8baf648b9da2269af9862483b00e5d518e2a4cb0908a24e2eb52a68aef44
Sep 04 21:54:50.509072 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:50.509037    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 04 21:54:50.585539 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:50.585508    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bc7b4"
Sep 04 21:54:50.619961 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:50.619920    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m25wx" event={"ID":"6420905f-c55a-4e5a-aa93-b36775b74d93","Type":"ContainerStarted","Data":"eccb8baf648b9da2269af9862483b00e5d518e2a4cb0908a24e2eb52a68aef44"}
Sep 04 21:54:50.622859 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:50.622827    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fsbsn" event={"ID":"1f435c58-0d37-40d1-8bf3-4cad62f0944d","Type":"ContainerStarted","Data":"c55a0f3a2d0ff12d43249beaf953a71dfac9befd273613b8bd189599ec9d6cdb"}
Sep 04 21:54:50.643223 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:50.643166    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-657fd6c54-fsbsn" podStartSLOduration=1.2652966860000001 podStartE2EDuration="2.643142244s" totalImagesPullingTime="1.377845558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:50.641981835 +0000 UTC m=+64.958548575" watchObservedRunningTime="2026-09-04 21:54:50.643142244 +0000 UTC m=+64.959708983"
Sep 04 21:54:50.865225 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:54:50.865150    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf720a6df_15a0_41b0_8367_58de99e72753.slice/crio-143e66caa134dbd31b69c487ac497eded1f59663a92562ea4129d7a016062110 WatchSource:0}: Error finding container 143e66caa134dbd31b69c487ac497eded1f59663a92562ea4129d7a016062110: Status 404 returned error can't find the container with id 143e66caa134dbd31b69c487ac497eded1f59663a92562ea4129d7a016062110
Sep 04 21:54:51.629834 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:51.629787    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f720a6df-15a0-41b0-8367-58de99e72753","Type":"ContainerStarted","Data":"143e66caa134dbd31b69c487ac497eded1f59663a92562ea4129d7a016062110"}
Sep 04 21:54:51.633585 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:51.632450    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd" event={"ID":"dd9483eb-f008-4171-9233-2724cb1ee215","Type":"ContainerStarted","Data":"fe8bacd53b58e91887a4dba4316bbc5246dc19d42254e7fbbc75440fc60c2142"}
Sep 04 21:54:51.633585 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:51.632529    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd" event={"ID":"dd9483eb-f008-4171-9233-2724cb1ee215","Type":"ContainerStarted","Data":"9b5490c4ffcf0fbafda4ed8fee8dc04883318790c86bdbe13fc98ef457d414dd"}
Sep 04 21:54:52.127288 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.127248    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret\") pod \"global-pull-secret-syncer-p7b77\" (UID: \"8fcc2326-c662-4e67-bdaf-60a54cce4235\") " pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:52.127496 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.127312    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhjmp\" (UniqueName: \"kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp\") pod \"network-check-target-dflhd\" (UID: \"10da7e37-b753-4b5f-93f5-07e7a48ba85b\") " pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:52.127496 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.127345    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs\") pod \"network-metrics-daemon-46fbr\" (UID: \"eadd93c2-4a94-4046-9924-6e342406d1f8\") " pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:52.129925 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.129902    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 04 21:54:52.130667 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.130647    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 04 21:54:52.130754 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.130647    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 04 21:54:52.140703 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.140666    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 04 21:54:52.141042 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.141018    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/eadd93c2-4a94-4046-9924-6e342406d1f8-metrics-certs\") pod \"network-metrics-daemon-46fbr\" (UID: \"eadd93c2-4a94-4046-9924-6e342406d1f8\") " pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:52.141154 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.141125    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8fcc2326-c662-4e67-bdaf-60a54cce4235-original-pull-secret\") pod \"global-pull-secret-syncer-p7b77\" (UID: \"8fcc2326-c662-4e67-bdaf-60a54cce4235\") " pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:52.151519 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.151482    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhjmp\" (UniqueName: \"kubernetes.io/projected/10da7e37-b753-4b5f-93f5-07e7a48ba85b-kube-api-access-fhjmp\") pod \"network-check-target-dflhd\" (UID: \"10da7e37-b753-4b5f-93f5-07e7a48ba85b\") " pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:52.268055 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.268028    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p7b77"
Sep 04 21:54:52.277460 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.277436    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-268hq\""
Sep 04 21:54:52.282245 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.282217    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-xzsq2\""
Sep 04 21:54:52.286000 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.285976    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-46fbr"
Sep 04 21:54:52.291397 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.291360    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:52.541613 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.541577    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p7b77"]
Sep 04 21:54:52.544545 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:54:52.544518    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fcc2326_c662_4e67_bdaf_60a54cce4235.slice/crio-214092ee0012e4b4e31be1c5eb56355f51ed2bd947f03807d3ba453678c7bdf0 WatchSource:0}: Error finding container 214092ee0012e4b4e31be1c5eb56355f51ed2bd947f03807d3ba453678c7bdf0: Status 404 returned error can't find the container with id 214092ee0012e4b4e31be1c5eb56355f51ed2bd947f03807d3ba453678c7bdf0
Sep 04 21:54:52.571494 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.571321    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-46fbr"]
Sep 04 21:54:52.573954 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:54:52.573925    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeadd93c2_4a94_4046_9924_6e342406d1f8.slice/crio-9270ee9db3a465e17bf92fb25878de496b46d7d92c49fd3931679f7d0a88e5f0 WatchSource:0}: Error finding container 9270ee9db3a465e17bf92fb25878de496b46d7d92c49fd3931679f7d0a88e5f0: Status 404 returned error can't find the container with id 9270ee9db3a465e17bf92fb25878de496b46d7d92c49fd3931679f7d0a88e5f0
Sep 04 21:54:52.583745 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.583718    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dflhd"]
Sep 04 21:54:52.586472 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:54:52.586445    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10da7e37_b753_4b5f_93f5_07e7a48ba85b.slice/crio-5b86650bc83dff98956c43a9748bb72021bb2edb8c026a8af9fa94005762cbe3 WatchSource:0}: Error finding container 5b86650bc83dff98956c43a9748bb72021bb2edb8c026a8af9fa94005762cbe3: Status 404 returned error can't find the container with id 5b86650bc83dff98956c43a9748bb72021bb2edb8c026a8af9fa94005762cbe3
Sep 04 21:54:52.637351 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.637269    2557 generic.go:356] "Generic (PLEG): container finished" podID="f720a6df-15a0-41b0-8367-58de99e72753" containerID="f01043cccf89a24987830cf1a69171451e3128ecec44ed0ee5e058baae8cae20" exitCode=0
Sep 04 21:54:52.637351 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.637321    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f720a6df-15a0-41b0-8367-58de99e72753","Type":"ContainerDied","Data":"f01043cccf89a24987830cf1a69171451e3128ecec44ed0ee5e058baae8cae20"}
Sep 04 21:54:52.638739 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.638717    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-46fbr" event={"ID":"eadd93c2-4a94-4046-9924-6e342406d1f8","Type":"ContainerStarted","Data":"9270ee9db3a465e17bf92fb25878de496b46d7d92c49fd3931679f7d0a88e5f0"}
Sep 04 21:54:52.640923 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.640900    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd" event={"ID":"dd9483eb-f008-4171-9233-2724cb1ee215","Type":"ContainerStarted","Data":"d21ee747a48af20dce9d06b9ac104672b07aad364a71f92d0892f75269878bfa"}
Sep 04 21:54:52.642516 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.642492    2557 generic.go:356] "Generic (PLEG): container finished" podID="6420905f-c55a-4e5a-aa93-b36775b74d93" containerID="dc2dd48e659e50a564254b81ff45438a9baf3f98d2a7ab479297e0f7f773a5c6" exitCode=0
Sep 04 21:54:52.642599 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.642528    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m25wx" event={"ID":"6420905f-c55a-4e5a-aa93-b36775b74d93","Type":"ContainerDied","Data":"dc2dd48e659e50a564254b81ff45438a9baf3f98d2a7ab479297e0f7f773a5c6"}
Sep 04 21:54:52.643939 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.643695    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p7b77" event={"ID":"8fcc2326-c662-4e67-bdaf-60a54cce4235","Type":"ContainerStarted","Data":"214092ee0012e4b4e31be1c5eb56355f51ed2bd947f03807d3ba453678c7bdf0"}
Sep 04 21:54:52.644876 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.644852    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dflhd" event={"ID":"10da7e37-b753-4b5f-93f5-07e7a48ba85b","Type":"ContainerStarted","Data":"5b86650bc83dff98956c43a9748bb72021bb2edb8c026a8af9fa94005762cbe3"}
Sep 04 21:54:52.708301 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:52.708245    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-qwtkd" podStartSLOduration=2.907113748 podStartE2EDuration="4.708228509s" totalImagesPullingTime="1.801114761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:52.706933633 +0000 UTC m=+67.023500384" watchObservedRunningTime="2026-09-04 21:54:52.708228509 +0000 UTC m=+67.024795248"
Sep 04 21:54:53.655121 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:53.654957    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m25wx" event={"ID":"6420905f-c55a-4e5a-aa93-b36775b74d93","Type":"ContainerStarted","Data":"7bce6d8d70f189b275ce9e5ab1e77e6ab4498025a0414babbdfd9cebd248be2a"}
Sep 04 21:54:53.655121 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:53.655069    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m25wx" event={"ID":"6420905f-c55a-4e5a-aa93-b36775b74d93","Type":"ContainerStarted","Data":"755fb8adfff61ff2e9626520c853f74d0cb1cf6d5fd3db097218ae992be49680"}
Sep 04 21:54:53.676245 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:53.675926    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-m25wx" podStartSLOduration=3.892950064 podStartE2EDuration="5.67590033s" totalImagesPullingTime="1.710292521s" totalInitContainerRuntime="72.657745ms" isStatefulPod=true podCreationTimestamp="2026-09-04 21:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:53.674298334 +0000 UTC m=+67.990865073" watchObservedRunningTime="2026-09-04 21:54:53.67590033 +0000 UTC m=+67.992467069"
Sep 04 21:54:54.129900 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.128602    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b4cfb948b-kmt5z"]
Sep 04 21:54:54.133083 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.132336    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:54:54.134829 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.134799    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 04 21:54:54.135116 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.135098    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 04 21:54:54.136273 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.136247    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 04 21:54:54.136376 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.136254    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 04 21:54:54.136376 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.136328    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-l72hl\""
Sep 04 21:54:54.136770 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.136751    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 04 21:54:54.136864 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.136820    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 04 21:54:54.137264 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.137246    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 04 21:54:54.142700 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.142498    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 04 21:54:54.143447 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.143172    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b4cfb948b-kmt5z"]
Sep 04 21:54:54.248326 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.248164    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-trusted-ca-bundle\") pod \"console-5b4cfb948b-kmt5z\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") " pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:54:54.248326 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.248223    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cznrv\" (UniqueName: \"kubernetes.io/projected/5cb48c90-9c68-4350-8570-b79956d6a56d-kube-api-access-cznrv\") pod \"console-5b4cfb948b-kmt5z\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") " pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:54:54.248326 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.248266    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5cb48c90-9c68-4350-8570-b79956d6a56d-console-oauth-config\") pod \"console-5b4cfb948b-kmt5z\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") " pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:54:54.248326 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.248295    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-oauth-serving-cert\") pod \"console-5b4cfb948b-kmt5z\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") " pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:54:54.248704 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.248386    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-service-ca\") pod \"console-5b4cfb948b-kmt5z\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") " pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:54:54.248704 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.248475    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-console-config\") pod \"console-5b4cfb948b-kmt5z\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") " pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:54:54.248704 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.248512    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5cb48c90-9c68-4350-8570-b79956d6a56d-console-serving-cert\") pod \"console-5b4cfb948b-kmt5z\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") " pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:54:54.352283 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.349439    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cznrv\" (UniqueName: \"kubernetes.io/projected/5cb48c90-9c68-4350-8570-b79956d6a56d-kube-api-access-cznrv\") pod \"console-5b4cfb948b-kmt5z\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") " pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:54:54.352283 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.349504    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5cb48c90-9c68-4350-8570-b79956d6a56d-console-oauth-config\") pod \"console-5b4cfb948b-kmt5z\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") " pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:54:54.352283 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.349537    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-oauth-serving-cert\") pod \"console-5b4cfb948b-kmt5z\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") " pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:54:54.352283 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.349564    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-service-ca\") pod \"console-5b4cfb948b-kmt5z\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") " pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:54:54.352283 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.349619    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-console-config\") pod \"console-5b4cfb948b-kmt5z\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") " pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:54:54.352283 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.349649    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5cb48c90-9c68-4350-8570-b79956d6a56d-console-serving-cert\") pod \"console-5b4cfb948b-kmt5z\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") " pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:54:54.352283 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.349704    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-trusted-ca-bundle\") pod \"console-5b4cfb948b-kmt5z\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") " pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:54:54.352283 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.350603    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-service-ca\") pod \"console-5b4cfb948b-kmt5z\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") " pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:54:54.352283 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.350648    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-trusted-ca-bundle\") pod \"console-5b4cfb948b-kmt5z\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") " pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:54:54.352283 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.351219    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-oauth-serving-cert\") pod \"console-5b4cfb948b-kmt5z\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") " pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:54:54.352283 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.351255    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-console-config\") pod \"console-5b4cfb948b-kmt5z\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") " pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:54:54.356057 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.354902    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5cb48c90-9c68-4350-8570-b79956d6a56d-console-oauth-config\") pod \"console-5b4cfb948b-kmt5z\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") " pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:54:54.356057 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.355517    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5cb48c90-9c68-4350-8570-b79956d6a56d-console-serving-cert\") pod \"console-5b4cfb948b-kmt5z\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") " pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:54:54.361773 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.361720    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cznrv\" (UniqueName: \"kubernetes.io/projected/5cb48c90-9c68-4350-8570-b79956d6a56d-kube-api-access-cznrv\") pod \"console-5b4cfb948b-kmt5z\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") " pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:54:54.448739 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.448385    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:54:54.663211 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.663171    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-46fbr" event={"ID":"eadd93c2-4a94-4046-9924-6e342406d1f8","Type":"ContainerStarted","Data":"7b28ccf7dfe972e26f566f9c6596774cdb66a98e792021ed372f3506571a99b3"}
Sep 04 21:54:54.663699 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.663278    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-46fbr" event={"ID":"eadd93c2-4a94-4046-9924-6e342406d1f8","Type":"ContainerStarted","Data":"d302efa8c9e8b6240afabe3ab5e743eceb62492b488775eb0e0d0efaee97a6e4"}
Sep 04 21:54:54.682975 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.682902    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-46fbr" podStartSLOduration=67.482425092 podStartE2EDuration="1m8.682878452s" totalImagesPullingTime="1.20045336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:54.679757442 +0000 UTC m=+68.996324196" watchObservedRunningTime="2026-09-04 21:54:54.682878452 +0000 UTC m=+68.999445191"
Sep 04 21:54:54.826654 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:54.826619    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b4cfb948b-kmt5z"]
Sep 04 21:54:55.669429 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:55.669382    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b4cfb948b-kmt5z" event={"ID":"5cb48c90-9c68-4350-8570-b79956d6a56d","Type":"ContainerStarted","Data":"cbd5ef9f61f526e41cea38e5ff57be0dc3263218e2d4c42984673f55fcc8b85e"}
Sep 04 21:54:55.673346 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:55.673305    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f720a6df-15a0-41b0-8367-58de99e72753","Type":"ContainerStarted","Data":"d0b89d4f1640205b5b5afbac1927d93d3f5ca63a9656f90efcd552ba6e33d996"}
Sep 04 21:54:55.673517 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:55.673381    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f720a6df-15a0-41b0-8367-58de99e72753","Type":"ContainerStarted","Data":"f8d3df8f9289c169511694f777b330d1c44f0bdfcc347e4cc4a7fee33a0eddb9"}
Sep 04 21:54:55.673517 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:55.673434    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f720a6df-15a0-41b0-8367-58de99e72753","Type":"ContainerStarted","Data":"b88afe5262839fa536dbac1811e611eef08e4740cb4f9dae8d2ce47df6ca5485"}
Sep 04 21:54:55.673517 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:55.673460    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f720a6df-15a0-41b0-8367-58de99e72753","Type":"ContainerStarted","Data":"09f9ebbc06dfb08bcaa05133743f48315c8bf359acee86dd911a7303b1b997f8"}
Sep 04 21:54:55.673517 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:55.673484    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f720a6df-15a0-41b0-8367-58de99e72753","Type":"ContainerStarted","Data":"8ed22e9ec8f443cf9584ac45285816a327fd64b691b051c8c2759bc45a58ae92"}
Sep 04 21:54:56.323474 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:56.323320    2557 patch_prober.go:28] interesting pod/image-registry-86f655765c-vdjrp container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 04 21:54:56.323474 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:56.323394    2557 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-86f655765c-vdjrp" podUID="cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 04 21:54:58.573782 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:58.573750    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-86f655765c-vdjrp"
Sep 04 21:54:59.694834 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:59.694785    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p7b77" event={"ID":"8fcc2326-c662-4e67-bdaf-60a54cce4235","Type":"ContainerStarted","Data":"bafb4660d0392e65865894f6b8edc4c64d7d4e958f5ade03a7ec56d49c4311f2"}
Sep 04 21:54:59.696287 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:59.696258    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dflhd" event={"ID":"10da7e37-b753-4b5f-93f5-07e7a48ba85b","Type":"ContainerStarted","Data":"bceb4db7031113ee8925a9567cb2fbab1b0a9dccaff1e8aa891bc2a10259a6a4"}
Sep 04 21:54:59.696447 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:59.696313    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:54:59.697896 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:59.697866    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b4cfb948b-kmt5z" event={"ID":"5cb48c90-9c68-4350-8570-b79956d6a56d","Type":"ContainerStarted","Data":"8a57bc89c889299ae8fe7c9b714d3c4669cb4db34572507c7e895c81f325a8a5"}
Sep 04 21:54:59.701157 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:59.701134    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f720a6df-15a0-41b0-8367-58de99e72753","Type":"ContainerStarted","Data":"ce1100f0f09e3ca65c804b68b7af703df72c0d93d30deadbc3adb65f044a1f33"}
Sep 04 21:54:59.709465 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:59.709396    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-p7b77" podStartSLOduration=67.482941103 podStartE2EDuration="1m13.709379121s" totalImagesPullingTime="6.226438018s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:59.70863817 +0000 UTC m=+74.025204908" watchObservedRunningTime="2026-09-04 21:54:59.709379121 +0000 UTC m=+74.025945859"
Sep 04 21:54:59.725038 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:59.724921    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b4cfb948b-kmt5z" podStartSLOduration=1.6253332409999999 podStartE2EDuration="5.724900468s" totalImagesPullingTime="4.099567227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:54:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:59.724596733 +0000 UTC m=+74.041163484" watchObservedRunningTime="2026-09-04 21:54:59.724900468 +0000 UTC m=+74.041467206"
Sep 04 21:54:59.740514 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:59.740456    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dflhd" podStartSLOduration=67.569238655 podStartE2EDuration="1m13.740437157s" totalImagesPullingTime="6.171198502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:59.739151112 +0000 UTC m=+74.055717852" watchObservedRunningTime="2026-09-04 21:54:59.740437157 +0000 UTC m=+74.057003887"
Sep 04 21:54:59.763230 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:59.763164    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.453722016 podStartE2EDuration="10.763140661s" totalImagesPullingTime="7.286260919s" totalInitContainerRuntime="23.157726ms" isStatefulPod=false podCreationTimestamp="2026-09-04 21:54:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:54:59.761740219 +0000 UTC m=+74.078306957" watchObservedRunningTime="2026-09-04 21:54:59.763140661 +0000 UTC m=+74.079707399"
Sep 04 21:54:59.939233 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:59.939199    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b4cfb948b-kmt5z"]
Sep 04 21:54:59.970681 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:59.970603    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-679d94bfd6-bvcbg"]
Sep 04 21:54:59.973951 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:59.973934    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:54:59.985102 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:54:59.985073    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-679d94bfd6-bvcbg"]
Sep 04 21:55:00.100517 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.100474    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19d12fc6-eabf-4dd0-9639-7c8cbd557795-console-oauth-config\") pod \"console-679d94bfd6-bvcbg\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") " pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:00.100517 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.100511    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-oauth-serving-cert\") pod \"console-679d94bfd6-bvcbg\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") " pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:00.100733 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.100557    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-console-config\") pod \"console-679d94bfd6-bvcbg\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") " pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:00.100733 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.100574    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-service-ca\") pod \"console-679d94bfd6-bvcbg\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") " pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:00.100733 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.100592    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-trusted-ca-bundle\") pod \"console-679d94bfd6-bvcbg\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") " pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:00.100733 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.100613    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19d12fc6-eabf-4dd0-9639-7c8cbd557795-console-serving-cert\") pod \"console-679d94bfd6-bvcbg\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") " pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:00.100733 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.100637    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrq2p\" (UniqueName: \"kubernetes.io/projected/19d12fc6-eabf-4dd0-9639-7c8cbd557795-kube-api-access-jrq2p\") pod \"console-679d94bfd6-bvcbg\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") " pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:00.201149 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.201108    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19d12fc6-eabf-4dd0-9639-7c8cbd557795-console-oauth-config\") pod \"console-679d94bfd6-bvcbg\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") " pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:00.201149 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.201142    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-oauth-serving-cert\") pod \"console-679d94bfd6-bvcbg\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") " pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:00.201379 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.201177    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-console-config\") pod \"console-679d94bfd6-bvcbg\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") " pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:00.201379 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.201194    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-service-ca\") pod \"console-679d94bfd6-bvcbg\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") " pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:00.201379 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.201211    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-trusted-ca-bundle\") pod \"console-679d94bfd6-bvcbg\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") " pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:00.201379 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.201239    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19d12fc6-eabf-4dd0-9639-7c8cbd557795-console-serving-cert\") pod \"console-679d94bfd6-bvcbg\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") " pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:00.201379 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.201264    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrq2p\" (UniqueName: \"kubernetes.io/projected/19d12fc6-eabf-4dd0-9639-7c8cbd557795-kube-api-access-jrq2p\") pod \"console-679d94bfd6-bvcbg\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") " pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:00.202067 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.202038    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-console-config\") pod \"console-679d94bfd6-bvcbg\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") " pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:00.202192 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.202077    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-service-ca\") pod \"console-679d94bfd6-bvcbg\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") " pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:00.202192 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.202077    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-oauth-serving-cert\") pod \"console-679d94bfd6-bvcbg\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") " pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:00.202192 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.202157    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-trusted-ca-bundle\") pod \"console-679d94bfd6-bvcbg\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") " pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:00.203680 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.203654    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19d12fc6-eabf-4dd0-9639-7c8cbd557795-console-oauth-config\") pod \"console-679d94bfd6-bvcbg\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") " pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:00.203791 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.203771    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19d12fc6-eabf-4dd0-9639-7c8cbd557795-console-serving-cert\") pod \"console-679d94bfd6-bvcbg\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") " pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:00.209890 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.209867    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrq2p\" (UniqueName: \"kubernetes.io/projected/19d12fc6-eabf-4dd0-9639-7c8cbd557795-kube-api-access-jrq2p\") pod \"console-679d94bfd6-bvcbg\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") " pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:00.282886 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.282853    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:00.557693 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.557595    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-679d94bfd6-bvcbg"]
Sep 04 21:55:00.560617 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:55:00.560587    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19d12fc6_eabf_4dd0_9639_7c8cbd557795.slice/crio-f5e554acad1fbcaa306b40efa68c8055747aba5c7b29feae9d5e52c7327f3595 WatchSource:0}: Error finding container f5e554acad1fbcaa306b40efa68c8055747aba5c7b29feae9d5e52c7327f3595: Status 404 returned error can't find the container with id f5e554acad1fbcaa306b40efa68c8055747aba5c7b29feae9d5e52c7327f3595
Sep 04 21:55:00.707051 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.707009    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-679d94bfd6-bvcbg" event={"ID":"19d12fc6-eabf-4dd0-9639-7c8cbd557795","Type":"ContainerStarted","Data":"1eb07c401af55907eb5094d26adfffe4c1b727ae43ac0d1d267ea6b77ef9a901"}
Sep 04 21:55:00.707568 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.707074    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-679d94bfd6-bvcbg" event={"ID":"19d12fc6-eabf-4dd0-9639-7c8cbd557795","Type":"ContainerStarted","Data":"f5e554acad1fbcaa306b40efa68c8055747aba5c7b29feae9d5e52c7327f3595"}
Sep 04 21:55:00.730514 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:00.730452    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-679d94bfd6-bvcbg" podStartSLOduration=1.730434148 podStartE2EDuration="1.730434148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:54:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:55:00.72930863 +0000 UTC m=+75.045875371" watchObservedRunningTime="2026-09-04 21:55:00.730434148 +0000 UTC m=+75.047000877"
Sep 04 21:55:04.449827 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:04.449788    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:55:10.283390 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:10.283238    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:10.283390 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:10.283322    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:10.288157 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:10.288128    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:10.748695 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:10.748668    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:55:26.730931 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:26.730864    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5b4cfb948b-kmt5z" podUID="5cb48c90-9c68-4350-8570-b79956d6a56d" containerName="console" containerID="cri-o://8a57bc89c889299ae8fe7c9b714d3c4669cb4db34572507c7e895c81f325a8a5" gracePeriod=15
Sep 04 21:55:26.999155 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:26.999119    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:55:27.024157 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.024121    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-trusted-ca-bundle\") pod \"5cb48c90-9c68-4350-8570-b79956d6a56d\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") "
Sep 04 21:55:27.024339 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.024169    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5cb48c90-9c68-4350-8570-b79956d6a56d-console-serving-cert\") pod \"5cb48c90-9c68-4350-8570-b79956d6a56d\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") "
Sep 04 21:55:27.024339 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.024203    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-service-ca\") pod \"5cb48c90-9c68-4350-8570-b79956d6a56d\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") "
Sep 04 21:55:27.024339 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.024226    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5cb48c90-9c68-4350-8570-b79956d6a56d-console-oauth-config\") pod \"5cb48c90-9c68-4350-8570-b79956d6a56d\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") "
Sep 04 21:55:27.024339 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.024288    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cznrv\" (UniqueName: \"kubernetes.io/projected/5cb48c90-9c68-4350-8570-b79956d6a56d-kube-api-access-cznrv\") pod \"5cb48c90-9c68-4350-8570-b79956d6a56d\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") "
Sep 04 21:55:27.024339 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.024322    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-console-config\") pod \"5cb48c90-9c68-4350-8570-b79956d6a56d\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") "
Sep 04 21:55:27.024617 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.024360    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-oauth-serving-cert\") pod \"5cb48c90-9c68-4350-8570-b79956d6a56d\" (UID: \"5cb48c90-9c68-4350-8570-b79956d6a56d\") "
Sep 04 21:55:27.024617 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.024546    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5cb48c90-9c68-4350-8570-b79956d6a56d" (UID: "5cb48c90-9c68-4350-8570-b79956d6a56d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 21:55:27.024846 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.024804    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-console-config" (OuterVolumeSpecName: "console-config") pod "5cb48c90-9c68-4350-8570-b79956d6a56d" (UID: "5cb48c90-9c68-4350-8570-b79956d6a56d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 21:55:27.024846 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.024834    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-service-ca" (OuterVolumeSpecName: "service-ca") pod "5cb48c90-9c68-4350-8570-b79956d6a56d" (UID: "5cb48c90-9c68-4350-8570-b79956d6a56d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 21:55:27.025026 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.024868    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5cb48c90-9c68-4350-8570-b79956d6a56d" (UID: "5cb48c90-9c68-4350-8570-b79956d6a56d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 21:55:27.026749 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.026717    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5cb48c90-9c68-4350-8570-b79956d6a56d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5cb48c90-9c68-4350-8570-b79956d6a56d" (UID: "5cb48c90-9c68-4350-8570-b79956d6a56d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 21:55:27.026878 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.026812    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5cb48c90-9c68-4350-8570-b79956d6a56d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5cb48c90-9c68-4350-8570-b79956d6a56d" (UID: "5cb48c90-9c68-4350-8570-b79956d6a56d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 21:55:27.026878 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.026847    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cb48c90-9c68-4350-8570-b79956d6a56d-kube-api-access-cznrv" (OuterVolumeSpecName: "kube-api-access-cznrv") pod "5cb48c90-9c68-4350-8570-b79956d6a56d" (UID: "5cb48c90-9c68-4350-8570-b79956d6a56d"). InnerVolumeSpecName "kube-api-access-cznrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 21:55:27.125192 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.125153    2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-trusted-ca-bundle\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:55:27.125192 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.125188    2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5cb48c90-9c68-4350-8570-b79956d6a56d-console-serving-cert\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:55:27.125192 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.125198    2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-service-ca\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:55:27.125451 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.125207    2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5cb48c90-9c68-4350-8570-b79956d6a56d-console-oauth-config\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:55:27.125451 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.125217    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cznrv\" (UniqueName: \"kubernetes.io/projected/5cb48c90-9c68-4350-8570-b79956d6a56d-kube-api-access-cznrv\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:55:27.125451 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.125225    2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-console-config\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:55:27.125451 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.125233    2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5cb48c90-9c68-4350-8570-b79956d6a56d-oauth-serving-cert\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:55:27.809609 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.809571    2557 generic.go:356] "Generic (PLEG): container finished" podID="5cb48c90-9c68-4350-8570-b79956d6a56d" containerID="8a57bc89c889299ae8fe7c9b714d3c4669cb4db34572507c7e895c81f325a8a5" exitCode=0
Sep 04 21:55:27.810011 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.809623    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b4cfb948b-kmt5z"
Sep 04 21:55:27.810011 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.809662    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b4cfb948b-kmt5z" event={"ID":"5cb48c90-9c68-4350-8570-b79956d6a56d","Type":"ContainerDied","Data":"8a57bc89c889299ae8fe7c9b714d3c4669cb4db34572507c7e895c81f325a8a5"}
Sep 04 21:55:27.810011 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.809718    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b4cfb948b-kmt5z" event={"ID":"5cb48c90-9c68-4350-8570-b79956d6a56d","Type":"ContainerDied","Data":"cbd5ef9f61f526e41cea38e5ff57be0dc3263218e2d4c42984673f55fcc8b85e"}
Sep 04 21:55:27.810011 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.809743    2557 scope.go:117] "RemoveContainer" containerID="8a57bc89c889299ae8fe7c9b714d3c4669cb4db34572507c7e895c81f325a8a5"
Sep 04 21:55:27.826511 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.826486    2557 scope.go:117] "RemoveContainer" containerID="8a57bc89c889299ae8fe7c9b714d3c4669cb4db34572507c7e895c81f325a8a5"
Sep 04 21:55:27.826874 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:55:27.826854    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a57bc89c889299ae8fe7c9b714d3c4669cb4db34572507c7e895c81f325a8a5\": container with ID starting with 8a57bc89c889299ae8fe7c9b714d3c4669cb4db34572507c7e895c81f325a8a5 not found: ID does not exist" containerID="8a57bc89c889299ae8fe7c9b714d3c4669cb4db34572507c7e895c81f325a8a5"
Sep 04 21:55:27.826929 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.826883    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a57bc89c889299ae8fe7c9b714d3c4669cb4db34572507c7e895c81f325a8a5"} err="failed to get container status \"8a57bc89c889299ae8fe7c9b714d3c4669cb4db34572507c7e895c81f325a8a5\": rpc error: code = NotFound desc = could not find container \"8a57bc89c889299ae8fe7c9b714d3c4669cb4db34572507c7e895c81f325a8a5\": container with ID starting with 8a57bc89c889299ae8fe7c9b714d3c4669cb4db34572507c7e895c81f325a8a5 not found: ID does not exist"
Sep 04 21:55:27.830649 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.830620    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b4cfb948b-kmt5z"]
Sep 04 21:55:27.838257 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:27.838225    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b4cfb948b-kmt5z"]
Sep 04 21:55:28.351601 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:28.351567    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cb48c90-9c68-4350-8570-b79956d6a56d" path="/var/lib/kubelet/pods/5cb48c90-9c68-4350-8570-b79956d6a56d/volumes"
Sep 04 21:55:30.710237 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:55:30.710205    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dflhd"
Sep 04 21:56:19.685120 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.685083    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-598c465776-gs2ts"]
Sep 04 21:56:19.685752 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.685493    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cb48c90-9c68-4350-8570-b79956d6a56d" containerName="console"
Sep 04 21:56:19.685752 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.685514    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cb48c90-9c68-4350-8570-b79956d6a56d" containerName="console"
Sep 04 21:56:19.685752 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.685590    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cb48c90-9c68-4350-8570-b79956d6a56d" containerName="console"
Sep 04 21:56:19.687449 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.687429    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:19.704281 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.704247    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-598c465776-gs2ts"]
Sep 04 21:56:19.812557 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.812502    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afcdfa91-13e3-4315-8eec-0554a5195a99-console-serving-cert\") pod \"console-598c465776-gs2ts\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") " pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:19.812557 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.812559    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52mzb\" (UniqueName: \"kubernetes.io/projected/afcdfa91-13e3-4315-8eec-0554a5195a99-kube-api-access-52mzb\") pod \"console-598c465776-gs2ts\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") " pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:19.812807 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.812580    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-trusted-ca-bundle\") pod \"console-598c465776-gs2ts\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") " pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:19.812807 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.812650    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-console-config\") pod \"console-598c465776-gs2ts\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") " pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:19.812807 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.812687    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-oauth-serving-cert\") pod \"console-598c465776-gs2ts\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") " pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:19.812807 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.812707    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afcdfa91-13e3-4315-8eec-0554a5195a99-console-oauth-config\") pod \"console-598c465776-gs2ts\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") " pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:19.812807 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.812724    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-service-ca\") pod \"console-598c465776-gs2ts\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") " pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:19.913825 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.913777    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afcdfa91-13e3-4315-8eec-0554a5195a99-console-serving-cert\") pod \"console-598c465776-gs2ts\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") " pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:19.913825 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.913832    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52mzb\" (UniqueName: \"kubernetes.io/projected/afcdfa91-13e3-4315-8eec-0554a5195a99-kube-api-access-52mzb\") pod \"console-598c465776-gs2ts\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") " pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:19.914008 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.913854    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-trusted-ca-bundle\") pod \"console-598c465776-gs2ts\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") " pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:19.914008 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.913888    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-console-config\") pod \"console-598c465776-gs2ts\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") " pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:19.914008 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.913908    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-oauth-serving-cert\") pod \"console-598c465776-gs2ts\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") " pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:19.914008 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.913932    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afcdfa91-13e3-4315-8eec-0554a5195a99-console-oauth-config\") pod \"console-598c465776-gs2ts\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") " pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:19.914008 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.913948    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-service-ca\") pod \"console-598c465776-gs2ts\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") " pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:19.914726 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.914700    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-service-ca\") pod \"console-598c465776-gs2ts\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") " pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:19.914852 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.914768    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-oauth-serving-cert\") pod \"console-598c465776-gs2ts\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") " pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:19.914915 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.914853    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-console-config\") pod \"console-598c465776-gs2ts\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") " pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:19.915070 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.915050    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-trusted-ca-bundle\") pod \"console-598c465776-gs2ts\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") " pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:19.916438 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.916384    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afcdfa91-13e3-4315-8eec-0554a5195a99-console-serving-cert\") pod \"console-598c465776-gs2ts\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") " pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:19.916536 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.916476    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afcdfa91-13e3-4315-8eec-0554a5195a99-console-oauth-config\") pod \"console-598c465776-gs2ts\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") " pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:19.923214 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.923185    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52mzb\" (UniqueName: \"kubernetes.io/projected/afcdfa91-13e3-4315-8eec-0554a5195a99-kube-api-access-52mzb\") pod \"console-598c465776-gs2ts\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") " pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:19.997361 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:19.997272    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:20.271166 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:20.271082    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-598c465776-gs2ts"]
Sep 04 21:56:20.274695 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:56:20.274667    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafcdfa91_13e3_4315_8eec_0554a5195a99.slice/crio-7a56bb1ff681db3888a1ac6e1d3a5c5c485d4bfb3cb1e1b73ce1b0e603db084a WatchSource:0}: Error finding container 7a56bb1ff681db3888a1ac6e1d3a5c5c485d4bfb3cb1e1b73ce1b0e603db084a: Status 404 returned error can't find the container with id 7a56bb1ff681db3888a1ac6e1d3a5c5c485d4bfb3cb1e1b73ce1b0e603db084a
Sep 04 21:56:21.005597 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:21.005561    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-598c465776-gs2ts" event={"ID":"afcdfa91-13e3-4315-8eec-0554a5195a99","Type":"ContainerStarted","Data":"6f151ef201e9bbbafafffa2f73fef84bc5e21dca2a512e3667cb4508861bb774"}
Sep 04 21:56:21.006083 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:21.005619    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-598c465776-gs2ts" event={"ID":"afcdfa91-13e3-4315-8eec-0554a5195a99","Type":"ContainerStarted","Data":"7a56bb1ff681db3888a1ac6e1d3a5c5c485d4bfb3cb1e1b73ce1b0e603db084a"}
Sep 04 21:56:21.032197 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:21.032135    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-598c465776-gs2ts" podStartSLOduration=2.032117542 podStartE2EDuration="2.032117542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:56:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:56:21.031164348 +0000 UTC m=+155.347731088" watchObservedRunningTime="2026-09-04 21:56:21.032117542 +0000 UTC m=+155.348684278"
Sep 04 21:56:29.997953 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:29.997912    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:29.998333 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:29.997989    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:30.002543 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:30.002519    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:30.043025 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:30.042999    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-598c465776-gs2ts"
Sep 04 21:56:30.089952 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:30.089914    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-679d94bfd6-bvcbg"]
Sep 04 21:56:55.111930 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:55.111839    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-679d94bfd6-bvcbg" podUID="19d12fc6-eabf-4dd0-9639-7c8cbd557795" containerName="console" containerID="cri-o://1eb07c401af55907eb5094d26adfffe4c1b727ae43ac0d1d267ea6b77ef9a901" gracePeriod=15
Sep 04 21:56:55.356635 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:55.356608    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:56:55.401701 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:55.401601    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-service-ca\") pod \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") "
Sep 04 21:56:55.401701 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:55.401670    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrq2p\" (UniqueName: \"kubernetes.io/projected/19d12fc6-eabf-4dd0-9639-7c8cbd557795-kube-api-access-jrq2p\") pod \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") "
Sep 04 21:56:55.401701 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:55.401693    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-oauth-serving-cert\") pod \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") "
Sep 04 21:56:55.401960 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:55.401712    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-console-config\") pod \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") "
Sep 04 21:56:55.401960 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:55.401734    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19d12fc6-eabf-4dd0-9639-7c8cbd557795-console-serving-cert\") pod \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") "
Sep 04 21:56:55.401960 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:55.401753    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-trusted-ca-bundle\") pod \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") "
Sep 04 21:56:55.401960 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:55.401772    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19d12fc6-eabf-4dd0-9639-7c8cbd557795-console-oauth-config\") pod \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\" (UID: \"19d12fc6-eabf-4dd0-9639-7c8cbd557795\") "
Sep 04 21:56:55.402152 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:55.402038    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-service-ca" (OuterVolumeSpecName: "service-ca") pod "19d12fc6-eabf-4dd0-9639-7c8cbd557795" (UID: "19d12fc6-eabf-4dd0-9639-7c8cbd557795"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 21:56:55.402286 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:55.402251    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "19d12fc6-eabf-4dd0-9639-7c8cbd557795" (UID: "19d12fc6-eabf-4dd0-9639-7c8cbd557795"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 21:56:55.402286 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:55.402262    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "19d12fc6-eabf-4dd0-9639-7c8cbd557795" (UID: "19d12fc6-eabf-4dd0-9639-7c8cbd557795"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 21:56:55.402478 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:55.402327    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-console-config" (OuterVolumeSpecName: "console-config") pod "19d12fc6-eabf-4dd0-9639-7c8cbd557795" (UID: "19d12fc6-eabf-4dd0-9639-7c8cbd557795"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 21:56:55.404000 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:55.403968    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19d12fc6-eabf-4dd0-9639-7c8cbd557795-kube-api-access-jrq2p" (OuterVolumeSpecName: "kube-api-access-jrq2p") pod "19d12fc6-eabf-4dd0-9639-7c8cbd557795" (UID: "19d12fc6-eabf-4dd0-9639-7c8cbd557795"). InnerVolumeSpecName "kube-api-access-jrq2p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 21:56:55.404078 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:55.403990    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19d12fc6-eabf-4dd0-9639-7c8cbd557795-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "19d12fc6-eabf-4dd0-9639-7c8cbd557795" (UID: "19d12fc6-eabf-4dd0-9639-7c8cbd557795"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 21:56:55.404078 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:55.404005    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19d12fc6-eabf-4dd0-9639-7c8cbd557795-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "19d12fc6-eabf-4dd0-9639-7c8cbd557795" (UID: "19d12fc6-eabf-4dd0-9639-7c8cbd557795"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 21:56:55.502878 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:55.502825    2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-service-ca\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:56:55.502878 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:55.502870    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrq2p\" (UniqueName: \"kubernetes.io/projected/19d12fc6-eabf-4dd0-9639-7c8cbd557795-kube-api-access-jrq2p\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:56:55.502878 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:55.502880    2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-oauth-serving-cert\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:56:55.502878 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:55.502890    2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-console-config\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:56:55.502878 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:55.502899    2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/19d12fc6-eabf-4dd0-9639-7c8cbd557795-console-serving-cert\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:56:55.503157 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:55.502907    2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/19d12fc6-eabf-4dd0-9639-7c8cbd557795-trusted-ca-bundle\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:56:55.503157 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:55.502915    2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/19d12fc6-eabf-4dd0-9639-7c8cbd557795-console-oauth-config\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:56:56.134285 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:56.134246    2557 generic.go:356] "Generic (PLEG): container finished" podID="19d12fc6-eabf-4dd0-9639-7c8cbd557795" containerID="1eb07c401af55907eb5094d26adfffe4c1b727ae43ac0d1d267ea6b77ef9a901" exitCode=0
Sep 04 21:56:56.134710 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:56.134292    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-679d94bfd6-bvcbg"
Sep 04 21:56:56.134710 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:56.134323    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-679d94bfd6-bvcbg" event={"ID":"19d12fc6-eabf-4dd0-9639-7c8cbd557795","Type":"ContainerDied","Data":"1eb07c401af55907eb5094d26adfffe4c1b727ae43ac0d1d267ea6b77ef9a901"}
Sep 04 21:56:56.134710 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:56.134367    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-679d94bfd6-bvcbg" event={"ID":"19d12fc6-eabf-4dd0-9639-7c8cbd557795","Type":"ContainerDied","Data":"f5e554acad1fbcaa306b40efa68c8055747aba5c7b29feae9d5e52c7327f3595"}
Sep 04 21:56:56.134710 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:56.134390    2557 scope.go:117] "RemoveContainer" containerID="1eb07c401af55907eb5094d26adfffe4c1b727ae43ac0d1d267ea6b77ef9a901"
Sep 04 21:56:56.143305 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:56.143283    2557 scope.go:117] "RemoveContainer" containerID="1eb07c401af55907eb5094d26adfffe4c1b727ae43ac0d1d267ea6b77ef9a901"
Sep 04 21:56:56.143680 ip-10-0-134-141 kubenswrapper[2557]: E0904 21:56:56.143658    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1eb07c401af55907eb5094d26adfffe4c1b727ae43ac0d1d267ea6b77ef9a901\": container with ID starting with 1eb07c401af55907eb5094d26adfffe4c1b727ae43ac0d1d267ea6b77ef9a901 not found: ID does not exist" containerID="1eb07c401af55907eb5094d26adfffe4c1b727ae43ac0d1d267ea6b77ef9a901"
Sep 04 21:56:56.143743 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:56.143691    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1eb07c401af55907eb5094d26adfffe4c1b727ae43ac0d1d267ea6b77ef9a901"} err="failed to get container status \"1eb07c401af55907eb5094d26adfffe4c1b727ae43ac0d1d267ea6b77ef9a901\": rpc error: code = NotFound desc = could not find container \"1eb07c401af55907eb5094d26adfffe4c1b727ae43ac0d1d267ea6b77ef9a901\": container with ID starting with 1eb07c401af55907eb5094d26adfffe4c1b727ae43ac0d1d267ea6b77ef9a901 not found: ID does not exist"
Sep 04 21:56:56.154875 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:56.154844    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-679d94bfd6-bvcbg"]
Sep 04 21:56:56.158379 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:56.158352    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-679d94bfd6-bvcbg"]
Sep 04 21:56:56.351690 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:56.351657    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19d12fc6-eabf-4dd0-9639-7c8cbd557795" path="/var/lib/kubelet/pods/19d12fc6-eabf-4dd0-9639-7c8cbd557795/volumes"
Sep 04 21:56:59.264254 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:59.264211    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4"]
Sep 04 21:56:59.264729 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:59.264645    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19d12fc6-eabf-4dd0-9639-7c8cbd557795" containerName="console"
Sep 04 21:56:59.264729 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:59.264664    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="19d12fc6-eabf-4dd0-9639-7c8cbd557795" containerName="console"
Sep 04 21:56:59.264861 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:59.264769    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="19d12fc6-eabf-4dd0-9639-7c8cbd557795" containerName="console"
Sep 04 21:56:59.268005 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:59.267980    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4"
Sep 04 21:56:59.270298 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:59.270269    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 04 21:56:59.271126 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:59.271110    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jkcjf\""
Sep 04 21:56:59.271218 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:59.271114    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 04 21:56:59.280646 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:59.280613    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4"]
Sep 04 21:56:59.327897 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:59.327841    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rcf2\" (UniqueName: \"kubernetes.io/projected/b72332af-0b5f-4552-a5ed-17d932f5dac8-kube-api-access-2rcf2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4\" (UID: \"b72332af-0b5f-4552-a5ed-17d932f5dac8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4"
Sep 04 21:56:59.328072 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:59.327910    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b72332af-0b5f-4552-a5ed-17d932f5dac8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4\" (UID: \"b72332af-0b5f-4552-a5ed-17d932f5dac8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4"
Sep 04 21:56:59.328072 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:59.327963    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b72332af-0b5f-4552-a5ed-17d932f5dac8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4\" (UID: \"b72332af-0b5f-4552-a5ed-17d932f5dac8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4"
Sep 04 21:56:59.428719 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:59.428674    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b72332af-0b5f-4552-a5ed-17d932f5dac8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4\" (UID: \"b72332af-0b5f-4552-a5ed-17d932f5dac8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4"
Sep 04 21:56:59.428828 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:59.428745    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rcf2\" (UniqueName: \"kubernetes.io/projected/b72332af-0b5f-4552-a5ed-17d932f5dac8-kube-api-access-2rcf2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4\" (UID: \"b72332af-0b5f-4552-a5ed-17d932f5dac8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4"
Sep 04 21:56:59.428828 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:59.428768    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b72332af-0b5f-4552-a5ed-17d932f5dac8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4\" (UID: \"b72332af-0b5f-4552-a5ed-17d932f5dac8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4"
Sep 04 21:56:59.429088 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:59.429059    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b72332af-0b5f-4552-a5ed-17d932f5dac8-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4\" (UID: \"b72332af-0b5f-4552-a5ed-17d932f5dac8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4"
Sep 04 21:56:59.429124 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:59.429088    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b72332af-0b5f-4552-a5ed-17d932f5dac8-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4\" (UID: \"b72332af-0b5f-4552-a5ed-17d932f5dac8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4"
Sep 04 21:56:59.437723 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:59.437689    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rcf2\" (UniqueName: \"kubernetes.io/projected/b72332af-0b5f-4552-a5ed-17d932f5dac8-kube-api-access-2rcf2\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4\" (UID: \"b72332af-0b5f-4552-a5ed-17d932f5dac8\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4"
Sep 04 21:56:59.577536 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:59.577455    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4"
Sep 04 21:56:59.847835 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:56:59.847801    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4"]
Sep 04 21:56:59.850363 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:56:59.850335    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb72332af_0b5f_4552_a5ed_17d932f5dac8.slice/crio-2e2595af79dc7b2b2b6d2b026801cb4cc0c06ba020733217a5b5d1d6daa9c01f WatchSource:0}: Error finding container 2e2595af79dc7b2b2b6d2b026801cb4cc0c06ba020733217a5b5d1d6daa9c01f: Status 404 returned error can't find the container with id 2e2595af79dc7b2b2b6d2b026801cb4cc0c06ba020733217a5b5d1d6daa9c01f
Sep 04 21:57:00.150699 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:00.150610    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4" event={"ID":"b72332af-0b5f-4552-a5ed-17d932f5dac8","Type":"ContainerStarted","Data":"2e2595af79dc7b2b2b6d2b026801cb4cc0c06ba020733217a5b5d1d6daa9c01f"}
Sep 04 21:57:06.179837 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:06.179804    2557 generic.go:356] "Generic (PLEG): container finished" podID="b72332af-0b5f-4552-a5ed-17d932f5dac8" containerID="72e749c56a201c9dd1f96f7140531df8356bf7643d97943864702eacf8931639" exitCode=0
Sep 04 21:57:06.180195 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:06.179845    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4" event={"ID":"b72332af-0b5f-4552-a5ed-17d932f5dac8","Type":"ContainerDied","Data":"72e749c56a201c9dd1f96f7140531df8356bf7643d97943864702eacf8931639"}
Sep 04 21:57:09.194032 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:09.193994    2557 generic.go:356] "Generic (PLEG): container finished" podID="b72332af-0b5f-4552-a5ed-17d932f5dac8" containerID="4b852a53d980b58ed313f93f0b499d752e00c91bdb6e0b8f77956aa3863a4c63" exitCode=0
Sep 04 21:57:09.194469 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:09.194076    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4" event={"ID":"b72332af-0b5f-4552-a5ed-17d932f5dac8","Type":"ContainerDied","Data":"4b852a53d980b58ed313f93f0b499d752e00c91bdb6e0b8f77956aa3863a4c63"}
Sep 04 21:57:17.229926 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:17.229878    2557 generic.go:356] "Generic (PLEG): container finished" podID="b72332af-0b5f-4552-a5ed-17d932f5dac8" containerID="c71ece6f8bf0368c711a1abe82b265414d98dacd260eb0c6ae4b23770f77f700" exitCode=0
Sep 04 21:57:17.229926 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:17.229926    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4" event={"ID":"b72332af-0b5f-4552-a5ed-17d932f5dac8","Type":"ContainerDied","Data":"c71ece6f8bf0368c711a1abe82b265414d98dacd260eb0c6ae4b23770f77f700"}
Sep 04 21:57:18.362257 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:18.362226    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4"
Sep 04 21:57:18.485991 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:18.485897    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rcf2\" (UniqueName: \"kubernetes.io/projected/b72332af-0b5f-4552-a5ed-17d932f5dac8-kube-api-access-2rcf2\") pod \"b72332af-0b5f-4552-a5ed-17d932f5dac8\" (UID: \"b72332af-0b5f-4552-a5ed-17d932f5dac8\") "
Sep 04 21:57:18.486137 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:18.486085    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b72332af-0b5f-4552-a5ed-17d932f5dac8-util\") pod \"b72332af-0b5f-4552-a5ed-17d932f5dac8\" (UID: \"b72332af-0b5f-4552-a5ed-17d932f5dac8\") "
Sep 04 21:57:18.486137 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:18.486103    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b72332af-0b5f-4552-a5ed-17d932f5dac8-bundle\") pod \"b72332af-0b5f-4552-a5ed-17d932f5dac8\" (UID: \"b72332af-0b5f-4552-a5ed-17d932f5dac8\") "
Sep 04 21:57:18.486833 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:18.486805    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b72332af-0b5f-4552-a5ed-17d932f5dac8-bundle" (OuterVolumeSpecName: "bundle") pod "b72332af-0b5f-4552-a5ed-17d932f5dac8" (UID: "b72332af-0b5f-4552-a5ed-17d932f5dac8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 21:57:18.488125 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:18.488101    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b72332af-0b5f-4552-a5ed-17d932f5dac8-kube-api-access-2rcf2" (OuterVolumeSpecName: "kube-api-access-2rcf2") pod "b72332af-0b5f-4552-a5ed-17d932f5dac8" (UID: "b72332af-0b5f-4552-a5ed-17d932f5dac8"). InnerVolumeSpecName "kube-api-access-2rcf2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 21:57:18.490430 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:18.490399    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b72332af-0b5f-4552-a5ed-17d932f5dac8-util" (OuterVolumeSpecName: "util") pod "b72332af-0b5f-4552-a5ed-17d932f5dac8" (UID: "b72332af-0b5f-4552-a5ed-17d932f5dac8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 21:57:18.587289 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:18.587202    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b72332af-0b5f-4552-a5ed-17d932f5dac8-util\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:57:18.587289 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:18.587278    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b72332af-0b5f-4552-a5ed-17d932f5dac8-bundle\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:57:18.587289 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:18.587289    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rcf2\" (UniqueName: \"kubernetes.io/projected/b72332af-0b5f-4552-a5ed-17d932f5dac8-kube-api-access-2rcf2\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:57:19.239221 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:19.239187    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4" event={"ID":"b72332af-0b5f-4552-a5ed-17d932f5dac8","Type":"ContainerDied","Data":"2e2595af79dc7b2b2b6d2b026801cb4cc0c06ba020733217a5b5d1d6daa9c01f"}
Sep 04 21:57:19.239402 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:19.239238    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e2595af79dc7b2b2b6d2b026801cb4cc0c06ba020733217a5b5d1d6daa9c01f"
Sep 04 21:57:19.239402 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:19.239193    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540vmxd4"
Sep 04 21:57:22.271000 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:22.270965    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d4j7k"]
Sep 04 21:57:22.271383 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:22.271236    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b72332af-0b5f-4552-a5ed-17d932f5dac8" containerName="pull"
Sep 04 21:57:22.271383 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:22.271248    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b72332af-0b5f-4552-a5ed-17d932f5dac8" containerName="pull"
Sep 04 21:57:22.271383 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:22.271265    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b72332af-0b5f-4552-a5ed-17d932f5dac8" containerName="extract"
Sep 04 21:57:22.271383 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:22.271271    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b72332af-0b5f-4552-a5ed-17d932f5dac8" containerName="extract"
Sep 04 21:57:22.271383 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:22.271280    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b72332af-0b5f-4552-a5ed-17d932f5dac8" containerName="util"
Sep 04 21:57:22.271383 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:22.271285    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b72332af-0b5f-4552-a5ed-17d932f5dac8" containerName="util"
Sep 04 21:57:22.271383 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:22.271359    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b72332af-0b5f-4552-a5ed-17d932f5dac8" containerName="extract"
Sep 04 21:57:22.299606 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:22.299576    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d4j7k"
Sep 04 21:57:22.302278 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:22.302248    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-s2vbn\""
Sep 04 21:57:22.308809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:22.302599    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 04 21:57:22.308809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:22.302649    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 04 21:57:22.308809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:22.305966    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d4j7k"]
Sep 04 21:57:22.418239 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:22.418201    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv8qp\" (UniqueName: \"kubernetes.io/projected/73577f5a-cc4b-46c2-8352-791365f2bd71-kube-api-access-hv8qp\") pod \"cert-manager-operator-controller-manager-55c444bf86-d4j7k\" (UID: \"73577f5a-cc4b-46c2-8352-791365f2bd71\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d4j7k"
Sep 04 21:57:22.418407 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:22.418256    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73577f5a-cc4b-46c2-8352-791365f2bd71-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-d4j7k\" (UID: \"73577f5a-cc4b-46c2-8352-791365f2bd71\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d4j7k"
Sep 04 21:57:22.519222 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:22.519175    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv8qp\" (UniqueName: \"kubernetes.io/projected/73577f5a-cc4b-46c2-8352-791365f2bd71-kube-api-access-hv8qp\") pod \"cert-manager-operator-controller-manager-55c444bf86-d4j7k\" (UID: \"73577f5a-cc4b-46c2-8352-791365f2bd71\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d4j7k"
Sep 04 21:57:22.519440 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:22.519239    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73577f5a-cc4b-46c2-8352-791365f2bd71-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-d4j7k\" (UID: \"73577f5a-cc4b-46c2-8352-791365f2bd71\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d4j7k"
Sep 04 21:57:22.519662 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:22.519641    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73577f5a-cc4b-46c2-8352-791365f2bd71-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-d4j7k\" (UID: \"73577f5a-cc4b-46c2-8352-791365f2bd71\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d4j7k"
Sep 04 21:57:22.535921 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:22.535865    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv8qp\" (UniqueName: \"kubernetes.io/projected/73577f5a-cc4b-46c2-8352-791365f2bd71-kube-api-access-hv8qp\") pod \"cert-manager-operator-controller-manager-55c444bf86-d4j7k\" (UID: \"73577f5a-cc4b-46c2-8352-791365f2bd71\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d4j7k"
Sep 04 21:57:22.610134 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:22.610095    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d4j7k"
Sep 04 21:57:22.906180 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:22.906152    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d4j7k"]
Sep 04 21:57:22.908937 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:57:22.908907    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73577f5a_cc4b_46c2_8352_791365f2bd71.slice/crio-a2ba7176e1f4cd7737a6405a0e84f36bfd7e374e444c9e0d600926145981a4c2 WatchSource:0}: Error finding container a2ba7176e1f4cd7737a6405a0e84f36bfd7e374e444c9e0d600926145981a4c2: Status 404 returned error can't find the container with id a2ba7176e1f4cd7737a6405a0e84f36bfd7e374e444c9e0d600926145981a4c2
Sep 04 21:57:23.255809 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:23.255771    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d4j7k" event={"ID":"73577f5a-cc4b-46c2-8352-791365f2bd71","Type":"ContainerStarted","Data":"a2ba7176e1f4cd7737a6405a0e84f36bfd7e374e444c9e0d600926145981a4c2"}
Sep 04 21:57:26.167741 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:26.167698    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8kh9c"]
Sep 04 21:57:26.183258 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:26.183228    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8kh9c"]
Sep 04 21:57:26.183466 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:26.183337    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8kh9c"
Sep 04 21:57:26.185776 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:26.185749    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 04 21:57:26.186644 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:26.186616    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 04 21:57:26.186736 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:26.186630    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-w9kfv\""
Sep 04 21:57:26.257146 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:26.257108    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xslf\" (UniqueName: \"kubernetes.io/projected/d398c6b4-d059-43d7-9066-f6f0a52e05d4-kube-api-access-9xslf\") pod \"cert-manager-webhook-6486ffd997-8kh9c\" (UID: \"d398c6b4-d059-43d7-9066-f6f0a52e05d4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8kh9c"
Sep 04 21:57:26.257345 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:26.257178    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d398c6b4-d059-43d7-9066-f6f0a52e05d4-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8kh9c\" (UID: \"d398c6b4-d059-43d7-9066-f6f0a52e05d4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8kh9c"
Sep 04 21:57:26.272823 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:26.272786    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d4j7k" event={"ID":"73577f5a-cc4b-46c2-8352-791365f2bd71","Type":"ContainerStarted","Data":"681b2eca1c2414d8d0bb61b681b14644c6c1674a4c6379db4dad1dac920e65bc"}
Sep 04 21:57:26.291965 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:26.291909    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-d4j7k" podStartSLOduration=1.905245196 podStartE2EDuration="4.291891285s" totalImagesPullingTime="2.386646089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:57:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:57:26.2907243 +0000 UTC m=+220.607291039" watchObservedRunningTime="2026-09-04 21:57:26.291891285 +0000 UTC m=+220.608458023"
Sep 04 21:57:26.358503 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:26.358471    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d398c6b4-d059-43d7-9066-f6f0a52e05d4-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8kh9c\" (UID: \"d398c6b4-d059-43d7-9066-f6f0a52e05d4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8kh9c"
Sep 04 21:57:26.358677 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:26.358536    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xslf\" (UniqueName: \"kubernetes.io/projected/d398c6b4-d059-43d7-9066-f6f0a52e05d4-kube-api-access-9xslf\") pod \"cert-manager-webhook-6486ffd997-8kh9c\" (UID: \"d398c6b4-d059-43d7-9066-f6f0a52e05d4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8kh9c"
Sep 04 21:57:26.369492 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:26.369452    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d398c6b4-d059-43d7-9066-f6f0a52e05d4-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8kh9c\" (UID: \"d398c6b4-d059-43d7-9066-f6f0a52e05d4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8kh9c"
Sep 04 21:57:26.369638 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:26.369558    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xslf\" (UniqueName: \"kubernetes.io/projected/d398c6b4-d059-43d7-9066-f6f0a52e05d4-kube-api-access-9xslf\") pod \"cert-manager-webhook-6486ffd997-8kh9c\" (UID: \"d398c6b4-d059-43d7-9066-f6f0a52e05d4\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8kh9c"
Sep 04 21:57:26.500943 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:26.500902    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8kh9c"
Sep 04 21:57:26.831961 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:26.831778    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8kh9c"]
Sep 04 21:57:26.834650 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:57:26.834606    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd398c6b4_d059_43d7_9066_f6f0a52e05d4.slice/crio-17ef7e66868509fd0e4c0e514f0bc2cd5b683c93a45e1138b71248e7986eaaab WatchSource:0}: Error finding container 17ef7e66868509fd0e4c0e514f0bc2cd5b683c93a45e1138b71248e7986eaaab: Status 404 returned error can't find the container with id 17ef7e66868509fd0e4c0e514f0bc2cd5b683c93a45e1138b71248e7986eaaab
Sep 04 21:57:27.041373 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:27.041338    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp"]
Sep 04 21:57:27.077851 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:27.077808    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp"]
Sep 04 21:57:27.078013 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:27.077943    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp"
Sep 04 21:57:27.080803 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:27.080769    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jkcjf\""
Sep 04 21:57:27.080973 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:27.080884    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 04 21:57:27.080973 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:27.080884    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 04 21:57:27.163947 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:27.163906    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06e5db57-65d2-40d2-9a5e-305e4d35ab6d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp\" (UID: \"06e5db57-65d2-40d2-9a5e-305e4d35ab6d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp"
Sep 04 21:57:27.163947 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:27.163952    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgxmt\" (UniqueName: \"kubernetes.io/projected/06e5db57-65d2-40d2-9a5e-305e4d35ab6d-kube-api-access-lgxmt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp\" (UID: \"06e5db57-65d2-40d2-9a5e-305e4d35ab6d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp"
Sep 04 21:57:27.164160 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:27.164029    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06e5db57-65d2-40d2-9a5e-305e4d35ab6d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp\" (UID: \"06e5db57-65d2-40d2-9a5e-305e4d35ab6d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp"
Sep 04 21:57:27.265535 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:27.265492    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06e5db57-65d2-40d2-9a5e-305e4d35ab6d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp\" (UID: \"06e5db57-65d2-40d2-9a5e-305e4d35ab6d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp"
Sep 04 21:57:27.265535 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:27.265540    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgxmt\" (UniqueName: \"kubernetes.io/projected/06e5db57-65d2-40d2-9a5e-305e4d35ab6d-kube-api-access-lgxmt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp\" (UID: \"06e5db57-65d2-40d2-9a5e-305e4d35ab6d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp"
Sep 04 21:57:27.265946 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:27.265584    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06e5db57-65d2-40d2-9a5e-305e4d35ab6d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp\" (UID: \"06e5db57-65d2-40d2-9a5e-305e4d35ab6d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp"
Sep 04 21:57:27.265988 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:27.265956    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06e5db57-65d2-40d2-9a5e-305e4d35ab6d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp\" (UID: \"06e5db57-65d2-40d2-9a5e-305e4d35ab6d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp"
Sep 04 21:57:27.265988 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:27.265954    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06e5db57-65d2-40d2-9a5e-305e4d35ab6d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp\" (UID: \"06e5db57-65d2-40d2-9a5e-305e4d35ab6d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp"
Sep 04 21:57:27.273874 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:27.273846    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgxmt\" (UniqueName: \"kubernetes.io/projected/06e5db57-65d2-40d2-9a5e-305e4d35ab6d-kube-api-access-lgxmt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp\" (UID: \"06e5db57-65d2-40d2-9a5e-305e4d35ab6d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp"
Sep 04 21:57:27.278679 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:27.278646    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8kh9c" event={"ID":"d398c6b4-d059-43d7-9066-f6f0a52e05d4","Type":"ContainerStarted","Data":"17ef7e66868509fd0e4c0e514f0bc2cd5b683c93a45e1138b71248e7986eaaab"}
Sep 04 21:57:27.388352 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:27.388253    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp"
Sep 04 21:57:27.677712 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:27.677598    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp"]
Sep 04 21:57:27.679969 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:57:27.679933    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06e5db57_65d2_40d2_9a5e_305e4d35ab6d.slice/crio-74f953f62d6b110450f19cbcb6623fca7fe3a423987b970812ca8708f075ede9 WatchSource:0}: Error finding container 74f953f62d6b110450f19cbcb6623fca7fe3a423987b970812ca8708f075ede9: Status 404 returned error can't find the container with id 74f953f62d6b110450f19cbcb6623fca7fe3a423987b970812ca8708f075ede9
Sep 04 21:57:28.285713 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:28.285671    2557 generic.go:356] "Generic (PLEG): container finished" podID="06e5db57-65d2-40d2-9a5e-305e4d35ab6d" containerID="d684c96e0759e394d27b5c037ff3c38110dffa3b40303f2959f4b9fbc2271732" exitCode=0
Sep 04 21:57:28.286137 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:28.285735    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp" event={"ID":"06e5db57-65d2-40d2-9a5e-305e4d35ab6d","Type":"ContainerDied","Data":"d684c96e0759e394d27b5c037ff3c38110dffa3b40303f2959f4b9fbc2271732"}
Sep 04 21:57:28.286137 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:28.285777    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp" event={"ID":"06e5db57-65d2-40d2-9a5e-305e4d35ab6d","Type":"ContainerStarted","Data":"74f953f62d6b110450f19cbcb6623fca7fe3a423987b970812ca8708f075ede9"}
Sep 04 21:57:30.296689 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:30.296646    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8kh9c" event={"ID":"d398c6b4-d059-43d7-9066-f6f0a52e05d4","Type":"ContainerStarted","Data":"36a45335473a855a90765e0412f40354b39b6048d2432c69c52fcbe88c555dad"}
Sep 04 21:57:30.297104 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:30.296772    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8kh9c"
Sep 04 21:57:30.313588 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:30.313535    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-8kh9c" podStartSLOduration=1.484101139 podStartE2EDuration="4.313512656s" totalImagesPullingTime="2.829411517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:57:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:57:30.313044697 +0000 UTC m=+224.629611437" watchObservedRunningTime="2026-09-04 21:57:30.313512656 +0000 UTC m=+224.630079395"
Sep 04 21:57:31.302767 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:31.302727    2557 generic.go:356] "Generic (PLEG): container finished" podID="06e5db57-65d2-40d2-9a5e-305e4d35ab6d" containerID="e389521590b3e63f79ca7b982aa44e6c5ad9671ef428b6cabc7760f578956fb5" exitCode=0
Sep 04 21:57:31.303143 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:31.302816    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp" event={"ID":"06e5db57-65d2-40d2-9a5e-305e4d35ab6d","Type":"ContainerDied","Data":"e389521590b3e63f79ca7b982aa44e6c5ad9671ef428b6cabc7760f578956fb5"}
Sep 04 21:57:32.309681 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:32.309645    2557 generic.go:356] "Generic (PLEG): container finished" podID="06e5db57-65d2-40d2-9a5e-305e4d35ab6d" containerID="89ad6169e7af095ad72e96338b37aa521a964303af593dbb46a3e526290b05e1" exitCode=0
Sep 04 21:57:32.310050 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:32.309699    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp" event={"ID":"06e5db57-65d2-40d2-9a5e-305e4d35ab6d","Type":"ContainerDied","Data":"89ad6169e7af095ad72e96338b37aa521a964303af593dbb46a3e526290b05e1"}
Sep 04 21:57:33.163656 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:33.163618    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6hgh6"]
Sep 04 21:57:33.166055 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:33.166038    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6hgh6"
Sep 04 21:57:33.168213 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:33.168189    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-lds55\""
Sep 04 21:57:33.174108 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:33.174081    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6hgh6"]
Sep 04 21:57:33.317280 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:33.317245    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e99a4c7d-366a-4681-8642-95bd035bf3ca-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6hgh6\" (UID: \"e99a4c7d-366a-4681-8642-95bd035bf3ca\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6hgh6"
Sep 04 21:57:33.317619 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:33.317361    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89cj5\" (UniqueName: \"kubernetes.io/projected/e99a4c7d-366a-4681-8642-95bd035bf3ca-kube-api-access-89cj5\") pod \"cert-manager-cainjector-559cdd8c87-6hgh6\" (UID: \"e99a4c7d-366a-4681-8642-95bd035bf3ca\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6hgh6"
Sep 04 21:57:33.418286 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:33.418191    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89cj5\" (UniqueName: \"kubernetes.io/projected/e99a4c7d-366a-4681-8642-95bd035bf3ca-kube-api-access-89cj5\") pod \"cert-manager-cainjector-559cdd8c87-6hgh6\" (UID: \"e99a4c7d-366a-4681-8642-95bd035bf3ca\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6hgh6"
Sep 04 21:57:33.418462 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:33.418245    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e99a4c7d-366a-4681-8642-95bd035bf3ca-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6hgh6\" (UID: \"e99a4c7d-366a-4681-8642-95bd035bf3ca\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6hgh6"
Sep 04 21:57:33.426144 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:33.426105    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e99a4c7d-366a-4681-8642-95bd035bf3ca-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6hgh6\" (UID: \"e99a4c7d-366a-4681-8642-95bd035bf3ca\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6hgh6"
Sep 04 21:57:33.426336 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:33.426318    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89cj5\" (UniqueName: \"kubernetes.io/projected/e99a4c7d-366a-4681-8642-95bd035bf3ca-kube-api-access-89cj5\") pod \"cert-manager-cainjector-559cdd8c87-6hgh6\" (UID: \"e99a4c7d-366a-4681-8642-95bd035bf3ca\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6hgh6"
Sep 04 21:57:33.440831 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:33.440806    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp"
Sep 04 21:57:33.475225 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:33.475189    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6hgh6"
Sep 04 21:57:33.519588 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:33.519551    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06e5db57-65d2-40d2-9a5e-305e4d35ab6d-bundle\") pod \"06e5db57-65d2-40d2-9a5e-305e4d35ab6d\" (UID: \"06e5db57-65d2-40d2-9a5e-305e4d35ab6d\") "
Sep 04 21:57:33.519803 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:33.519636    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgxmt\" (UniqueName: \"kubernetes.io/projected/06e5db57-65d2-40d2-9a5e-305e4d35ab6d-kube-api-access-lgxmt\") pod \"06e5db57-65d2-40d2-9a5e-305e4d35ab6d\" (UID: \"06e5db57-65d2-40d2-9a5e-305e4d35ab6d\") "
Sep 04 21:57:33.519803 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:33.519697    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06e5db57-65d2-40d2-9a5e-305e4d35ab6d-util\") pod \"06e5db57-65d2-40d2-9a5e-305e4d35ab6d\" (UID: \"06e5db57-65d2-40d2-9a5e-305e4d35ab6d\") "
Sep 04 21:57:33.520256 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:33.520221    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06e5db57-65d2-40d2-9a5e-305e4d35ab6d-bundle" (OuterVolumeSpecName: "bundle") pod "06e5db57-65d2-40d2-9a5e-305e4d35ab6d" (UID: "06e5db57-65d2-40d2-9a5e-305e4d35ab6d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 21:57:33.523947 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:33.523914    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06e5db57-65d2-40d2-9a5e-305e4d35ab6d-kube-api-access-lgxmt" (OuterVolumeSpecName: "kube-api-access-lgxmt") pod "06e5db57-65d2-40d2-9a5e-305e4d35ab6d" (UID: "06e5db57-65d2-40d2-9a5e-305e4d35ab6d"). InnerVolumeSpecName "kube-api-access-lgxmt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 21:57:33.527581 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:33.527543    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06e5db57-65d2-40d2-9a5e-305e4d35ab6d-util" (OuterVolumeSpecName: "util") pod "06e5db57-65d2-40d2-9a5e-305e4d35ab6d" (UID: "06e5db57-65d2-40d2-9a5e-305e4d35ab6d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 21:57:33.620804 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:33.620767    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgxmt\" (UniqueName: \"kubernetes.io/projected/06e5db57-65d2-40d2-9a5e-305e4d35ab6d-kube-api-access-lgxmt\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:57:33.620804 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:33.620800    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06e5db57-65d2-40d2-9a5e-305e4d35ab6d-util\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:57:33.620804 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:33.620809    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06e5db57-65d2-40d2-9a5e-305e4d35ab6d-bundle\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:57:33.747545 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:33.747519    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6hgh6"]
Sep 04 21:57:33.750485 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:57:33.750453    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode99a4c7d_366a_4681_8642_95bd035bf3ca.slice/crio-7e10e57cfe7aef2c78b72162d4309a4239191d0af40519de3714527d860e03c7 WatchSource:0}: Error finding container 7e10e57cfe7aef2c78b72162d4309a4239191d0af40519de3714527d860e03c7: Status 404 returned error can't find the container with id 7e10e57cfe7aef2c78b72162d4309a4239191d0af40519de3714527d860e03c7
Sep 04 21:57:34.319898 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:34.319857    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6hgh6" event={"ID":"e99a4c7d-366a-4681-8642-95bd035bf3ca","Type":"ContainerStarted","Data":"8da50c49cc247bfe9aa99846db0bc6313584d3539361a8298a75a6549e29e345"}
Sep 04 21:57:34.320349 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:34.319907    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6hgh6" event={"ID":"e99a4c7d-366a-4681-8642-95bd035bf3ca","Type":"ContainerStarted","Data":"7e10e57cfe7aef2c78b72162d4309a4239191d0af40519de3714527d860e03c7"}
Sep 04 21:57:34.321820 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:34.321795    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp" event={"ID":"06e5db57-65d2-40d2-9a5e-305e4d35ab6d","Type":"ContainerDied","Data":"74f953f62d6b110450f19cbcb6623fca7fe3a423987b970812ca8708f075ede9"}
Sep 04 21:57:34.321939 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:34.321830    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fb2thp"
Sep 04 21:57:34.321939 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:34.321837    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="74f953f62d6b110450f19cbcb6623fca7fe3a423987b970812ca8708f075ede9"
Sep 04 21:57:34.335545 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:34.335493    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6hgh6" podStartSLOduration=1.335475703 podStartE2EDuration="1.335475703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:57:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:57:34.33441011 +0000 UTC m=+228.650976848" watchObservedRunningTime="2026-09-04 21:57:34.335475703 +0000 UTC m=+228.652042440"
Sep 04 21:57:36.305851 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:36.305822    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8kh9c"
Sep 04 21:57:51.316463 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.316425    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc"]
Sep 04 21:57:51.316817 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.316721    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06e5db57-65d2-40d2-9a5e-305e4d35ab6d" containerName="util"
Sep 04 21:57:51.316817 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.316732    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="06e5db57-65d2-40d2-9a5e-305e4d35ab6d" containerName="util"
Sep 04 21:57:51.316817 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.316742    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06e5db57-65d2-40d2-9a5e-305e4d35ab6d" containerName="pull"
Sep 04 21:57:51.316817 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.316748    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="06e5db57-65d2-40d2-9a5e-305e4d35ab6d" containerName="pull"
Sep 04 21:57:51.316817 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.316762    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06e5db57-65d2-40d2-9a5e-305e4d35ab6d" containerName="extract"
Sep 04 21:57:51.316817 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.316768    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="06e5db57-65d2-40d2-9a5e-305e4d35ab6d" containerName="extract"
Sep 04 21:57:51.316817 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.316812    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="06e5db57-65d2-40d2-9a5e-305e4d35ab6d" containerName="extract"
Sep 04 21:57:51.319459 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.319436    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc"
Sep 04 21:57:51.322391 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.322367    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 04 21:57:51.322737 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.322719    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 04 21:57:51.323233 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.323217    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jkcjf\""
Sep 04 21:57:51.330860 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.330832    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc"]
Sep 04 21:57:51.366732 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.366691    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ccw55\" (UniqueName: \"kubernetes.io/projected/5c43bb1a-8e16-4a0d-935f-070eeb8d0248-kube-api-access-ccw55\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc\" (UID: \"5c43bb1a-8e16-4a0d-935f-070eeb8d0248\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc"
Sep 04 21:57:51.366906 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.366746    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c43bb1a-8e16-4a0d-935f-070eeb8d0248-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc\" (UID: \"5c43bb1a-8e16-4a0d-935f-070eeb8d0248\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc"
Sep 04 21:57:51.366906 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.366778    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c43bb1a-8e16-4a0d-935f-070eeb8d0248-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc\" (UID: \"5c43bb1a-8e16-4a0d-935f-070eeb8d0248\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc"
Sep 04 21:57:51.467974 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.467942    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ccw55\" (UniqueName: \"kubernetes.io/projected/5c43bb1a-8e16-4a0d-935f-070eeb8d0248-kube-api-access-ccw55\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc\" (UID: \"5c43bb1a-8e16-4a0d-935f-070eeb8d0248\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc"
Sep 04 21:57:51.468145 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.467986    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c43bb1a-8e16-4a0d-935f-070eeb8d0248-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc\" (UID: \"5c43bb1a-8e16-4a0d-935f-070eeb8d0248\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc"
Sep 04 21:57:51.468145 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.468102    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c43bb1a-8e16-4a0d-935f-070eeb8d0248-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc\" (UID: \"5c43bb1a-8e16-4a0d-935f-070eeb8d0248\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc"
Sep 04 21:57:51.468314 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.468298    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c43bb1a-8e16-4a0d-935f-070eeb8d0248-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc\" (UID: \"5c43bb1a-8e16-4a0d-935f-070eeb8d0248\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc"
Sep 04 21:57:51.468408 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.468391    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c43bb1a-8e16-4a0d-935f-070eeb8d0248-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc\" (UID: \"5c43bb1a-8e16-4a0d-935f-070eeb8d0248\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc"
Sep 04 21:57:51.476786 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.476758    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ccw55\" (UniqueName: \"kubernetes.io/projected/5c43bb1a-8e16-4a0d-935f-070eeb8d0248-kube-api-access-ccw55\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc\" (UID: \"5c43bb1a-8e16-4a0d-935f-070eeb8d0248\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc"
Sep 04 21:57:51.628263 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.628169    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc"
Sep 04 21:57:51.901989 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:51.901900    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc"]
Sep 04 21:57:51.905343 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:57:51.905298    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c43bb1a_8e16_4a0d_935f_070eeb8d0248.slice/crio-b5b5bfd942aff1e2a24a6743ea56a8deee697b1dac86a80bcd1f61b7b47808a4 WatchSource:0}: Error finding container b5b5bfd942aff1e2a24a6743ea56a8deee697b1dac86a80bcd1f61b7b47808a4: Status 404 returned error can't find the container with id b5b5bfd942aff1e2a24a6743ea56a8deee697b1dac86a80bcd1f61b7b47808a4
Sep 04 21:57:52.399022 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:52.398984    2557 generic.go:356] "Generic (PLEG): container finished" podID="5c43bb1a-8e16-4a0d-935f-070eeb8d0248" containerID="b58549144c41c90450a3228f807a4a60945f6c97896539b2353f1976b34046fe" exitCode=0
Sep 04 21:57:52.399449 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:52.399055    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc" event={"ID":"5c43bb1a-8e16-4a0d-935f-070eeb8d0248","Type":"ContainerDied","Data":"b58549144c41c90450a3228f807a4a60945f6c97896539b2353f1976b34046fe"}
Sep 04 21:57:52.399449 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:52.399113    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc" event={"ID":"5c43bb1a-8e16-4a0d-935f-070eeb8d0248","Type":"ContainerStarted","Data":"b5b5bfd942aff1e2a24a6743ea56a8deee697b1dac86a80bcd1f61b7b47808a4"}
Sep 04 21:57:53.405786 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:53.405684    2557 generic.go:356] "Generic (PLEG): container finished" podID="5c43bb1a-8e16-4a0d-935f-070eeb8d0248" containerID="95821646ce12dd587e0db9b4f716300629a06bb716f8af7568aa4c549af49cb5" exitCode=0
Sep 04 21:57:53.405786 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:53.405767    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc" event={"ID":"5c43bb1a-8e16-4a0d-935f-070eeb8d0248","Type":"ContainerDied","Data":"95821646ce12dd587e0db9b4f716300629a06bb716f8af7568aa4c549af49cb5"}
Sep 04 21:57:54.412123 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:54.412085    2557 generic.go:356] "Generic (PLEG): container finished" podID="5c43bb1a-8e16-4a0d-935f-070eeb8d0248" containerID="767e1509aaeaa337cff93b6e19795b161b1682666ba33ce9109a7225c0ba4674" exitCode=0
Sep 04 21:57:54.412123 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:54.412128    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc" event={"ID":"5c43bb1a-8e16-4a0d-935f-070eeb8d0248","Type":"ContainerDied","Data":"767e1509aaeaa337cff93b6e19795b161b1682666ba33ce9109a7225c0ba4674"}
Sep 04 21:57:55.542253 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:55.542225    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc"
Sep 04 21:57:55.597997 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:55.597952    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ccw55\" (UniqueName: \"kubernetes.io/projected/5c43bb1a-8e16-4a0d-935f-070eeb8d0248-kube-api-access-ccw55\") pod \"5c43bb1a-8e16-4a0d-935f-070eeb8d0248\" (UID: \"5c43bb1a-8e16-4a0d-935f-070eeb8d0248\") "
Sep 04 21:57:55.597997 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:55.597995    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c43bb1a-8e16-4a0d-935f-070eeb8d0248-util\") pod \"5c43bb1a-8e16-4a0d-935f-070eeb8d0248\" (UID: \"5c43bb1a-8e16-4a0d-935f-070eeb8d0248\") "
Sep 04 21:57:55.598204 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:55.598058    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c43bb1a-8e16-4a0d-935f-070eeb8d0248-bundle\") pod \"5c43bb1a-8e16-4a0d-935f-070eeb8d0248\" (UID: \"5c43bb1a-8e16-4a0d-935f-070eeb8d0248\") "
Sep 04 21:57:55.598806 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:55.598773    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c43bb1a-8e16-4a0d-935f-070eeb8d0248-bundle" (OuterVolumeSpecName: "bundle") pod "5c43bb1a-8e16-4a0d-935f-070eeb8d0248" (UID: "5c43bb1a-8e16-4a0d-935f-070eeb8d0248"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 21:57:55.600178 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:55.600149    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c43bb1a-8e16-4a0d-935f-070eeb8d0248-kube-api-access-ccw55" (OuterVolumeSpecName: "kube-api-access-ccw55") pod "5c43bb1a-8e16-4a0d-935f-070eeb8d0248" (UID: "5c43bb1a-8e16-4a0d-935f-070eeb8d0248"). InnerVolumeSpecName "kube-api-access-ccw55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 21:57:55.603756 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:55.603729    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c43bb1a-8e16-4a0d-935f-070eeb8d0248-util" (OuterVolumeSpecName: "util") pod "5c43bb1a-8e16-4a0d-935f-070eeb8d0248" (UID: "5c43bb1a-8e16-4a0d-935f-070eeb8d0248"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 21:57:55.698666 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:55.698562    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ccw55\" (UniqueName: \"kubernetes.io/projected/5c43bb1a-8e16-4a0d-935f-070eeb8d0248-kube-api-access-ccw55\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:57:55.698666 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:55.698607    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5c43bb1a-8e16-4a0d-935f-070eeb8d0248-util\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:57:55.698666 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:55.698617    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5c43bb1a-8e16-4a0d-935f-070eeb8d0248-bundle\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:57:56.423002 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:56.422970    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc"
Sep 04 21:57:56.423002 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:56.422984    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af8tzmc" event={"ID":"5c43bb1a-8e16-4a0d-935f-070eeb8d0248","Type":"ContainerDied","Data":"b5b5bfd942aff1e2a24a6743ea56a8deee697b1dac86a80bcd1f61b7b47808a4"}
Sep 04 21:57:56.423199 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:57:56.423025    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b5b5bfd942aff1e2a24a6743ea56a8deee697b1dac86a80bcd1f61b7b47808a4"
Sep 04 21:58:01.922158 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:01.922121    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5"]
Sep 04 21:58:01.922761 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:01.922589    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c43bb1a-8e16-4a0d-935f-070eeb8d0248" containerName="util"
Sep 04 21:58:01.922761 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:01.922609    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c43bb1a-8e16-4a0d-935f-070eeb8d0248" containerName="util"
Sep 04 21:58:01.922761 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:01.922622    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c43bb1a-8e16-4a0d-935f-070eeb8d0248" containerName="extract"
Sep 04 21:58:01.922761 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:01.922632    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c43bb1a-8e16-4a0d-935f-070eeb8d0248" containerName="extract"
Sep 04 21:58:01.922761 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:01.922657    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c43bb1a-8e16-4a0d-935f-070eeb8d0248" containerName="pull"
Sep 04 21:58:01.922761 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:01.922668    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c43bb1a-8e16-4a0d-935f-070eeb8d0248" containerName="pull"
Sep 04 21:58:01.922761 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:01.922737    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c43bb1a-8e16-4a0d-935f-070eeb8d0248" containerName="extract"
Sep 04 21:58:01.927492 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:01.927467    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5"
Sep 04 21:58:01.929839 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:01.929811    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jkcjf\""
Sep 04 21:58:01.930185 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:01.930169    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 04 21:58:01.930587 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:01.930569    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 04 21:58:01.939326 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:01.939298    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5"]
Sep 04 21:58:01.952128 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:01.952074    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67a35fc9-d315-4221-8d7d-e102a0ec53bb-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5\" (UID: \"67a35fc9-d315-4221-8d7d-e102a0ec53bb\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5"
Sep 04 21:58:01.952317 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:01.952295    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67a35fc9-d315-4221-8d7d-e102a0ec53bb-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5\" (UID: \"67a35fc9-d315-4221-8d7d-e102a0ec53bb\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5"
Sep 04 21:58:01.952485 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:01.952459    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqrxh\" (UniqueName: \"kubernetes.io/projected/67a35fc9-d315-4221-8d7d-e102a0ec53bb-kube-api-access-kqrxh\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5\" (UID: \"67a35fc9-d315-4221-8d7d-e102a0ec53bb\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5"
Sep 04 21:58:02.053277 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:02.053228    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqrxh\" (UniqueName: \"kubernetes.io/projected/67a35fc9-d315-4221-8d7d-e102a0ec53bb-kube-api-access-kqrxh\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5\" (UID: \"67a35fc9-d315-4221-8d7d-e102a0ec53bb\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5"
Sep 04 21:58:02.053477 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:02.053315    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67a35fc9-d315-4221-8d7d-e102a0ec53bb-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5\" (UID: \"67a35fc9-d315-4221-8d7d-e102a0ec53bb\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5"
Sep 04 21:58:02.053477 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:02.053345    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67a35fc9-d315-4221-8d7d-e102a0ec53bb-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5\" (UID: \"67a35fc9-d315-4221-8d7d-e102a0ec53bb\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5"
Sep 04 21:58:02.053782 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:02.053762    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67a35fc9-d315-4221-8d7d-e102a0ec53bb-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5\" (UID: \"67a35fc9-d315-4221-8d7d-e102a0ec53bb\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5"
Sep 04 21:58:02.053818 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:02.053771    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67a35fc9-d315-4221-8d7d-e102a0ec53bb-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5\" (UID: \"67a35fc9-d315-4221-8d7d-e102a0ec53bb\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5"
Sep 04 21:58:02.067088 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:02.067056    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqrxh\" (UniqueName: \"kubernetes.io/projected/67a35fc9-d315-4221-8d7d-e102a0ec53bb-kube-api-access-kqrxh\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5\" (UID: \"67a35fc9-d315-4221-8d7d-e102a0ec53bb\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5"
Sep 04 21:58:02.236386 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:02.236291    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5"
Sep 04 21:58:02.520260 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:02.520227    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5"]
Sep 04 21:58:02.523353 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:58:02.523315    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67a35fc9_d315_4221_8d7d_e102a0ec53bb.slice/crio-721cbfe9df637721e1ed0c22f8c73366c8bf81e2cd110d6ae6c29b6d8c052901 WatchSource:0}: Error finding container 721cbfe9df637721e1ed0c22f8c73366c8bf81e2cd110d6ae6c29b6d8c052901: Status 404 returned error can't find the container with id 721cbfe9df637721e1ed0c22f8c73366c8bf81e2cd110d6ae6c29b6d8c052901
Sep 04 21:58:03.454748 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.454618    2557 generic.go:356] "Generic (PLEG): container finished" podID="67a35fc9-d315-4221-8d7d-e102a0ec53bb" containerID="0c1c169c3c8471bd6faf7dc62a5329f50cecbb2ffcde49abd5e5f76f09f183a3" exitCode=0
Sep 04 21:58:03.454748 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.454710    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5" event={"ID":"67a35fc9-d315-4221-8d7d-e102a0ec53bb","Type":"ContainerDied","Data":"0c1c169c3c8471bd6faf7dc62a5329f50cecbb2ffcde49abd5e5f76f09f183a3"}
Sep 04 21:58:03.455250 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.454764    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5" event={"ID":"67a35fc9-d315-4221-8d7d-e102a0ec53bb","Type":"ContainerStarted","Data":"721cbfe9df637721e1ed0c22f8c73366c8bf81e2cd110d6ae6c29b6d8c052901"}
Sep 04 21:58:03.832237 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.832194    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd"]
Sep 04 21:58:03.835259 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.835236    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd"
Sep 04 21:58:03.839692 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.839661    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 04 21:58:03.839865 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.839729    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jrmss\""
Sep 04 21:58:03.840039 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.840016    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 04 21:58:03.840146 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.840017    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 04 21:58:03.840760 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.840744    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 04 21:58:03.858390 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.858352    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd"]
Sep 04 21:58:03.870191 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.870156    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c8ffe98-d551-44a2-ab6a-53ee22459e1d-tmp\") pod \"opendatahub-operator-controller-manager-764b98687b-dtxjd\" (UID: \"9c8ffe98-d551-44a2-ab6a-53ee22459e1d\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd"
Sep 04 21:58:03.870365 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.870222    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9c8ffe98-d551-44a2-ab6a-53ee22459e1d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-764b98687b-dtxjd\" (UID: \"9c8ffe98-d551-44a2-ab6a-53ee22459e1d\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd"
Sep 04 21:58:03.870365 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.870240    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c8ffe98-d551-44a2-ab6a-53ee22459e1d-webhook-cert\") pod \"opendatahub-operator-controller-manager-764b98687b-dtxjd\" (UID: \"9c8ffe98-d551-44a2-ab6a-53ee22459e1d\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd"
Sep 04 21:58:03.870365 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.870276    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9c8ffe98-d551-44a2-ab6a-53ee22459e1d-manifests\") pod \"opendatahub-operator-controller-manager-764b98687b-dtxjd\" (UID: \"9c8ffe98-d551-44a2-ab6a-53ee22459e1d\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd"
Sep 04 21:58:03.870365 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.870291    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r2r6\" (UniqueName: \"kubernetes.io/projected/9c8ffe98-d551-44a2-ab6a-53ee22459e1d-kube-api-access-8r2r6\") pod \"opendatahub-operator-controller-manager-764b98687b-dtxjd\" (UID: \"9c8ffe98-d551-44a2-ab6a-53ee22459e1d\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd"
Sep 04 21:58:03.971626 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.971589    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9c8ffe98-d551-44a2-ab6a-53ee22459e1d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-764b98687b-dtxjd\" (UID: \"9c8ffe98-d551-44a2-ab6a-53ee22459e1d\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd"
Sep 04 21:58:03.971626 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.971627    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c8ffe98-d551-44a2-ab6a-53ee22459e1d-webhook-cert\") pod \"opendatahub-operator-controller-manager-764b98687b-dtxjd\" (UID: \"9c8ffe98-d551-44a2-ab6a-53ee22459e1d\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd"
Sep 04 21:58:03.971892 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.971648    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9c8ffe98-d551-44a2-ab6a-53ee22459e1d-manifests\") pod \"opendatahub-operator-controller-manager-764b98687b-dtxjd\" (UID: \"9c8ffe98-d551-44a2-ab6a-53ee22459e1d\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd"
Sep 04 21:58:03.971892 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.971665    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r2r6\" (UniqueName: \"kubernetes.io/projected/9c8ffe98-d551-44a2-ab6a-53ee22459e1d-kube-api-access-8r2r6\") pod \"opendatahub-operator-controller-manager-764b98687b-dtxjd\" (UID: \"9c8ffe98-d551-44a2-ab6a-53ee22459e1d\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd"
Sep 04 21:58:03.971892 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.971697    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c8ffe98-d551-44a2-ab6a-53ee22459e1d-tmp\") pod \"opendatahub-operator-controller-manager-764b98687b-dtxjd\" (UID: \"9c8ffe98-d551-44a2-ab6a-53ee22459e1d\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd"
Sep 04 21:58:03.972039 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.972024    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9c8ffe98-d551-44a2-ab6a-53ee22459e1d-manifests\") pod \"opendatahub-operator-controller-manager-764b98687b-dtxjd\" (UID: \"9c8ffe98-d551-44a2-ab6a-53ee22459e1d\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd"
Sep 04 21:58:03.973942 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.973900    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c8ffe98-d551-44a2-ab6a-53ee22459e1d-tmp\") pod \"opendatahub-operator-controller-manager-764b98687b-dtxjd\" (UID: \"9c8ffe98-d551-44a2-ab6a-53ee22459e1d\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd"
Sep 04 21:58:03.974241 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.974220    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9c8ffe98-d551-44a2-ab6a-53ee22459e1d-webhook-cert\") pod \"opendatahub-operator-controller-manager-764b98687b-dtxjd\" (UID: \"9c8ffe98-d551-44a2-ab6a-53ee22459e1d\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd"
Sep 04 21:58:03.974340 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.974316    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9c8ffe98-d551-44a2-ab6a-53ee22459e1d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-764b98687b-dtxjd\" (UID: \"9c8ffe98-d551-44a2-ab6a-53ee22459e1d\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd"
Sep 04 21:58:03.984075 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:03.984033    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r2r6\" (UniqueName: \"kubernetes.io/projected/9c8ffe98-d551-44a2-ab6a-53ee22459e1d-kube-api-access-8r2r6\") pod \"opendatahub-operator-controller-manager-764b98687b-dtxjd\" (UID: \"9c8ffe98-d551-44a2-ab6a-53ee22459e1d\") " pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd"
Sep 04 21:58:04.146128 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:04.146096    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd"
Sep 04 21:58:04.428260 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:04.428232    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd"]
Sep 04 21:58:04.429108 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:58:04.429082    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c8ffe98_d551_44a2_ab6a_53ee22459e1d.slice/crio-6d71997372339d77966dbce0d926a784f75bd2861cb06d30455f876c21269334 WatchSource:0}: Error finding container 6d71997372339d77966dbce0d926a784f75bd2861cb06d30455f876c21269334: Status 404 returned error can't find the container with id 6d71997372339d77966dbce0d926a784f75bd2861cb06d30455f876c21269334
Sep 04 21:58:04.460975 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:04.460939    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd" event={"ID":"9c8ffe98-d551-44a2-ab6a-53ee22459e1d","Type":"ContainerStarted","Data":"6d71997372339d77966dbce0d926a784f75bd2861cb06d30455f876c21269334"}
Sep 04 21:58:04.462893 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:04.462861    2557 generic.go:356] "Generic (PLEG): container finished" podID="67a35fc9-d315-4221-8d7d-e102a0ec53bb" containerID="85e3ba99a7e2b5e23d26731d554728eaf4e048565f01406e4f8bf33de7303599" exitCode=0
Sep 04 21:58:04.463033 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:04.462950    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5" event={"ID":"67a35fc9-d315-4221-8d7d-e102a0ec53bb","Type":"ContainerDied","Data":"85e3ba99a7e2b5e23d26731d554728eaf4e048565f01406e4f8bf33de7303599"}
Sep 04 21:58:05.471721 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:05.471677    2557 generic.go:356] "Generic (PLEG): container finished" podID="67a35fc9-d315-4221-8d7d-e102a0ec53bb" containerID="b69ba0fa7456abb36883137e14730d8e285001625566df9c5ca97c4197965411" exitCode=0
Sep 04 21:58:05.472187 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:05.471741    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5" event={"ID":"67a35fc9-d315-4221-8d7d-e102a0ec53bb","Type":"ContainerDied","Data":"b69ba0fa7456abb36883137e14730d8e285001625566df9c5ca97c4197965411"}
Sep 04 21:58:07.437171 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:07.437141    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5"
Sep 04 21:58:07.482858 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:07.482832    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5"
Sep 04 21:58:07.482921 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:07.482848    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0jl9p5" event={"ID":"67a35fc9-d315-4221-8d7d-e102a0ec53bb","Type":"ContainerDied","Data":"721cbfe9df637721e1ed0c22f8c73366c8bf81e2cd110d6ae6c29b6d8c052901"}
Sep 04 21:58:07.482921 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:07.482897    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="721cbfe9df637721e1ed0c22f8c73366c8bf81e2cd110d6ae6c29b6d8c052901"
Sep 04 21:58:07.504194 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:07.504157    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67a35fc9-d315-4221-8d7d-e102a0ec53bb-util\") pod \"67a35fc9-d315-4221-8d7d-e102a0ec53bb\" (UID: \"67a35fc9-d315-4221-8d7d-e102a0ec53bb\") "
Sep 04 21:58:07.504357 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:07.504284    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67a35fc9-d315-4221-8d7d-e102a0ec53bb-bundle\") pod \"67a35fc9-d315-4221-8d7d-e102a0ec53bb\" (UID: \"67a35fc9-d315-4221-8d7d-e102a0ec53bb\") "
Sep 04 21:58:07.504357 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:07.504317    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqrxh\" (UniqueName: \"kubernetes.io/projected/67a35fc9-d315-4221-8d7d-e102a0ec53bb-kube-api-access-kqrxh\") pod \"67a35fc9-d315-4221-8d7d-e102a0ec53bb\" (UID: \"67a35fc9-d315-4221-8d7d-e102a0ec53bb\") "
Sep 04 21:58:07.505048 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:07.505013    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67a35fc9-d315-4221-8d7d-e102a0ec53bb-bundle" (OuterVolumeSpecName: "bundle") pod "67a35fc9-d315-4221-8d7d-e102a0ec53bb" (UID: "67a35fc9-d315-4221-8d7d-e102a0ec53bb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 21:58:07.506368 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:07.506348    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67a35fc9-d315-4221-8d7d-e102a0ec53bb-kube-api-access-kqrxh" (OuterVolumeSpecName: "kube-api-access-kqrxh") pod "67a35fc9-d315-4221-8d7d-e102a0ec53bb" (UID: "67a35fc9-d315-4221-8d7d-e102a0ec53bb"). InnerVolumeSpecName "kube-api-access-kqrxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 21:58:07.509965 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:07.509918    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67a35fc9-d315-4221-8d7d-e102a0ec53bb-util" (OuterVolumeSpecName: "util") pod "67a35fc9-d315-4221-8d7d-e102a0ec53bb" (UID: "67a35fc9-d315-4221-8d7d-e102a0ec53bb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 21:58:07.605760 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:07.605723    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/67a35fc9-d315-4221-8d7d-e102a0ec53bb-bundle\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:58:07.605760 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:07.605756    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqrxh\" (UniqueName: \"kubernetes.io/projected/67a35fc9-d315-4221-8d7d-e102a0ec53bb-kube-api-access-kqrxh\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:58:07.605760 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:07.605768    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/67a35fc9-d315-4221-8d7d-e102a0ec53bb-util\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:58:08.489970 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:08.489875    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd" event={"ID":"9c8ffe98-d551-44a2-ab6a-53ee22459e1d","Type":"ContainerStarted","Data":"c2d86dba27a142e7fa3f21998311dc6f60fa82b009e579a127ad99e700aec2a8"}
Sep 04 21:58:09.496714 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:09.496669    2557 generic.go:356] "Generic (PLEG): container finished" podID="9c8ffe98-d551-44a2-ab6a-53ee22459e1d" containerID="c2d86dba27a142e7fa3f21998311dc6f60fa82b009e579a127ad99e700aec2a8" exitCode=0
Sep 04 21:58:09.497141 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:09.496720    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd" event={"ID":"9c8ffe98-d551-44a2-ab6a-53ee22459e1d","Type":"ContainerDied","Data":"c2d86dba27a142e7fa3f21998311dc6f60fa82b009e579a127ad99e700aec2a8"}
Sep 04 21:58:10.507576 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:10.507540    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd" event={"ID":"9c8ffe98-d551-44a2-ab6a-53ee22459e1d","Type":"ContainerStarted","Data":"738f83e2b28dd0fe5f2a88abf707cbbe9e5247c1ff27d4c4c1e16a65849d1dc3"}
Sep 04 21:58:10.507993 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:10.507638    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd"
Sep 04 21:58:10.529984 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:10.529928    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd" podStartSLOduration=3.009900088 podStartE2EDuration="7.529908386s" totalImagesPullingTime="3.384196255s" totalInitContainerRuntime="1.135812043s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:58:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:58:10.529330417 +0000 UTC m=+264.845897155" watchObservedRunningTime="2026-09-04 21:58:10.529908386 +0000 UTC m=+264.846475125"
Sep 04 21:58:21.515035 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:21.515003    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-764b98687b-dtxjd"
Sep 04 21:58:23.859703 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:23.859663    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-587bddd5d4-qppmg"]
Sep 04 21:58:23.860093 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:23.859980    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67a35fc9-d315-4221-8d7d-e102a0ec53bb" containerName="extract"
Sep 04 21:58:23.860093 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:23.859993    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="67a35fc9-d315-4221-8d7d-e102a0ec53bb" containerName="extract"
Sep 04 21:58:23.860093 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:23.860005    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67a35fc9-d315-4221-8d7d-e102a0ec53bb" containerName="pull"
Sep 04 21:58:23.860093 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:23.860010    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="67a35fc9-d315-4221-8d7d-e102a0ec53bb" containerName="pull"
Sep 04 21:58:23.860093 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:23.860022    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67a35fc9-d315-4221-8d7d-e102a0ec53bb" containerName="util"
Sep 04 21:58:23.860093 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:23.860028    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="67a35fc9-d315-4221-8d7d-e102a0ec53bb" containerName="util"
Sep 04 21:58:23.860093 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:23.860076    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="67a35fc9-d315-4221-8d7d-e102a0ec53bb" containerName="extract"
Sep 04 21:58:23.868196 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:23.868172    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qppmg"
Sep 04 21:58:23.870536 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:23.870505    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 04 21:58:23.871649 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:23.871626    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 04 21:58:23.871822 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:23.871795    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 04 21:58:23.871883 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:23.871848    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 04 21:58:23.871932 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:23.871883    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 04 21:58:23.871978 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:23.871941    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ms7cd\""
Sep 04 21:58:23.877207 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:23.877178    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-587bddd5d4-qppmg"]
Sep 04 21:58:23.944366 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:23.944322    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eafa9ce5-8bfc-49e9-8579-dabf50b16b1a-metrics-cert\") pod \"lws-controller-manager-587bddd5d4-qppmg\" (UID: \"eafa9ce5-8bfc-49e9-8579-dabf50b16b1a\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qppmg"
Sep 04 21:58:23.944366 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:23.944371    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eafa9ce5-8bfc-49e9-8579-dabf50b16b1a-cert\") pod \"lws-controller-manager-587bddd5d4-qppmg\" (UID: \"eafa9ce5-8bfc-49e9-8579-dabf50b16b1a\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qppmg"
Sep 04 21:58:23.944646 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:23.944514    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd6hp\" (UniqueName: \"kubernetes.io/projected/eafa9ce5-8bfc-49e9-8579-dabf50b16b1a-kube-api-access-hd6hp\") pod \"lws-controller-manager-587bddd5d4-qppmg\" (UID: \"eafa9ce5-8bfc-49e9-8579-dabf50b16b1a\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qppmg"
Sep 04 21:58:23.944646 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:23.944548    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/eafa9ce5-8bfc-49e9-8579-dabf50b16b1a-manager-config\") pod \"lws-controller-manager-587bddd5d4-qppmg\" (UID: \"eafa9ce5-8bfc-49e9-8579-dabf50b16b1a\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qppmg"
Sep 04 21:58:24.045690 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:24.045648    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd6hp\" (UniqueName: \"kubernetes.io/projected/eafa9ce5-8bfc-49e9-8579-dabf50b16b1a-kube-api-access-hd6hp\") pod \"lws-controller-manager-587bddd5d4-qppmg\" (UID: \"eafa9ce5-8bfc-49e9-8579-dabf50b16b1a\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qppmg"
Sep 04 21:58:24.045690 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:24.045690    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/eafa9ce5-8bfc-49e9-8579-dabf50b16b1a-manager-config\") pod \"lws-controller-manager-587bddd5d4-qppmg\" (UID: \"eafa9ce5-8bfc-49e9-8579-dabf50b16b1a\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qppmg"
Sep 04 21:58:24.045959 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:24.045867    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eafa9ce5-8bfc-49e9-8579-dabf50b16b1a-metrics-cert\") pod \"lws-controller-manager-587bddd5d4-qppmg\" (UID: \"eafa9ce5-8bfc-49e9-8579-dabf50b16b1a\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qppmg"
Sep 04 21:58:24.045959 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:24.045907    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eafa9ce5-8bfc-49e9-8579-dabf50b16b1a-cert\") pod \"lws-controller-manager-587bddd5d4-qppmg\" (UID: \"eafa9ce5-8bfc-49e9-8579-dabf50b16b1a\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qppmg"
Sep 04 21:58:24.046349 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:24.046321    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/eafa9ce5-8bfc-49e9-8579-dabf50b16b1a-manager-config\") pod \"lws-controller-manager-587bddd5d4-qppmg\" (UID: \"eafa9ce5-8bfc-49e9-8579-dabf50b16b1a\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qppmg"
Sep 04 21:58:24.048453 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:24.048399    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/eafa9ce5-8bfc-49e9-8579-dabf50b16b1a-metrics-cert\") pod \"lws-controller-manager-587bddd5d4-qppmg\" (UID: \"eafa9ce5-8bfc-49e9-8579-dabf50b16b1a\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qppmg"
Sep 04 21:58:24.048579 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:24.048537    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eafa9ce5-8bfc-49e9-8579-dabf50b16b1a-cert\") pod \"lws-controller-manager-587bddd5d4-qppmg\" (UID: \"eafa9ce5-8bfc-49e9-8579-dabf50b16b1a\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qppmg"
Sep 04 21:58:24.054116 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:24.054087    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd6hp\" (UniqueName: \"kubernetes.io/projected/eafa9ce5-8bfc-49e9-8579-dabf50b16b1a-kube-api-access-hd6hp\") pod \"lws-controller-manager-587bddd5d4-qppmg\" (UID: \"eafa9ce5-8bfc-49e9-8579-dabf50b16b1a\") " pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qppmg"
Sep 04 21:58:24.179205 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:24.179111    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qppmg"
Sep 04 21:58:24.452470 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:24.452446    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-587bddd5d4-qppmg"]
Sep 04 21:58:24.455076 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:58:24.455050    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeafa9ce5_8bfc_49e9_8579_dabf50b16b1a.slice/crio-97f5bdb77e575cc766bf036e28bd5c7fe3a55d19dba02c0e7f04f9f6926be029 WatchSource:0}: Error finding container 97f5bdb77e575cc766bf036e28bd5c7fe3a55d19dba02c0e7f04f9f6926be029: Status 404 returned error can't find the container with id 97f5bdb77e575cc766bf036e28bd5c7fe3a55d19dba02c0e7f04f9f6926be029
Sep 04 21:58:24.569857 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:24.569816    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qppmg" event={"ID":"eafa9ce5-8bfc-49e9-8579-dabf50b16b1a","Type":"ContainerStarted","Data":"97f5bdb77e575cc766bf036e28bd5c7fe3a55d19dba02c0e7f04f9f6926be029"}
Sep 04 21:58:26.581529 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:26.581488    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qppmg" event={"ID":"eafa9ce5-8bfc-49e9-8579-dabf50b16b1a","Type":"ContainerStarted","Data":"020d5a83bccf325e25a372e68928e7cbb0d2e966d0f0d03e0fda8ae5030e955c"}
Sep 04 21:58:26.581907 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:26.581622    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qppmg"
Sep 04 21:58:26.603672 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:26.603618    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qppmg" podStartSLOduration=1.9580034419999999 podStartE2EDuration="3.603600231s" totalImagesPullingTime="1.645596789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:58:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:58:26.602288531 +0000 UTC m=+280.918855292" watchObservedRunningTime="2026-09-04 21:58:26.603600231 +0000 UTC m=+280.920166969"
Sep 04 21:58:37.587624 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:37.587591    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-587bddd5d4-qppmg"
Sep 04 21:58:42.926553 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:42.926512    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"]
Sep 04 21:58:42.928906 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:42.928881    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:42.932681 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:42.932646    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 04 21:58:42.932819 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:42.932692    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 04 21:58:42.932819 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:42.932740    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-rq85s\""
Sep 04 21:58:42.932988 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:42.932973    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 04 21:58:42.949850 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:42.949814    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"]
Sep 04 21:58:43.013143 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.013110    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1f7a723-2ac2-494d-99c1-8343f76130bb-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.013318 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.013156    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1f7a723-2ac2-494d-99c1-8343f76130bb-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.013318 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.013186    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1f7a723-2ac2-494d-99c1-8343f76130bb-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.013318 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.013226    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1f7a723-2ac2-494d-99c1-8343f76130bb-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.013318 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.013257    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1f7a723-2ac2-494d-99c1-8343f76130bb-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.013318 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.013307    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1f7a723-2ac2-494d-99c1-8343f76130bb-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.013541 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.013326    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6ggd\" (UniqueName: \"kubernetes.io/projected/a1f7a723-2ac2-494d-99c1-8343f76130bb-kube-api-access-m6ggd\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.013541 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.013354    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1f7a723-2ac2-494d-99c1-8343f76130bb-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.013541 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.013389    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1f7a723-2ac2-494d-99c1-8343f76130bb-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.013942 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.013919    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"]
Sep 04 21:58:43.016203 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.016181    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:43.018721 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.018694    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 04 21:58:43.018866 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.018847    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 04 21:58:43.019039 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.019018    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-ccwvs\""
Sep 04 21:58:43.019140 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.019069    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 04 21:58:43.031608 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.031571    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"]
Sep 04 21:58:43.113877 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.113834    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1f7a723-2ac2-494d-99c1-8343f76130bb-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.113877 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.113878    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6ggd\" (UniqueName: \"kubernetes.io/projected/a1f7a723-2ac2-494d-99c1-8343f76130bb-kube-api-access-m6ggd\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.114137 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.113901    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1f7a723-2ac2-494d-99c1-8343f76130bb-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.114137 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.113923    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1f7a723-2ac2-494d-99c1-8343f76130bb-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.114137 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.113941    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1f7a723-2ac2-494d-99c1-8343f76130bb-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.114137 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.113962    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d94c6c55-a84c-4c9c-8f75-f772730fa0dd-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-rmlv4\" (UID: \"d94c6c55-a84c-4c9c-8f75-f772730fa0dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:43.114137 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.113982    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d94c6c55-a84c-4c9c-8f75-f772730fa0dd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-rmlv4\" (UID: \"d94c6c55-a84c-4c9c-8f75-f772730fa0dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:43.114137 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.114001    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1f7a723-2ac2-494d-99c1-8343f76130bb-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.114137 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.114066    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr62f\" (UniqueName: \"kubernetes.io/projected/d94c6c55-a84c-4c9c-8f75-f772730fa0dd-kube-api-access-fr62f\") pod \"istiod-openshift-gateway-5677ccb49b-rmlv4\" (UID: \"d94c6c55-a84c-4c9c-8f75-f772730fa0dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:43.114137 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.114135    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d94c6c55-a84c-4c9c-8f75-f772730fa0dd-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-rmlv4\" (UID: \"d94c6c55-a84c-4c9c-8f75-f772730fa0dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:43.114582 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.114166    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d94c6c55-a84c-4c9c-8f75-f772730fa0dd-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-rmlv4\" (UID: \"d94c6c55-a84c-4c9c-8f75-f772730fa0dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:43.114582 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.114203    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d94c6c55-a84c-4c9c-8f75-f772730fa0dd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-rmlv4\" (UID: \"d94c6c55-a84c-4c9c-8f75-f772730fa0dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:43.114582 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.114261    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1f7a723-2ac2-494d-99c1-8343f76130bb-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.114582 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.114316    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d94c6c55-a84c-4c9c-8f75-f772730fa0dd-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-rmlv4\" (UID: \"d94c6c55-a84c-4c9c-8f75-f772730fa0dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:43.114582 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.114345    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1f7a723-2ac2-494d-99c1-8343f76130bb-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.114582 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.114408    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1f7a723-2ac2-494d-99c1-8343f76130bb-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.114582 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.114506    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1f7a723-2ac2-494d-99c1-8343f76130bb-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.114582 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.114570    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1f7a723-2ac2-494d-99c1-8343f76130bb-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.114910 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.114659    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1f7a723-2ac2-494d-99c1-8343f76130bb-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.114910 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.114751    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1f7a723-2ac2-494d-99c1-8343f76130bb-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.114910 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.114761    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1f7a723-2ac2-494d-99c1-8343f76130bb-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.116592 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.116576    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1f7a723-2ac2-494d-99c1-8343f76130bb-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.116675 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.116659    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1f7a723-2ac2-494d-99c1-8343f76130bb-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.122280 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.122250    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6ggd\" (UniqueName: \"kubernetes.io/projected/a1f7a723-2ac2-494d-99c1-8343f76130bb-kube-api-access-m6ggd\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.122280 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.122276    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1f7a723-2ac2-494d-99c1-8343f76130bb-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr\" (UID: \"a1f7a723-2ac2-494d-99c1-8343f76130bb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.215225 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.215137    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d94c6c55-a84c-4c9c-8f75-f772730fa0dd-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-rmlv4\" (UID: \"d94c6c55-a84c-4c9c-8f75-f772730fa0dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:43.215225 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.215183    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d94c6c55-a84c-4c9c-8f75-f772730fa0dd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-rmlv4\" (UID: \"d94c6c55-a84c-4c9c-8f75-f772730fa0dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:43.215225 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.215214    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr62f\" (UniqueName: \"kubernetes.io/projected/d94c6c55-a84c-4c9c-8f75-f772730fa0dd-kube-api-access-fr62f\") pod \"istiod-openshift-gateway-5677ccb49b-rmlv4\" (UID: \"d94c6c55-a84c-4c9c-8f75-f772730fa0dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:43.215551 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.215239    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d94c6c55-a84c-4c9c-8f75-f772730fa0dd-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-rmlv4\" (UID: \"d94c6c55-a84c-4c9c-8f75-f772730fa0dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:43.215551 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.215262    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d94c6c55-a84c-4c9c-8f75-f772730fa0dd-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-rmlv4\" (UID: \"d94c6c55-a84c-4c9c-8f75-f772730fa0dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:43.215551 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.215283    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d94c6c55-a84c-4c9c-8f75-f772730fa0dd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-rmlv4\" (UID: \"d94c6c55-a84c-4c9c-8f75-f772730fa0dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:43.215551 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.215318    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d94c6c55-a84c-4c9c-8f75-f772730fa0dd-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-rmlv4\" (UID: \"d94c6c55-a84c-4c9c-8f75-f772730fa0dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:43.216016 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.215974    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d94c6c55-a84c-4c9c-8f75-f772730fa0dd-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-rmlv4\" (UID: \"d94c6c55-a84c-4c9c-8f75-f772730fa0dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:43.217720 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.217694    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d94c6c55-a84c-4c9c-8f75-f772730fa0dd-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-rmlv4\" (UID: \"d94c6c55-a84c-4c9c-8f75-f772730fa0dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:43.217906 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.217881    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d94c6c55-a84c-4c9c-8f75-f772730fa0dd-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-rmlv4\" (UID: \"d94c6c55-a84c-4c9c-8f75-f772730fa0dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:43.218100 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.218075    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d94c6c55-a84c-4c9c-8f75-f772730fa0dd-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-rmlv4\" (UID: \"d94c6c55-a84c-4c9c-8f75-f772730fa0dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:43.218166 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.218102    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d94c6c55-a84c-4c9c-8f75-f772730fa0dd-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-rmlv4\" (UID: \"d94c6c55-a84c-4c9c-8f75-f772730fa0dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:43.226528 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.226499    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d94c6c55-a84c-4c9c-8f75-f772730fa0dd-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-rmlv4\" (UID: \"d94c6c55-a84c-4c9c-8f75-f772730fa0dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:43.226661 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.226548    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr62f\" (UniqueName: \"kubernetes.io/projected/d94c6c55-a84c-4c9c-8f75-f772730fa0dd-kube-api-access-fr62f\") pod \"istiod-openshift-gateway-5677ccb49b-rmlv4\" (UID: \"d94c6c55-a84c-4c9c-8f75-f772730fa0dd\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:43.239453 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.239402    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:43.326051 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.326014    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:43.537300 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.537272    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"]
Sep 04 21:58:43.539430 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:58:43.539386    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1f7a723_2ac2_494d_99c1_8343f76130bb.slice/crio-05bc1dee9fa3f43552ad544d9b0a01f035d84d33ad35bfdede3dc8afc664e763 WatchSource:0}: Error finding container 05bc1dee9fa3f43552ad544d9b0a01f035d84d33ad35bfdede3dc8afc664e763: Status 404 returned error can't find the container with id 05bc1dee9fa3f43552ad544d9b0a01f035d84d33ad35bfdede3dc8afc664e763
Sep 04 21:58:43.611524 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.611497    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"]
Sep 04 21:58:43.613283 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:58:43.613254    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd94c6c55_a84c_4c9c_8f75_f772730fa0dd.slice/crio-751f1bf62af5aee10a7b9b15f4068ef7740da76e80fed3174348f5c51f7bb5e5 WatchSource:0}: Error finding container 751f1bf62af5aee10a7b9b15f4068ef7740da76e80fed3174348f5c51f7bb5e5: Status 404 returned error can't find the container with id 751f1bf62af5aee10a7b9b15f4068ef7740da76e80fed3174348f5c51f7bb5e5
Sep 04 21:58:43.657820 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.657774    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4" event={"ID":"d94c6c55-a84c-4c9c-8f75-f772730fa0dd","Type":"ContainerStarted","Data":"751f1bf62af5aee10a7b9b15f4068ef7740da76e80fed3174348f5c51f7bb5e5"}
Sep 04 21:58:43.659008 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:43.658979    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr" event={"ID":"a1f7a723-2ac2-494d-99c1-8343f76130bb","Type":"ContainerStarted","Data":"05bc1dee9fa3f43552ad544d9b0a01f035d84d33ad35bfdede3dc8afc664e763"}
Sep 04 21:58:47.693786 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:47.692829    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 21:58:47.693786 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:47.692894    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 21:58:47.699091 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:47.699056    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 21:58:47.699198 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:47.699134    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 21:58:47.699198 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:47.699177    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 21:58:47.723431 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:47.723393    2557 kubelet.go:1648] "Image garbage collection succeeded"
Sep 04 21:58:48.693875 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:48.693839    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4" event={"ID":"d94c6c55-a84c-4c9c-8f75-f772730fa0dd","Type":"ContainerStarted","Data":"2e9d9ce0f06d7534e3c4a865dbe64ad9437da9fbbca90d110362b283bb12479a"}
Sep 04 21:58:48.694347 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:48.694035    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:48.695787 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:48.695754    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr" event={"ID":"a1f7a723-2ac2-494d-99c1-8343f76130bb","Type":"ContainerStarted","Data":"87595e911df40227c383a56bfdaa63fe560b204f383cb46acb05afd239df0649"}
Sep 04 21:58:48.696091 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:48.696067    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4"
Sep 04 21:58:48.738438 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:48.738354    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-rmlv4" podStartSLOduration=2.661157398 podStartE2EDuration="6.738325601s" totalImagesPullingTime="4.077168203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:58:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:58:48.737319144 +0000 UTC m=+303.053885883" watchObservedRunningTime="2026-09-04 21:58:48.738325601 +0000 UTC m=+303.054892340"
Sep 04 21:58:48.784807 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:48.784742    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr" podStartSLOduration=2.627473 podStartE2EDuration="6.78472056s" totalImagesPullingTime="4.15724756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 21:58:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 21:58:48.784403139 +0000 UTC m=+303.100969889" watchObservedRunningTime="2026-09-04 21:58:48.78472056 +0000 UTC m=+303.101287300"
Sep 04 21:58:49.239789 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:49.239748    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:49.281405 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:49.281366    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:49.701133 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:49.701104    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:58:49.702015 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:58:49.701998    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr"
Sep 04 21:59:38.797333 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:38.797247    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9"]
Sep 04 21:59:38.804937 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:38.804911    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9"
Sep 04 21:59:38.808165 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:38.808135    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 04 21:59:38.808914 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:38.808891    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jkcjf\""
Sep 04 21:59:38.809051 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:38.808893    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 04 21:59:38.809125 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:38.809101    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9"]
Sep 04 21:59:38.867051 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:38.867007    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e85da33-8bf0-4030-857d-a90bd697b0d6-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9\" (UID: \"4e85da33-8bf0-4030-857d-a90bd697b0d6\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9"
Sep 04 21:59:38.867229 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:38.867065    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r42tm\" (UniqueName: \"kubernetes.io/projected/4e85da33-8bf0-4030-857d-a90bd697b0d6-kube-api-access-r42tm\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9\" (UID: \"4e85da33-8bf0-4030-857d-a90bd697b0d6\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9"
Sep 04 21:59:38.867229 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:38.867153    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e85da33-8bf0-4030-857d-a90bd697b0d6-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9\" (UID: \"4e85da33-8bf0-4030-857d-a90bd697b0d6\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9"
Sep 04 21:59:38.902666 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:38.902626    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6"]
Sep 04 21:59:38.905987 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:38.905967    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6"
Sep 04 21:59:38.915510 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:38.915202    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6"]
Sep 04 21:59:38.968487 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:38.968449    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r42tm\" (UniqueName: \"kubernetes.io/projected/4e85da33-8bf0-4030-857d-a90bd697b0d6-kube-api-access-r42tm\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9\" (UID: \"4e85da33-8bf0-4030-857d-a90bd697b0d6\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9"
Sep 04 21:59:38.968670 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:38.968501    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e85da33-8bf0-4030-857d-a90bd697b0d6-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9\" (UID: \"4e85da33-8bf0-4030-857d-a90bd697b0d6\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9"
Sep 04 21:59:38.968670 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:38.968534    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prdrl\" (UniqueName: \"kubernetes.io/projected/0a4bee9a-5338-4814-99b5-000d72a18496-kube-api-access-prdrl\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6\" (UID: \"0a4bee9a-5338-4814-99b5-000d72a18496\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6"
Sep 04 21:59:38.968670 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:38.968566    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a4bee9a-5338-4814-99b5-000d72a18496-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6\" (UID: \"0a4bee9a-5338-4814-99b5-000d72a18496\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6"
Sep 04 21:59:38.968670 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:38.968606    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e85da33-8bf0-4030-857d-a90bd697b0d6-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9\" (UID: \"4e85da33-8bf0-4030-857d-a90bd697b0d6\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9"
Sep 04 21:59:38.968670 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:38.968628    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a4bee9a-5338-4814-99b5-000d72a18496-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6\" (UID: \"0a4bee9a-5338-4814-99b5-000d72a18496\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6"
Sep 04 21:59:38.968892 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:38.968873    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e85da33-8bf0-4030-857d-a90bd697b0d6-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9\" (UID: \"4e85da33-8bf0-4030-857d-a90bd697b0d6\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9"
Sep 04 21:59:38.968927 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:38.968903    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e85da33-8bf0-4030-857d-a90bd697b0d6-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9\" (UID: \"4e85da33-8bf0-4030-857d-a90bd697b0d6\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9"
Sep 04 21:59:38.976557 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:38.976526    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r42tm\" (UniqueName: \"kubernetes.io/projected/4e85da33-8bf0-4030-857d-a90bd697b0d6-kube-api-access-r42tm\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9\" (UID: \"4e85da33-8bf0-4030-857d-a90bd697b0d6\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9"
Sep 04 21:59:39.000236 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.000199    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz"]
Sep 04 21:59:39.003808 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.003789    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz"
Sep 04 21:59:39.012786 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.012758    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz"]
Sep 04 21:59:39.069171 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.069077    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a4bee9a-5338-4814-99b5-000d72a18496-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6\" (UID: \"0a4bee9a-5338-4814-99b5-000d72a18496\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6"
Sep 04 21:59:39.069171 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.069131    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec819057-9dcc-4e4d-8ca6-9292c3577bd1-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz\" (UID: \"ec819057-9dcc-4e4d-8ca6-9292c3577bd1\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz"
Sep 04 21:59:39.069171 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.069170    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a4bee9a-5338-4814-99b5-000d72a18496-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6\" (UID: \"0a4bee9a-5338-4814-99b5-000d72a18496\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6"
Sep 04 21:59:39.069405 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.069195    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec819057-9dcc-4e4d-8ca6-9292c3577bd1-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz\" (UID: \"ec819057-9dcc-4e4d-8ca6-9292c3577bd1\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz"
Sep 04 21:59:39.069405 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.069240    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxhln\" (UniqueName: \"kubernetes.io/projected/ec819057-9dcc-4e4d-8ca6-9292c3577bd1-kube-api-access-gxhln\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz\" (UID: \"ec819057-9dcc-4e4d-8ca6-9292c3577bd1\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz"
Sep 04 21:59:39.069405 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.069320    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prdrl\" (UniqueName: \"kubernetes.io/projected/0a4bee9a-5338-4814-99b5-000d72a18496-kube-api-access-prdrl\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6\" (UID: \"0a4bee9a-5338-4814-99b5-000d72a18496\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6"
Sep 04 21:59:39.069571 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.069550    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a4bee9a-5338-4814-99b5-000d72a18496-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6\" (UID: \"0a4bee9a-5338-4814-99b5-000d72a18496\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6"
Sep 04 21:59:39.069610 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.069571    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a4bee9a-5338-4814-99b5-000d72a18496-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6\" (UID: \"0a4bee9a-5338-4814-99b5-000d72a18496\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6"
Sep 04 21:59:39.077632 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.077607    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prdrl\" (UniqueName: \"kubernetes.io/projected/0a4bee9a-5338-4814-99b5-000d72a18496-kube-api-access-prdrl\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6\" (UID: \"0a4bee9a-5338-4814-99b5-000d72a18496\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6"
Sep 04 21:59:39.104075 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.104037    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql"]
Sep 04 21:59:39.107615 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.107595    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql"
Sep 04 21:59:39.116029 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.115999    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9"
Sep 04 21:59:39.118663 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.118637    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql"]
Sep 04 21:59:39.170198 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.169841    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49931db0-4470-4081-ad81-873349052a24-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql\" (UID: \"49931db0-4470-4081-ad81-873349052a24\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql"
Sep 04 21:59:39.170198 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.169931    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec819057-9dcc-4e4d-8ca6-9292c3577bd1-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz\" (UID: \"ec819057-9dcc-4e4d-8ca6-9292c3577bd1\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz"
Sep 04 21:59:39.170198 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.169957    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49931db0-4470-4081-ad81-873349052a24-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql\" (UID: \"49931db0-4470-4081-ad81-873349052a24\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql"
Sep 04 21:59:39.170198 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.169995    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec819057-9dcc-4e4d-8ca6-9292c3577bd1-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz\" (UID: \"ec819057-9dcc-4e4d-8ca6-9292c3577bd1\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz"
Sep 04 21:59:39.170198 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.170040    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxhln\" (UniqueName: \"kubernetes.io/projected/ec819057-9dcc-4e4d-8ca6-9292c3577bd1-kube-api-access-gxhln\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz\" (UID: \"ec819057-9dcc-4e4d-8ca6-9292c3577bd1\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz"
Sep 04 21:59:39.170198 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.170085    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ph7bl\" (UniqueName: \"kubernetes.io/projected/49931db0-4470-4081-ad81-873349052a24-kube-api-access-ph7bl\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql\" (UID: \"49931db0-4470-4081-ad81-873349052a24\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql"
Sep 04 21:59:39.170621 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.170521    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec819057-9dcc-4e4d-8ca6-9292c3577bd1-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz\" (UID: \"ec819057-9dcc-4e4d-8ca6-9292c3577bd1\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz"
Sep 04 21:59:39.171097 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.170782    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec819057-9dcc-4e4d-8ca6-9292c3577bd1-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz\" (UID: \"ec819057-9dcc-4e4d-8ca6-9292c3577bd1\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz"
Sep 04 21:59:39.180570 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.180537    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxhln\" (UniqueName: \"kubernetes.io/projected/ec819057-9dcc-4e4d-8ca6-9292c3577bd1-kube-api-access-gxhln\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz\" (UID: \"ec819057-9dcc-4e4d-8ca6-9292c3577bd1\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz"
Sep 04 21:59:39.218366 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.218326    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6"
Sep 04 21:59:39.271725 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.270886    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49931db0-4470-4081-ad81-873349052a24-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql\" (UID: \"49931db0-4470-4081-ad81-873349052a24\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql"
Sep 04 21:59:39.271725 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.270993    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ph7bl\" (UniqueName: \"kubernetes.io/projected/49931db0-4470-4081-ad81-873349052a24-kube-api-access-ph7bl\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql\" (UID: \"49931db0-4470-4081-ad81-873349052a24\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql"
Sep 04 21:59:39.271725 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.271042    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49931db0-4470-4081-ad81-873349052a24-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql\" (UID: \"49931db0-4470-4081-ad81-873349052a24\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql"
Sep 04 21:59:39.271725 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.271445    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49931db0-4470-4081-ad81-873349052a24-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql\" (UID: \"49931db0-4470-4081-ad81-873349052a24\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql"
Sep 04 21:59:39.271725 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.271498    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49931db0-4470-4081-ad81-873349052a24-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql\" (UID: \"49931db0-4470-4081-ad81-873349052a24\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql"
Sep 04 21:59:39.280102 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.280071    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ph7bl\" (UniqueName: \"kubernetes.io/projected/49931db0-4470-4081-ad81-873349052a24-kube-api-access-ph7bl\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql\" (UID: \"49931db0-4470-4081-ad81-873349052a24\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql"
Sep 04 21:59:39.313936 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.313894    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz"
Sep 04 21:59:39.393481 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.393452    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9"]
Sep 04 21:59:39.395620 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:59:39.395590    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e85da33_8bf0_4030_857d_a90bd697b0d6.slice/crio-0bf941d9051951bde86ccf49fcd4497e7ecdd808558546de1f158b71d02355a3 WatchSource:0}: Error finding container 0bf941d9051951bde86ccf49fcd4497e7ecdd808558546de1f158b71d02355a3: Status 404 returned error can't find the container with id 0bf941d9051951bde86ccf49fcd4497e7ecdd808558546de1f158b71d02355a3
Sep 04 21:59:39.417433 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.417382    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql"
Sep 04 21:59:39.494935 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.494906    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6"]
Sep 04 21:59:39.497055 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:59:39.497025    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a4bee9a_5338_4814_99b5_000d72a18496.slice/crio-35469739e306302fde6e9b4004fc4d7f27e67a435d220554696ddc7f53523768 WatchSource:0}: Error finding container 35469739e306302fde6e9b4004fc4d7f27e67a435d220554696ddc7f53523768: Status 404 returned error can't find the container with id 35469739e306302fde6e9b4004fc4d7f27e67a435d220554696ddc7f53523768
Sep 04 21:59:39.589026 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.589001    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz"]
Sep 04 21:59:39.590823 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:59:39.590798    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec819057_9dcc_4e4d_8ca6_9292c3577bd1.slice/crio-a672745bfe4c7801256eb2163289d7e960e5166a78dcf836f2515e29aedce37d WatchSource:0}: Error finding container a672745bfe4c7801256eb2163289d7e960e5166a78dcf836f2515e29aedce37d: Status 404 returned error can't find the container with id a672745bfe4c7801256eb2163289d7e960e5166a78dcf836f2515e29aedce37d
Sep 04 21:59:39.691328 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.691296    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql"]
Sep 04 21:59:39.741482 ip-10-0-134-141 kubenswrapper[2557]: W0904 21:59:39.741445    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49931db0_4470_4081_ad81_873349052a24.slice/crio-c4edd8a59f8b555544b1a2cc6cf8c696f95b05166d7ea16dc9845db973bbe1aa WatchSource:0}: Error finding container c4edd8a59f8b555544b1a2cc6cf8c696f95b05166d7ea16dc9845db973bbe1aa: Status 404 returned error can't find the container with id c4edd8a59f8b555544b1a2cc6cf8c696f95b05166d7ea16dc9845db973bbe1aa
Sep 04 21:59:39.936885 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.936844    2557 generic.go:356] "Generic (PLEG): container finished" podID="49931db0-4470-4081-ad81-873349052a24" containerID="1d9c657a3098a4710f0bce62b0261308dcab06afd94c69eed48a28b74e8a6c05" exitCode=0
Sep 04 21:59:39.937323 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.936931    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql" event={"ID":"49931db0-4470-4081-ad81-873349052a24","Type":"ContainerDied","Data":"1d9c657a3098a4710f0bce62b0261308dcab06afd94c69eed48a28b74e8a6c05"}
Sep 04 21:59:39.937323 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.936990    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql" event={"ID":"49931db0-4470-4081-ad81-873349052a24","Type":"ContainerStarted","Data":"c4edd8a59f8b555544b1a2cc6cf8c696f95b05166d7ea16dc9845db973bbe1aa"}
Sep 04 21:59:39.938040 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.938019    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 04 21:59:39.938737 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.938704    2557 generic.go:356] "Generic (PLEG): container finished" podID="4e85da33-8bf0-4030-857d-a90bd697b0d6" containerID="e54e072997874d8e0879ef1b3cbbc044c9dd4bf417d27e0c7f5906a3e677fddb" exitCode=0
Sep 04 21:59:39.938806 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.938759    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9" event={"ID":"4e85da33-8bf0-4030-857d-a90bd697b0d6","Type":"ContainerDied","Data":"e54e072997874d8e0879ef1b3cbbc044c9dd4bf417d27e0c7f5906a3e677fddb"}
Sep 04 21:59:39.938867 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.938804    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9" event={"ID":"4e85da33-8bf0-4030-857d-a90bd697b0d6","Type":"ContainerStarted","Data":"0bf941d9051951bde86ccf49fcd4497e7ecdd808558546de1f158b71d02355a3"}
Sep 04 21:59:39.940441 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.940399    2557 generic.go:356] "Generic (PLEG): container finished" podID="0a4bee9a-5338-4814-99b5-000d72a18496" containerID="b976c97a06a8c53c05ca01bc9c182aa4b320e67be13ad5aba87c67d1e4e8a227" exitCode=0
Sep 04 21:59:39.940529 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.940455    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6" event={"ID":"0a4bee9a-5338-4814-99b5-000d72a18496","Type":"ContainerDied","Data":"b976c97a06a8c53c05ca01bc9c182aa4b320e67be13ad5aba87c67d1e4e8a227"}
Sep 04 21:59:39.940529 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.940497    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6" event={"ID":"0a4bee9a-5338-4814-99b5-000d72a18496","Type":"ContainerStarted","Data":"35469739e306302fde6e9b4004fc4d7f27e67a435d220554696ddc7f53523768"}
Sep 04 21:59:39.942189 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.942168    2557 generic.go:356] "Generic (PLEG): container finished" podID="ec819057-9dcc-4e4d-8ca6-9292c3577bd1" containerID="965dfba3de91de576ac1399aaa17e628afe50e530a30a1c95b0acab34a56563c" exitCode=0
Sep 04 21:59:39.942271 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.942207    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz" event={"ID":"ec819057-9dcc-4e4d-8ca6-9292c3577bd1","Type":"ContainerDied","Data":"965dfba3de91de576ac1399aaa17e628afe50e530a30a1c95b0acab34a56563c"}
Sep 04 21:59:39.942271 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:39.942234    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz" event={"ID":"ec819057-9dcc-4e4d-8ca6-9292c3577bd1","Type":"ContainerStarted","Data":"a672745bfe4c7801256eb2163289d7e960e5166a78dcf836f2515e29aedce37d"}
Sep 04 21:59:40.949824 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:40.949782    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql" event={"ID":"49931db0-4470-4081-ad81-873349052a24","Type":"ContainerStarted","Data":"97881b8fe78eb0c3dc41ac66e6ceb9afd215901d88aedc658cb44aa7e5be97a5"}
Sep 04 21:59:40.951996 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:40.951962    2557 generic.go:356] "Generic (PLEG): container finished" podID="4e85da33-8bf0-4030-857d-a90bd697b0d6" containerID="8d03ce6f4bf069ed02d54c47fefdf2d0020f39d91a3ef9993a8c259dbce49f62" exitCode=0
Sep 04 21:59:40.952134 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:40.952022    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9" event={"ID":"4e85da33-8bf0-4030-857d-a90bd697b0d6","Type":"ContainerDied","Data":"8d03ce6f4bf069ed02d54c47fefdf2d0020f39d91a3ef9993a8c259dbce49f62"}
Sep 04 21:59:40.954265 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:40.954236    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz" event={"ID":"ec819057-9dcc-4e4d-8ca6-9292c3577bd1","Type":"ContainerStarted","Data":"6106ecd06ae0a67fb9e55fc38aa7c6b5640ae39281b5dcc564c71e3bd9131d5c"}
Sep 04 21:59:41.961346 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:41.961308    2557 generic.go:356] "Generic (PLEG): container finished" podID="49931db0-4470-4081-ad81-873349052a24" containerID="97881b8fe78eb0c3dc41ac66e6ceb9afd215901d88aedc658cb44aa7e5be97a5" exitCode=0
Sep 04 21:59:41.961857 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:41.961395    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql" event={"ID":"49931db0-4470-4081-ad81-873349052a24","Type":"ContainerDied","Data":"97881b8fe78eb0c3dc41ac66e6ceb9afd215901d88aedc658cb44aa7e5be97a5"}
Sep 04 21:59:41.963632 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:41.963608    2557 generic.go:356] "Generic (PLEG): container finished" podID="4e85da33-8bf0-4030-857d-a90bd697b0d6" containerID="f986ebdb13bdbc3db31df2d7d3d5ee76cfcd88c22ccbb9d31e130f6118ec6fe1" exitCode=0
Sep 04 21:59:41.963759 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:41.963643    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9" event={"ID":"4e85da33-8bf0-4030-857d-a90bd697b0d6","Type":"ContainerDied","Data":"f986ebdb13bdbc3db31df2d7d3d5ee76cfcd88c22ccbb9d31e130f6118ec6fe1"}
Sep 04 21:59:41.965551 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:41.965528    2557 generic.go:356] "Generic (PLEG): container finished" podID="ec819057-9dcc-4e4d-8ca6-9292c3577bd1" containerID="6106ecd06ae0a67fb9e55fc38aa7c6b5640ae39281b5dcc564c71e3bd9131d5c" exitCode=0
Sep 04 21:59:41.965651 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:41.965566    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz" event={"ID":"ec819057-9dcc-4e4d-8ca6-9292c3577bd1","Type":"ContainerDied","Data":"6106ecd06ae0a67fb9e55fc38aa7c6b5640ae39281b5dcc564c71e3bd9131d5c"}
Sep 04 21:59:42.972914 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:42.972883    2557 generic.go:356] "Generic (PLEG): container finished" podID="ec819057-9dcc-4e4d-8ca6-9292c3577bd1" containerID="c44218ac6c6476e909a2f05fd39d9584d046a856f48db62d4edd9fd9e9801293" exitCode=0
Sep 04 21:59:42.973334 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:42.973002    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz" event={"ID":"ec819057-9dcc-4e4d-8ca6-9292c3577bd1","Type":"ContainerDied","Data":"c44218ac6c6476e909a2f05fd39d9584d046a856f48db62d4edd9fd9e9801293"}
Sep 04 21:59:42.974977 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:42.974951    2557 generic.go:356] "Generic (PLEG): container finished" podID="49931db0-4470-4081-ad81-873349052a24" containerID="6d0228dbb2f721fd9e9247eab9f6a40bbd5cc38843a95022f22b3e3e5d7d7a24" exitCode=0
Sep 04 21:59:42.975107 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:42.974986    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql" event={"ID":"49931db0-4470-4081-ad81-873349052a24","Type":"ContainerDied","Data":"6d0228dbb2f721fd9e9247eab9f6a40bbd5cc38843a95022f22b3e3e5d7d7a24"}
Sep 04 21:59:43.103897 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:43.103868    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9"
Sep 04 21:59:43.208831 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:43.208787    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e85da33-8bf0-4030-857d-a90bd697b0d6-util\") pod \"4e85da33-8bf0-4030-857d-a90bd697b0d6\" (UID: \"4e85da33-8bf0-4030-857d-a90bd697b0d6\") "
Sep 04 21:59:43.209029 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:43.208904    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e85da33-8bf0-4030-857d-a90bd697b0d6-bundle\") pod \"4e85da33-8bf0-4030-857d-a90bd697b0d6\" (UID: \"4e85da33-8bf0-4030-857d-a90bd697b0d6\") "
Sep 04 21:59:43.209029 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:43.208938    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r42tm\" (UniqueName: \"kubernetes.io/projected/4e85da33-8bf0-4030-857d-a90bd697b0d6-kube-api-access-r42tm\") pod \"4e85da33-8bf0-4030-857d-a90bd697b0d6\" (UID: \"4e85da33-8bf0-4030-857d-a90bd697b0d6\") "
Sep 04 21:59:43.209408 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:43.209381    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e85da33-8bf0-4030-857d-a90bd697b0d6-bundle" (OuterVolumeSpecName: "bundle") pod "4e85da33-8bf0-4030-857d-a90bd697b0d6" (UID: "4e85da33-8bf0-4030-857d-a90bd697b0d6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 21:59:43.211114 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:43.211078    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e85da33-8bf0-4030-857d-a90bd697b0d6-kube-api-access-r42tm" (OuterVolumeSpecName: "kube-api-access-r42tm") pod "4e85da33-8bf0-4030-857d-a90bd697b0d6" (UID: "4e85da33-8bf0-4030-857d-a90bd697b0d6"). InnerVolumeSpecName "kube-api-access-r42tm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 21:59:43.215505 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:43.215462    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e85da33-8bf0-4030-857d-a90bd697b0d6-util" (OuterVolumeSpecName: "util") pod "4e85da33-8bf0-4030-857d-a90bd697b0d6" (UID: "4e85da33-8bf0-4030-857d-a90bd697b0d6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 21:59:43.310575 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:43.310534    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4e85da33-8bf0-4030-857d-a90bd697b0d6-bundle\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:59:43.310575 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:43.310569    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r42tm\" (UniqueName: \"kubernetes.io/projected/4e85da33-8bf0-4030-857d-a90bd697b0d6-kube-api-access-r42tm\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:59:43.310575 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:43.310581    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4e85da33-8bf0-4030-857d-a90bd697b0d6-util\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:59:43.982210 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:43.982176    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9"
Sep 04 21:59:43.982654 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:43.982202    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a07gnl9" event={"ID":"4e85da33-8bf0-4030-857d-a90bd697b0d6","Type":"ContainerDied","Data":"0bf941d9051951bde86ccf49fcd4497e7ecdd808558546de1f158b71d02355a3"}
Sep 04 21:59:43.982654 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:43.982292    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0bf941d9051951bde86ccf49fcd4497e7ecdd808558546de1f158b71d02355a3"
Sep 04 21:59:44.118827 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.118798    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz"
Sep 04 21:59:44.156275 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.156239    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql"
Sep 04 21:59:44.218410 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.218370    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxhln\" (UniqueName: \"kubernetes.io/projected/ec819057-9dcc-4e4d-8ca6-9292c3577bd1-kube-api-access-gxhln\") pod \"ec819057-9dcc-4e4d-8ca6-9292c3577bd1\" (UID: \"ec819057-9dcc-4e4d-8ca6-9292c3577bd1\") "
Sep 04 21:59:44.218590 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.218450    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec819057-9dcc-4e4d-8ca6-9292c3577bd1-util\") pod \"ec819057-9dcc-4e4d-8ca6-9292c3577bd1\" (UID: \"ec819057-9dcc-4e4d-8ca6-9292c3577bd1\") "
Sep 04 21:59:44.218590 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.218482    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec819057-9dcc-4e4d-8ca6-9292c3577bd1-bundle\") pod \"ec819057-9dcc-4e4d-8ca6-9292c3577bd1\" (UID: \"ec819057-9dcc-4e4d-8ca6-9292c3577bd1\") "
Sep 04 21:59:44.219027 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.218991    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec819057-9dcc-4e4d-8ca6-9292c3577bd1-bundle" (OuterVolumeSpecName: "bundle") pod "ec819057-9dcc-4e4d-8ca6-9292c3577bd1" (UID: "ec819057-9dcc-4e4d-8ca6-9292c3577bd1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 21:59:44.220533 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.220507    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec819057-9dcc-4e4d-8ca6-9292c3577bd1-kube-api-access-gxhln" (OuterVolumeSpecName: "kube-api-access-gxhln") pod "ec819057-9dcc-4e4d-8ca6-9292c3577bd1" (UID: "ec819057-9dcc-4e4d-8ca6-9292c3577bd1"). InnerVolumeSpecName "kube-api-access-gxhln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 21:59:44.223901 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.223867    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec819057-9dcc-4e4d-8ca6-9292c3577bd1-util" (OuterVolumeSpecName: "util") pod "ec819057-9dcc-4e4d-8ca6-9292c3577bd1" (UID: "ec819057-9dcc-4e4d-8ca6-9292c3577bd1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 21:59:44.319205 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.319166    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ph7bl\" (UniqueName: \"kubernetes.io/projected/49931db0-4470-4081-ad81-873349052a24-kube-api-access-ph7bl\") pod \"49931db0-4470-4081-ad81-873349052a24\" (UID: \"49931db0-4470-4081-ad81-873349052a24\") "
Sep 04 21:59:44.319205 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.319222    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49931db0-4470-4081-ad81-873349052a24-bundle\") pod \"49931db0-4470-4081-ad81-873349052a24\" (UID: \"49931db0-4470-4081-ad81-873349052a24\") "
Sep 04 21:59:44.319504 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.319250    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49931db0-4470-4081-ad81-873349052a24-util\") pod \"49931db0-4470-4081-ad81-873349052a24\" (UID: \"49931db0-4470-4081-ad81-873349052a24\") "
Sep 04 21:59:44.319504 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.319475    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxhln\" (UniqueName: \"kubernetes.io/projected/ec819057-9dcc-4e4d-8ca6-9292c3577bd1-kube-api-access-gxhln\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:59:44.319504 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.319486    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ec819057-9dcc-4e4d-8ca6-9292c3577bd1-util\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:59:44.319504 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.319494    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ec819057-9dcc-4e4d-8ca6-9292c3577bd1-bundle\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:59:44.319846 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.319822    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49931db0-4470-4081-ad81-873349052a24-bundle" (OuterVolumeSpecName: "bundle") pod "49931db0-4470-4081-ad81-873349052a24" (UID: "49931db0-4470-4081-ad81-873349052a24"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 21:59:44.321408 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.321387    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49931db0-4470-4081-ad81-873349052a24-kube-api-access-ph7bl" (OuterVolumeSpecName: "kube-api-access-ph7bl") pod "49931db0-4470-4081-ad81-873349052a24" (UID: "49931db0-4470-4081-ad81-873349052a24"). InnerVolumeSpecName "kube-api-access-ph7bl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 21:59:44.324642 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.324612    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49931db0-4470-4081-ad81-873349052a24-util" (OuterVolumeSpecName: "util") pod "49931db0-4470-4081-ad81-873349052a24" (UID: "49931db0-4470-4081-ad81-873349052a24"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 21:59:44.420685 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.420644    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ph7bl\" (UniqueName: \"kubernetes.io/projected/49931db0-4470-4081-ad81-873349052a24-kube-api-access-ph7bl\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:59:44.420823 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.420702    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/49931db0-4470-4081-ad81-873349052a24-bundle\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:59:44.420823 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.420714    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/49931db0-4470-4081-ad81-873349052a24-util\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 21:59:44.988833 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.988801    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql"
Sep 04 21:59:44.988833 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.988817    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183c6rnql" event={"ID":"49931db0-4470-4081-ad81-873349052a24","Type":"ContainerDied","Data":"c4edd8a59f8b555544b1a2cc6cf8c696f95b05166d7ea16dc9845db973bbe1aa"}
Sep 04 21:59:44.989302 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.988867    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c4edd8a59f8b555544b1a2cc6cf8c696f95b05166d7ea16dc9845db973bbe1aa"
Sep 04 21:59:44.990853 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.990826    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz" event={"ID":"ec819057-9dcc-4e4d-8ca6-9292c3577bd1","Type":"ContainerDied","Data":"a672745bfe4c7801256eb2163289d7e960e5166a78dcf836f2515e29aedce37d"}
Sep 04 21:59:44.990996 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.990872    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a672745bfe4c7801256eb2163289d7e960e5166a78dcf836f2515e29aedce37d"
Sep 04 21:59:44.990996 ip-10-0-134-141 kubenswrapper[2557]: I0904 21:59:44.990875    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbv8zkz"
Sep 04 22:00:02.075766 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:02.075672    2557 generic.go:356] "Generic (PLEG): container finished" podID="0a4bee9a-5338-4814-99b5-000d72a18496" containerID="a71d9ab69d96e90d2119012849711892117e5c5e5c2a3fbc297aa1e83b039adb" exitCode=0
Sep 04 22:00:02.075766 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:02.075747    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6" event={"ID":"0a4bee9a-5338-4814-99b5-000d72a18496","Type":"ContainerDied","Data":"a71d9ab69d96e90d2119012849711892117e5c5e5c2a3fbc297aa1e83b039adb"}
Sep 04 22:00:03.082822 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:03.082785    2557 generic.go:356] "Generic (PLEG): container finished" podID="0a4bee9a-5338-4814-99b5-000d72a18496" containerID="8681111060d742a650ccf798caca2982af0112cb5d8f59fbba10a49472d42993" exitCode=0
Sep 04 22:00:03.083263 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:03.082827    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6" event={"ID":"0a4bee9a-5338-4814-99b5-000d72a18496","Type":"ContainerDied","Data":"8681111060d742a650ccf798caca2982af0112cb5d8f59fbba10a49472d42993"}
Sep 04 22:00:04.211033 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:04.211003    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6"
Sep 04 22:00:04.291983 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:04.291945    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a4bee9a-5338-4814-99b5-000d72a18496-bundle\") pod \"0a4bee9a-5338-4814-99b5-000d72a18496\" (UID: \"0a4bee9a-5338-4814-99b5-000d72a18496\") "
Sep 04 22:00:04.291983 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:04.291988    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a4bee9a-5338-4814-99b5-000d72a18496-util\") pod \"0a4bee9a-5338-4814-99b5-000d72a18496\" (UID: \"0a4bee9a-5338-4814-99b5-000d72a18496\") "
Sep 04 22:00:04.292247 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:04.292030    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prdrl\" (UniqueName: \"kubernetes.io/projected/0a4bee9a-5338-4814-99b5-000d72a18496-kube-api-access-prdrl\") pod \"0a4bee9a-5338-4814-99b5-000d72a18496\" (UID: \"0a4bee9a-5338-4814-99b5-000d72a18496\") "
Sep 04 22:00:04.292778 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:04.292752    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a4bee9a-5338-4814-99b5-000d72a18496-bundle" (OuterVolumeSpecName: "bundle") pod "0a4bee9a-5338-4814-99b5-000d72a18496" (UID: "0a4bee9a-5338-4814-99b5-000d72a18496"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:00:04.294219 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:04.294195    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a4bee9a-5338-4814-99b5-000d72a18496-kube-api-access-prdrl" (OuterVolumeSpecName: "kube-api-access-prdrl") pod "0a4bee9a-5338-4814-99b5-000d72a18496" (UID: "0a4bee9a-5338-4814-99b5-000d72a18496"). InnerVolumeSpecName "kube-api-access-prdrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:00:04.296995 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:04.296964    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a4bee9a-5338-4814-99b5-000d72a18496-util" (OuterVolumeSpecName: "util") pod "0a4bee9a-5338-4814-99b5-000d72a18496" (UID: "0a4bee9a-5338-4814-99b5-000d72a18496"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:00:04.392973 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:04.392934    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prdrl\" (UniqueName: \"kubernetes.io/projected/0a4bee9a-5338-4814-99b5-000d72a18496-kube-api-access-prdrl\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:00:04.392973 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:04.392965    2557 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a4bee9a-5338-4814-99b5-000d72a18496-bundle\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:00:04.392973 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:04.392977    2557 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a4bee9a-5338-4814-99b5-000d72a18496-util\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:00:05.094730 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:05.094690    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6" event={"ID":"0a4bee9a-5338-4814-99b5-000d72a18496","Type":"ContainerDied","Data":"35469739e306302fde6e9b4004fc4d7f27e67a435d220554696ddc7f53523768"}
Sep 04 22:00:05.094730 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:05.094713    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053lklp6"
Sep 04 22:00:05.094730 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:05.094738    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="35469739e306302fde6e9b4004fc4d7f27e67a435d220554696ddc7f53523768"
Sep 04 22:00:05.847927 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:05.847885    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-598c465776-gs2ts"]
Sep 04 22:00:30.867809 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:30.867769    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-598c465776-gs2ts" podUID="afcdfa91-13e3-4315-8eec-0554a5195a99" containerName="console" containerID="cri-o://6f151ef201e9bbbafafffa2f73fef84bc5e21dca2a512e3667cb4508861bb774" gracePeriod=15
Sep 04 22:00:31.109913 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.109884    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-598c465776-gs2ts"
Sep 04 22:00:31.223780 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.223684    2557 generic.go:356] "Generic (PLEG): container finished" podID="afcdfa91-13e3-4315-8eec-0554a5195a99" containerID="6f151ef201e9bbbafafffa2f73fef84bc5e21dca2a512e3667cb4508861bb774" exitCode=0
Sep 04 22:00:31.223780 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.223730    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-598c465776-gs2ts"
Sep 04 22:00:31.224009 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.223774    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-598c465776-gs2ts" event={"ID":"afcdfa91-13e3-4315-8eec-0554a5195a99","Type":"ContainerDied","Data":"6f151ef201e9bbbafafffa2f73fef84bc5e21dca2a512e3667cb4508861bb774"}
Sep 04 22:00:31.224009 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.223831    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-598c465776-gs2ts" event={"ID":"afcdfa91-13e3-4315-8eec-0554a5195a99","Type":"ContainerDied","Data":"7a56bb1ff681db3888a1ac6e1d3a5c5c485d4bfb3cb1e1b73ce1b0e603db084a"}
Sep 04 22:00:31.224009 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.223859    2557 scope.go:117] "RemoveContainer" containerID="6f151ef201e9bbbafafffa2f73fef84bc5e21dca2a512e3667cb4508861bb774"
Sep 04 22:00:31.232084 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.232050    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-console-config\") pod \"afcdfa91-13e3-4315-8eec-0554a5195a99\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") "
Sep 04 22:00:31.232230 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.232097    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afcdfa91-13e3-4315-8eec-0554a5195a99-console-serving-cert\") pod \"afcdfa91-13e3-4315-8eec-0554a5195a99\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") "
Sep 04 22:00:31.232230 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.232133    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afcdfa91-13e3-4315-8eec-0554a5195a99-console-oauth-config\") pod \"afcdfa91-13e3-4315-8eec-0554a5195a99\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") "
Sep 04 22:00:31.232230 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.232178    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52mzb\" (UniqueName: \"kubernetes.io/projected/afcdfa91-13e3-4315-8eec-0554a5195a99-kube-api-access-52mzb\") pod \"afcdfa91-13e3-4315-8eec-0554a5195a99\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") "
Sep 04 22:00:31.232397 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.232321    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-oauth-serving-cert\") pod \"afcdfa91-13e3-4315-8eec-0554a5195a99\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") "
Sep 04 22:00:31.232397 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.232377    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-trusted-ca-bundle\") pod \"afcdfa91-13e3-4315-8eec-0554a5195a99\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") "
Sep 04 22:00:31.232518 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.232430    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-service-ca\") pod \"afcdfa91-13e3-4315-8eec-0554a5195a99\" (UID: \"afcdfa91-13e3-4315-8eec-0554a5195a99\") "
Sep 04 22:00:31.232518 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.232480    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-console-config" (OuterVolumeSpecName: "console-config") pod "afcdfa91-13e3-4315-8eec-0554a5195a99" (UID: "afcdfa91-13e3-4315-8eec-0554a5195a99"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:00:31.232751 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.232717    2557 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-console-config\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:00:31.232953 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.232823    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "afcdfa91-13e3-4315-8eec-0554a5195a99" (UID: "afcdfa91-13e3-4315-8eec-0554a5195a99"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:00:31.233040 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.232958    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "afcdfa91-13e3-4315-8eec-0554a5195a99" (UID: "afcdfa91-13e3-4315-8eec-0554a5195a99"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:00:31.233040 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.233003    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-service-ca" (OuterVolumeSpecName: "service-ca") pod "afcdfa91-13e3-4315-8eec-0554a5195a99" (UID: "afcdfa91-13e3-4315-8eec-0554a5195a99"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:00:31.233149 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.233125    2557 scope.go:117] "RemoveContainer" containerID="6f151ef201e9bbbafafffa2f73fef84bc5e21dca2a512e3667cb4508861bb774"
Sep 04 22:00:31.233533 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:00:31.233509    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f151ef201e9bbbafafffa2f73fef84bc5e21dca2a512e3667cb4508861bb774\": container with ID starting with 6f151ef201e9bbbafafffa2f73fef84bc5e21dca2a512e3667cb4508861bb774 not found: ID does not exist" containerID="6f151ef201e9bbbafafffa2f73fef84bc5e21dca2a512e3667cb4508861bb774"
Sep 04 22:00:31.233632 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.233543    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f151ef201e9bbbafafffa2f73fef84bc5e21dca2a512e3667cb4508861bb774"} err="failed to get container status \"6f151ef201e9bbbafafffa2f73fef84bc5e21dca2a512e3667cb4508861bb774\": rpc error: code = NotFound desc = could not find container \"6f151ef201e9bbbafafffa2f73fef84bc5e21dca2a512e3667cb4508861bb774\": container with ID starting with 6f151ef201e9bbbafafffa2f73fef84bc5e21dca2a512e3667cb4508861bb774 not found: ID does not exist"
Sep 04 22:00:31.234460 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.234438    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afcdfa91-13e3-4315-8eec-0554a5195a99-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "afcdfa91-13e3-4315-8eec-0554a5195a99" (UID: "afcdfa91-13e3-4315-8eec-0554a5195a99"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:00:31.234811 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.234786    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afcdfa91-13e3-4315-8eec-0554a5195a99-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "afcdfa91-13e3-4315-8eec-0554a5195a99" (UID: "afcdfa91-13e3-4315-8eec-0554a5195a99"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:00:31.234904 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.234882    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afcdfa91-13e3-4315-8eec-0554a5195a99-kube-api-access-52mzb" (OuterVolumeSpecName: "kube-api-access-52mzb") pod "afcdfa91-13e3-4315-8eec-0554a5195a99" (UID: "afcdfa91-13e3-4315-8eec-0554a5195a99"). InnerVolumeSpecName "kube-api-access-52mzb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:00:31.333702 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.333657    2557 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/afcdfa91-13e3-4315-8eec-0554a5195a99-console-oauth-config\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:00:31.333702 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.333695    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52mzb\" (UniqueName: \"kubernetes.io/projected/afcdfa91-13e3-4315-8eec-0554a5195a99-kube-api-access-52mzb\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:00:31.333702 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.333704    2557 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-oauth-serving-cert\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:00:31.333702 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.333713    2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-trusted-ca-bundle\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:00:31.333973 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.333723    2557 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/afcdfa91-13e3-4315-8eec-0554a5195a99-service-ca\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:00:31.333973 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.333733    2557 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/afcdfa91-13e3-4315-8eec-0554a5195a99-console-serving-cert\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:00:31.545603 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.545565    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-598c465776-gs2ts"]
Sep 04 22:00:31.548107 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:31.548073    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-598c465776-gs2ts"]
Sep 04 22:00:32.352429 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:00:32.352380    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afcdfa91-13e3-4315-8eec-0554a5195a99" path="/var/lib/kubelet/pods/afcdfa91-13e3-4315-8eec-0554a5195a99/volumes"
Sep 04 22:01:04.420942 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.420905    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"]
Sep 04 22:01:04.421402 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421265    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec819057-9dcc-4e4d-8ca6-9292c3577bd1" containerName="extract"
Sep 04 22:01:04.421402 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421285    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec819057-9dcc-4e4d-8ca6-9292c3577bd1" containerName="extract"
Sep 04 22:01:04.421402 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421307    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a4bee9a-5338-4814-99b5-000d72a18496" containerName="pull"
Sep 04 22:01:04.421402 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421315    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a4bee9a-5338-4814-99b5-000d72a18496" containerName="pull"
Sep 04 22:01:04.421402 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421328    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49931db0-4470-4081-ad81-873349052a24" containerName="extract"
Sep 04 22:01:04.421402 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421337    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="49931db0-4470-4081-ad81-873349052a24" containerName="extract"
Sep 04 22:01:04.421402 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421350    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afcdfa91-13e3-4315-8eec-0554a5195a99" containerName="console"
Sep 04 22:01:04.421402 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421359    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="afcdfa91-13e3-4315-8eec-0554a5195a99" containerName="console"
Sep 04 22:01:04.421402 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421367    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e85da33-8bf0-4030-857d-a90bd697b0d6" containerName="util"
Sep 04 22:01:04.421402 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421377    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e85da33-8bf0-4030-857d-a90bd697b0d6" containerName="util"
Sep 04 22:01:04.421402 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421385    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a4bee9a-5338-4814-99b5-000d72a18496" containerName="extract"
Sep 04 22:01:04.421402 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421393    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a4bee9a-5338-4814-99b5-000d72a18496" containerName="extract"
Sep 04 22:01:04.421402 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421407    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec819057-9dcc-4e4d-8ca6-9292c3577bd1" containerName="pull"
Sep 04 22:01:04.421870 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421429    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec819057-9dcc-4e4d-8ca6-9292c3577bd1" containerName="pull"
Sep 04 22:01:04.421870 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421441    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec819057-9dcc-4e4d-8ca6-9292c3577bd1" containerName="util"
Sep 04 22:01:04.421870 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421448    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec819057-9dcc-4e4d-8ca6-9292c3577bd1" containerName="util"
Sep 04 22:01:04.421870 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421455    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49931db0-4470-4081-ad81-873349052a24" containerName="util"
Sep 04 22:01:04.421870 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421463    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="49931db0-4470-4081-ad81-873349052a24" containerName="util"
Sep 04 22:01:04.421870 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421477    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e85da33-8bf0-4030-857d-a90bd697b0d6" containerName="extract"
Sep 04 22:01:04.421870 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421485    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e85da33-8bf0-4030-857d-a90bd697b0d6" containerName="extract"
Sep 04 22:01:04.421870 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421497    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49931db0-4470-4081-ad81-873349052a24" containerName="pull"
Sep 04 22:01:04.421870 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421506    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="49931db0-4470-4081-ad81-873349052a24" containerName="pull"
Sep 04 22:01:04.421870 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421518    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e85da33-8bf0-4030-857d-a90bd697b0d6" containerName="pull"
Sep 04 22:01:04.421870 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421525    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e85da33-8bf0-4030-857d-a90bd697b0d6" containerName="pull"
Sep 04 22:01:04.421870 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421539    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a4bee9a-5338-4814-99b5-000d72a18496" containerName="util"
Sep 04 22:01:04.421870 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421547    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a4bee9a-5338-4814-99b5-000d72a18496" containerName="util"
Sep 04 22:01:04.421870 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421628    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="afcdfa91-13e3-4315-8eec-0554a5195a99" containerName="console"
Sep 04 22:01:04.421870 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421639    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="49931db0-4470-4081-ad81-873349052a24" containerName="extract"
Sep 04 22:01:04.421870 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421645    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a4bee9a-5338-4814-99b5-000d72a18496" containerName="extract"
Sep 04 22:01:04.421870 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421652    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec819057-9dcc-4e4d-8ca6-9292c3577bd1" containerName="extract"
Sep 04 22:01:04.421870 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.421658    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e85da33-8bf0-4030-857d-a90bd697b0d6" containerName="extract"
Sep 04 22:01:04.425836 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.425813    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.428087 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.428061    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-kl7rz\""
Sep 04 22:01:04.436792 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.436763    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"]
Sep 04 22:01:04.504876 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.504832    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/921df383-5403-4688-a3f6-ffe059353a5a-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.505082 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.504890    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/921df383-5403-4688-a3f6-ffe059353a5a-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.505082 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.504912    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gk67\" (UniqueName: \"kubernetes.io/projected/921df383-5403-4688-a3f6-ffe059353a5a-kube-api-access-5gk67\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.505082 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.504982    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/921df383-5403-4688-a3f6-ffe059353a5a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.505082 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.505027    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/921df383-5403-4688-a3f6-ffe059353a5a-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.505082 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.505053    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/921df383-5403-4688-a3f6-ffe059353a5a-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.505296 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.505106    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/921df383-5403-4688-a3f6-ffe059353a5a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.505296 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.505172    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/921df383-5403-4688-a3f6-ffe059353a5a-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.505296 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.505250    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/921df383-5403-4688-a3f6-ffe059353a5a-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.606371 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.606328    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/921df383-5403-4688-a3f6-ffe059353a5a-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.606371 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.606372    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/921df383-5403-4688-a3f6-ffe059353a5a-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.606649 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.606399    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/921df383-5403-4688-a3f6-ffe059353a5a-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.606649 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.606479    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/921df383-5403-4688-a3f6-ffe059353a5a-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.606649 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.606508    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gk67\" (UniqueName: \"kubernetes.io/projected/921df383-5403-4688-a3f6-ffe059353a5a-kube-api-access-5gk67\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.606649 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.606544    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/921df383-5403-4688-a3f6-ffe059353a5a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.606649 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.606572    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/921df383-5403-4688-a3f6-ffe059353a5a-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.606649 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.606593    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/921df383-5403-4688-a3f6-ffe059353a5a-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.606649 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.606629    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/921df383-5403-4688-a3f6-ffe059353a5a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.606991 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.606924    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/921df383-5403-4688-a3f6-ffe059353a5a-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.607050 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.606995    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/921df383-5403-4688-a3f6-ffe059353a5a-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.607088 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.607053    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/921df383-5403-4688-a3f6-ffe059353a5a-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.607155 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.607135    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/921df383-5403-4688-a3f6-ffe059353a5a-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.607279 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.607259    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/921df383-5403-4688-a3f6-ffe059353a5a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.609267 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.609236    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/921df383-5403-4688-a3f6-ffe059353a5a-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.609599 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.609578    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/921df383-5403-4688-a3f6-ffe059353a5a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.614458 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.614430    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/921df383-5403-4688-a3f6-ffe059353a5a-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.614586 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.614570    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gk67\" (UniqueName: \"kubernetes.io/projected/921df383-5403-4688-a3f6-ffe059353a5a-kube-api-access-5gk67\") pod \"maas-default-gateway-openshift-default-78bb78ff78-8tbt9\" (UID: \"921df383-5403-4688-a3f6-ffe059353a5a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:04.736303 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:04.736217    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:05.014694 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:05.014666    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"]
Sep 04 22:01:05.017535 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:01:05.017507    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod921df383_5403_4688_a3f6_ffe059353a5a.slice/crio-32ec8707a1ea74f88947b8590bf53b61c2cb49faec88feaa554ace8af1a2941d WatchSource:0}: Error finding container 32ec8707a1ea74f88947b8590bf53b61c2cb49faec88feaa554ace8af1a2941d: Status 404 returned error can't find the container with id 32ec8707a1ea74f88947b8590bf53b61c2cb49faec88feaa554ace8af1a2941d
Sep 04 22:01:05.019997 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:05.019962    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:01:05.020089 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:05.020032    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:01:05.020089 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:05.020059    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:01:05.404278 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:05.404173    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9" event={"ID":"921df383-5403-4688-a3f6-ffe059353a5a","Type":"ContainerStarted","Data":"65d3988ba4b693cba5709f3327451456aff203b09be041215ce85c9c3a120101"}
Sep 04 22:01:05.404278 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:05.404241    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9" event={"ID":"921df383-5403-4688-a3f6-ffe059353a5a","Type":"ContainerStarted","Data":"32ec8707a1ea74f88947b8590bf53b61c2cb49faec88feaa554ace8af1a2941d"}
Sep 04 22:01:05.426622 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:05.426567    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9" podStartSLOduration=1.426548602 podStartE2EDuration="1.426548602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:01:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:01:05.425136365 +0000 UTC m=+439.741703107" watchObservedRunningTime="2026-09-04 22:01:05.426548602 +0000 UTC m=+439.743115340"
Sep 04 22:01:05.736999 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:05.736894    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:06.741612 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:06.741574    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:06.778733 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:06.778690    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:01:06.782176 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:06.782145    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-57drl"
Sep 04 22:01:06.784443 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:06.784399    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 04 22:01:06.784443 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:06.784436    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mlfpp\""
Sep 04 22:01:06.784652 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:06.784399    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 04 22:01:06.784652 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:06.784460    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 04 22:01:06.796117 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:06.796083    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:01:06.821863 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:06.821827    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:01:06.825825 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:06.825797    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq2rj\" (UniqueName: \"kubernetes.io/projected/655b62f3-2bbc-4aeb-a1fc-90274bebb94e-kube-api-access-sq2rj\") pod \"limitador-limitador-85d9db4d5d-57drl\" (UID: \"655b62f3-2bbc-4aeb-a1fc-90274bebb94e\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-57drl"
Sep 04 22:01:06.825909 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:06.825852    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/655b62f3-2bbc-4aeb-a1fc-90274bebb94e-config-file\") pod \"limitador-limitador-85d9db4d5d-57drl\" (UID: \"655b62f3-2bbc-4aeb-a1fc-90274bebb94e\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-57drl"
Sep 04 22:01:06.927172 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:06.927125    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq2rj\" (UniqueName: \"kubernetes.io/projected/655b62f3-2bbc-4aeb-a1fc-90274bebb94e-kube-api-access-sq2rj\") pod \"limitador-limitador-85d9db4d5d-57drl\" (UID: \"655b62f3-2bbc-4aeb-a1fc-90274bebb94e\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-57drl"
Sep 04 22:01:06.927351 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:06.927189    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/655b62f3-2bbc-4aeb-a1fc-90274bebb94e-config-file\") pod \"limitador-limitador-85d9db4d5d-57drl\" (UID: \"655b62f3-2bbc-4aeb-a1fc-90274bebb94e\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-57drl"
Sep 04 22:01:06.927848 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:06.927828    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/655b62f3-2bbc-4aeb-a1fc-90274bebb94e-config-file\") pod \"limitador-limitador-85d9db4d5d-57drl\" (UID: \"655b62f3-2bbc-4aeb-a1fc-90274bebb94e\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-57drl"
Sep 04 22:01:06.935814 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:06.935780    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq2rj\" (UniqueName: \"kubernetes.io/projected/655b62f3-2bbc-4aeb-a1fc-90274bebb94e-kube-api-access-sq2rj\") pod \"limitador-limitador-85d9db4d5d-57drl\" (UID: \"655b62f3-2bbc-4aeb-a1fc-90274bebb94e\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-57drl"
Sep 04 22:01:07.095609 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:07.095567    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-57drl"
Sep 04 22:01:07.371104 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:07.371014    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:01:07.374170 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:01:07.374134    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod655b62f3_2bbc_4aeb_a1fc_90274bebb94e.slice/crio-ebab153fbb75c42f86e39e0dfa45cdbafd9fa44377d5adda797585127d50608b WatchSource:0}: Error finding container ebab153fbb75c42f86e39e0dfa45cdbafd9fa44377d5adda797585127d50608b: Status 404 returned error can't find the container with id ebab153fbb75c42f86e39e0dfa45cdbafd9fa44377d5adda797585127d50608b
Sep 04 22:01:07.415511 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:07.415472    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-57drl" event={"ID":"655b62f3-2bbc-4aeb-a1fc-90274bebb94e","Type":"ContainerStarted","Data":"ebab153fbb75c42f86e39e0dfa45cdbafd9fa44377d5adda797585127d50608b"}
Sep 04 22:01:07.415829 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:07.415817    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:07.416711 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:07.416691    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:01:17.468364 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:17.468322    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-57drl" event={"ID":"655b62f3-2bbc-4aeb-a1fc-90274bebb94e","Type":"ContainerStarted","Data":"4d9be1d4314dc21bd00210eeab60ed187d3dd13f8389dfe0c91687bdc4d046e7"}
Sep 04 22:01:17.468785 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:17.468449    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-57drl"
Sep 04 22:01:17.485329 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:17.485267    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-85d9db4d5d-57drl" podStartSLOduration=1.5142962020000001 podStartE2EDuration="11.485248875s" totalImagesPullingTime="9.970952673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:01:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:01:17.483266871 +0000 UTC m=+451.799833634" watchObservedRunningTime="2026-09-04 22:01:17.485248875 +0000 UTC m=+451.801815690"
Sep 04 22:01:28.475977 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:01:28.475947    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-57drl"
Sep 04 22:02:32.633778 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:02:32.633733    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:02:41.893012 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:02:41.892926    2557 generic.go:356] "Generic (PLEG): container finished" podID="921df383-5403-4688-a3f6-ffe059353a5a" containerID="65d3988ba4b693cba5709f3327451456aff203b09be041215ce85c9c3a120101" exitCode=0
Sep 04 22:02:41.893390 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:02:41.893002    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9" event={"ID":"921df383-5403-4688-a3f6-ffe059353a5a","Type":"ContainerDied","Data":"65d3988ba4b693cba5709f3327451456aff203b09be041215ce85c9c3a120101"}
Sep 04 22:02:41.893446 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:02:41.893406    2557 scope.go:117] "RemoveContainer" containerID="65d3988ba4b693cba5709f3327451456aff203b09be041215ce85c9c3a120101"
Sep 04 22:02:41.894402 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:02:41.894375    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:02:41.894492 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:02:41.894476    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:02:41.894528 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:02:41.894517    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:02:42.900397 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:02:42.900350    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9" event={"ID":"921df383-5403-4688-a3f6-ffe059353a5a","Type":"ContainerStarted","Data":"7b3dabf383ee6d23dd2d9afb2befaecfd8102527b85b977d196fbdc1df451922"}
Sep 04 22:02:43.737284 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:02:43.737242    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:02:43.737619 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:02:43.737593    2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-8tbt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 04 22:02:43.737719 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:02:43.737651    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9" podUID="921df383-5403-4688-a3f6-ffe059353a5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 04 22:02:44.737337 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:02:44.737300    2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-8tbt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 04 22:02:44.737735 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:02:44.737374    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9" podUID="921df383-5403-4688-a3f6-ffe059353a5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 04 22:02:45.737526 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:02:45.737482    2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-8tbt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 04 22:02:45.737887 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:02:45.737547    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9" podUID="921df383-5403-4688-a3f6-ffe059353a5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 04 22:02:46.737501 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:02:46.737463    2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-8tbt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 04 22:02:46.737684 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:02:46.737524    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9" podUID="921df383-5403-4688-a3f6-ffe059353a5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 04 22:02:47.737520 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:02:47.737479    2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-8tbt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body=
Sep 04 22:02:47.737728 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:02:47.737542    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9" podUID="921df383-5403-4688-a3f6-ffe059353a5a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused"
Sep 04 22:02:48.740865 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:02:48.740833    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:02:48.741276 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:02:48.741122    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:02:48.741727 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:02:48.741709    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-8tbt9"
Sep 04 22:03:07.847314 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:07.847270    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:03:21.544491 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:21.544449    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:03:27.832201 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:27.832156    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"]
Sep 04 22:03:27.835866 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:27.835846    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:27.838159 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:27.838134    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 04 22:03:27.838952 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:27.838928    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 04 22:03:27.839068 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:27.838965    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6vlzh\""
Sep 04 22:03:27.839068 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:27.838927    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 04 22:03:27.845028 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:27.844991    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"]
Sep 04 22:03:27.972667 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:27.972622    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d65975c5-c461-4d5e-b216-1f3efd2bc7c5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c77vw\" (UID: \"d65975c5-c461-4d5e-b216-1f3efd2bc7c5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:27.972667 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:27.972673    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d65975c5-c461-4d5e-b216-1f3efd2bc7c5-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c77vw\" (UID: \"d65975c5-c461-4d5e-b216-1f3efd2bc7c5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:27.972926 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:27.972694    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sn4t\" (UniqueName: \"kubernetes.io/projected/d65975c5-c461-4d5e-b216-1f3efd2bc7c5-kube-api-access-8sn4t\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c77vw\" (UID: \"d65975c5-c461-4d5e-b216-1f3efd2bc7c5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:27.972926 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:27.972778    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d65975c5-c461-4d5e-b216-1f3efd2bc7c5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c77vw\" (UID: \"d65975c5-c461-4d5e-b216-1f3efd2bc7c5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:27.972926 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:27.972816    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d65975c5-c461-4d5e-b216-1f3efd2bc7c5-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c77vw\" (UID: \"d65975c5-c461-4d5e-b216-1f3efd2bc7c5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:27.972926 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:27.972838    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d65975c5-c461-4d5e-b216-1f3efd2bc7c5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c77vw\" (UID: \"d65975c5-c461-4d5e-b216-1f3efd2bc7c5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:28.073991 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:28.073943    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d65975c5-c461-4d5e-b216-1f3efd2bc7c5-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c77vw\" (UID: \"d65975c5-c461-4d5e-b216-1f3efd2bc7c5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:28.074201 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:28.073998    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sn4t\" (UniqueName: \"kubernetes.io/projected/d65975c5-c461-4d5e-b216-1f3efd2bc7c5-kube-api-access-8sn4t\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c77vw\" (UID: \"d65975c5-c461-4d5e-b216-1f3efd2bc7c5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:28.074201 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:28.074038    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d65975c5-c461-4d5e-b216-1f3efd2bc7c5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c77vw\" (UID: \"d65975c5-c461-4d5e-b216-1f3efd2bc7c5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:28.074201 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:28.074064    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d65975c5-c461-4d5e-b216-1f3efd2bc7c5-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c77vw\" (UID: \"d65975c5-c461-4d5e-b216-1f3efd2bc7c5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:28.074201 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:28.074084    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d65975c5-c461-4d5e-b216-1f3efd2bc7c5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c77vw\" (UID: \"d65975c5-c461-4d5e-b216-1f3efd2bc7c5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:28.074201 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:28.074175    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d65975c5-c461-4d5e-b216-1f3efd2bc7c5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c77vw\" (UID: \"d65975c5-c461-4d5e-b216-1f3efd2bc7c5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:28.074541 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:28.074515    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d65975c5-c461-4d5e-b216-1f3efd2bc7c5-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c77vw\" (UID: \"d65975c5-c461-4d5e-b216-1f3efd2bc7c5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:28.074602 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:28.074553    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d65975c5-c461-4d5e-b216-1f3efd2bc7c5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c77vw\" (UID: \"d65975c5-c461-4d5e-b216-1f3efd2bc7c5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:28.074602 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:28.074578    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d65975c5-c461-4d5e-b216-1f3efd2bc7c5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c77vw\" (UID: \"d65975c5-c461-4d5e-b216-1f3efd2bc7c5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:28.076267 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:28.076251    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d65975c5-c461-4d5e-b216-1f3efd2bc7c5-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c77vw\" (UID: \"d65975c5-c461-4d5e-b216-1f3efd2bc7c5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:28.076646 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:28.076626    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d65975c5-c461-4d5e-b216-1f3efd2bc7c5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c77vw\" (UID: \"d65975c5-c461-4d5e-b216-1f3efd2bc7c5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:28.081897 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:28.081868    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sn4t\" (UniqueName: \"kubernetes.io/projected/d65975c5-c461-4d5e-b216-1f3efd2bc7c5-kube-api-access-8sn4t\") pod \"e2e-trlp-test-simulated-kserve-598466548d-c77vw\" (UID: \"d65975c5-c461-4d5e-b216-1f3efd2bc7c5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:28.147740 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:28.147639    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:28.426976 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:28.426943    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"]
Sep 04 22:03:28.428653 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:03:28.428626    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd65975c5_c461_4d5e_b216_1f3efd2bc7c5.slice/crio-8806c264fac02ff19fd9ae8653dcdbbb7f7d8fda31216b97cde3db01a069c205 WatchSource:0}: Error finding container 8806c264fac02ff19fd9ae8653dcdbbb7f7d8fda31216b97cde3db01a069c205: Status 404 returned error can't find the container with id 8806c264fac02ff19fd9ae8653dcdbbb7f7d8fda31216b97cde3db01a069c205
Sep 04 22:03:28.852819 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:28.852773    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:03:29.133855 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:29.133766    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw" event={"ID":"d65975c5-c461-4d5e-b216-1f3efd2bc7c5","Type":"ContainerStarted","Data":"8806c264fac02ff19fd9ae8653dcdbbb7f7d8fda31216b97cde3db01a069c205"}
Sep 04 22:03:30.141011 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:30.140976    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw" event={"ID":"d65975c5-c461-4d5e-b216-1f3efd2bc7c5","Type":"ContainerStarted","Data":"51851c3f3f8c84d6cc7594fadb174efbd997946ba6c63a06273b3f3c33f28190"}
Sep 04 22:03:30.159811 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:30.159751    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw" podStartSLOduration=2.206576604 podStartE2EDuration="3.1597315s" totalImagesPullingTime="953.154896ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:03:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:03:30.157017233 +0000 UTC m=+584.473583983" watchObservedRunningTime="2026-09-04 22:03:30.1597315 +0000 UTC m=+584.476298304"
Sep 04 22:03:38.147811 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:38.147772    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:38.148327 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:38.147867    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:38.160401 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:38.160368    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:38.196455 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:38.196408    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-c77vw"
Sep 04 22:03:45.962299 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:03:45.962259    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:04:02.435976 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:02.435937    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"]
Sep 04 22:04:02.439692 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:02.439664    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:02.442132 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:02.442106    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 04 22:04:02.449582 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:02.449550    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"]
Sep 04 22:04:02.471167 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:02.471130    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jlpw\" (UniqueName: \"kubernetes.io/projected/a89fc6fc-d17b-4a96-97fd-a6d75fe6090d-kube-api-access-4jlpw\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4q4t7\" (UID: \"a89fc6fc-d17b-4a96-97fd-a6d75fe6090d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:02.471343 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:02.471188    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a89fc6fc-d17b-4a96-97fd-a6d75fe6090d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4q4t7\" (UID: \"a89fc6fc-d17b-4a96-97fd-a6d75fe6090d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:02.471343 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:02.471213    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a89fc6fc-d17b-4a96-97fd-a6d75fe6090d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4q4t7\" (UID: \"a89fc6fc-d17b-4a96-97fd-a6d75fe6090d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:02.471343 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:02.471256    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a89fc6fc-d17b-4a96-97fd-a6d75fe6090d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4q4t7\" (UID: \"a89fc6fc-d17b-4a96-97fd-a6d75fe6090d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:02.471343 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:02.471286    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a89fc6fc-d17b-4a96-97fd-a6d75fe6090d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4q4t7\" (UID: \"a89fc6fc-d17b-4a96-97fd-a6d75fe6090d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:02.471508 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:02.471338    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a89fc6fc-d17b-4a96-97fd-a6d75fe6090d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4q4t7\" (UID: \"a89fc6fc-d17b-4a96-97fd-a6d75fe6090d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:02.571800 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:02.571755    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a89fc6fc-d17b-4a96-97fd-a6d75fe6090d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4q4t7\" (UID: \"a89fc6fc-d17b-4a96-97fd-a6d75fe6090d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:02.571800 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:02.571805    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a89fc6fc-d17b-4a96-97fd-a6d75fe6090d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4q4t7\" (UID: \"a89fc6fc-d17b-4a96-97fd-a6d75fe6090d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:02.572081 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:02.571839    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a89fc6fc-d17b-4a96-97fd-a6d75fe6090d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4q4t7\" (UID: \"a89fc6fc-d17b-4a96-97fd-a6d75fe6090d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:02.572081 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:02.571888    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a89fc6fc-d17b-4a96-97fd-a6d75fe6090d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4q4t7\" (UID: \"a89fc6fc-d17b-4a96-97fd-a6d75fe6090d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:02.572081 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:02.571919    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a89fc6fc-d17b-4a96-97fd-a6d75fe6090d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4q4t7\" (UID: \"a89fc6fc-d17b-4a96-97fd-a6d75fe6090d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:02.572081 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:02.571986    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jlpw\" (UniqueName: \"kubernetes.io/projected/a89fc6fc-d17b-4a96-97fd-a6d75fe6090d-kube-api-access-4jlpw\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4q4t7\" (UID: \"a89fc6fc-d17b-4a96-97fd-a6d75fe6090d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:02.572280 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:02.572214    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a89fc6fc-d17b-4a96-97fd-a6d75fe6090d-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4q4t7\" (UID: \"a89fc6fc-d17b-4a96-97fd-a6d75fe6090d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:02.572280 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:02.572267    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a89fc6fc-d17b-4a96-97fd-a6d75fe6090d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4q4t7\" (UID: \"a89fc6fc-d17b-4a96-97fd-a6d75fe6090d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:02.572388 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:02.572318    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a89fc6fc-d17b-4a96-97fd-a6d75fe6090d-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4q4t7\" (UID: \"a89fc6fc-d17b-4a96-97fd-a6d75fe6090d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:02.574220 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:02.574201    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a89fc6fc-d17b-4a96-97fd-a6d75fe6090d-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4q4t7\" (UID: \"a89fc6fc-d17b-4a96-97fd-a6d75fe6090d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:02.574371 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:02.574352    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a89fc6fc-d17b-4a96-97fd-a6d75fe6090d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4q4t7\" (UID: \"a89fc6fc-d17b-4a96-97fd-a6d75fe6090d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:02.582237 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:02.582209    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jlpw\" (UniqueName: \"kubernetes.io/projected/a89fc6fc-d17b-4a96-97fd-a6d75fe6090d-kube-api-access-4jlpw\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-4q4t7\" (UID: \"a89fc6fc-d17b-4a96-97fd-a6d75fe6090d\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:02.750641 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:02.750597    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:03.027089 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:03.027062    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"]
Sep 04 22:04:03.029300 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:04:03.029264    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda89fc6fc_d17b_4a96_97fd_a6d75fe6090d.slice/crio-68da5dd7998d832f5b6463e158ccffb26a987dfda07747cf8e25b31726e95829 WatchSource:0}: Error finding container 68da5dd7998d832f5b6463e158ccffb26a987dfda07747cf8e25b31726e95829: Status 404 returned error can't find the container with id 68da5dd7998d832f5b6463e158ccffb26a987dfda07747cf8e25b31726e95829
Sep 04 22:04:03.308241 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:03.308147    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7" event={"ID":"a89fc6fc-d17b-4a96-97fd-a6d75fe6090d","Type":"ContainerStarted","Data":"67ba8fcb342478120dcf489b7126208a913bd40c831a1e29838b670d456792bf"}
Sep 04 22:04:03.308241 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:03.308201    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7" event={"ID":"a89fc6fc-d17b-4a96-97fd-a6d75fe6090d","Type":"ContainerStarted","Data":"68da5dd7998d832f5b6463e158ccffb26a987dfda07747cf8e25b31726e95829"}
Sep 04 22:04:03.327541 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:03.327474    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7" podStartSLOduration=1.152856461 podStartE2EDuration="1.327451886s" totalImagesPullingTime="174.595425ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:04:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:04:03.325801277 +0000 UTC m=+617.642368016" watchObservedRunningTime="2026-09-04 22:04:03.327451886 +0000 UTC m=+617.644018625"
Sep 04 22:04:04.038425 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:04.038377    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:04:12.751446 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:12.751330    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:12.751446 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:12.751440    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:12.764086 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:12.764054    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:13.228658 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.228553    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"]
Sep 04 22:04:13.236543 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.236517    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:04:13.238959 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.238934    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 04 22:04:13.243778 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.243746    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"]
Sep 04 22:04:13.365842 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.365799    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf\" (UID: \"c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:04:13.365842 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.365847    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxww7\" (UniqueName: \"kubernetes.io/projected/c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07-kube-api-access-lxww7\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf\" (UID: \"c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:04:13.366061 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.365903    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf\" (UID: \"c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:04:13.366061 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.365928    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf\" (UID: \"c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:04:13.366061 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.365952    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf\" (UID: \"c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:04:13.366061 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.365972    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf\" (UID: \"c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:04:13.372587 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.372561    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-4q4t7"
Sep 04 22:04:13.466458 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.466391    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf\" (UID: \"c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:04:13.466458 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.466462    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxww7\" (UniqueName: \"kubernetes.io/projected/c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07-kube-api-access-lxww7\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf\" (UID: \"c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:04:13.466706 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.466577    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf\" (UID: \"c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:04:13.466706 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.466610    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf\" (UID: \"c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:04:13.466706 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.466650    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf\" (UID: \"c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:04:13.466706 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.466675    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf\" (UID: \"c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:04:13.466935 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.466803    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf\" (UID: \"c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:04:13.467011 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.466985    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf\" (UID: \"c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:04:13.467318 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.467288    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf\" (UID: \"c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:04:13.468875 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.468853    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf\" (UID: \"c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:04:13.469231 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.469216    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf\" (UID: \"c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:04:13.474679 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.474654    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxww7\" (UniqueName: \"kubernetes.io/projected/c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07-kube-api-access-lxww7\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf\" (UID: \"c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:04:13.548091 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.548039    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:04:13.824631 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:13.824600    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"]
Sep 04 22:04:13.826757 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:04:13.826721    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3b9225b_32cb_4fc2_bbb5_bc84e0f68d07.slice/crio-917641e021319f8a046b5aed5f9163372ec0ae3b65982c50f78a8acd986d7a6d WatchSource:0}: Error finding container 917641e021319f8a046b5aed5f9163372ec0ae3b65982c50f78a8acd986d7a6d: Status 404 returned error can't find the container with id 917641e021319f8a046b5aed5f9163372ec0ae3b65982c50f78a8acd986d7a6d
Sep 04 22:04:14.367992 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:14.367946    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf" event={"ID":"c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07","Type":"ContainerStarted","Data":"575c9ffd7d4253e8772112b5612ad0b614d2de5bb117d32bdbbed7d59b8b2403"}
Sep 04 22:04:14.367992 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:14.367998    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf" event={"ID":"c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07","Type":"ContainerStarted","Data":"917641e021319f8a046b5aed5f9163372ec0ae3b65982c50f78a8acd986d7a6d"}
Sep 04 22:04:14.387937 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:14.387875    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf" podStartSLOduration=1.207971981 podStartE2EDuration="1.387856151s" totalImagesPullingTime="179.88417ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:04:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:04:14.38568664 +0000 UTC m=+628.702253379" watchObservedRunningTime="2026-09-04 22:04:14.387856151 +0000 UTC m=+628.704422889"
Sep 04 22:04:15.054112 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:15.054068    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:04:23.548970 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:23.548922    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:04:23.549369 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:23.548986    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:04:23.561658 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:23.561625    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:04:24.439279 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:04:24.439245    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf"
Sep 04 22:05:35.551334 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:05:35.551298    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:05:44.750142 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:05:44.750044    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:05:57.874495 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:05:57.874452    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:06:09.437138 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:09.437090    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f81019dc-4ll74"]
Sep 04 22:06:09.440854 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:09.440825    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f81019dc-4ll74"
Sep 04 22:06:09.445242 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:09.445207    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 04 22:06:09.445485 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:09.445244    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dftg6\""
Sep 04 22:06:09.454515 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:09.454481    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f81019dc-4ll74"]
Sep 04 22:06:09.494257 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:09.494206    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94"]
Sep 04 22:06:09.497634 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:09.497615    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94"
Sep 04 22:06:09.500652 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:09.500618    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-d4ad2d-dockercfg-d4zdh\""
Sep 04 22:06:09.500788 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:09.500667    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-d4ad2d\""
Sep 04 22:06:09.508618 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:09.508589    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94"]
Sep 04 22:06:09.536197 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:09.536156    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d245566-9664-459c-beac-a51fcbc60bae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f81019dc-4ll74\" (UID: \"5d245566-9664-459c-beac-a51fcbc60bae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f81019dc-4ll74"
Sep 04 22:06:09.636900 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:09.636857    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2vb7\" (UniqueName: \"kubernetes.io/projected/40e25751-a369-4ffc-93b6-5246cfdf7bb7-kube-api-access-b2vb7\") pod \"payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94\" (UID: \"40e25751-a369-4ffc-93b6-5246cfdf7bb7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94"
Sep 04 22:06:09.636900 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:09.636905    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d245566-9664-459c-beac-a51fcbc60bae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f81019dc-4ll74\" (UID: \"5d245566-9664-459c-beac-a51fcbc60bae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f81019dc-4ll74"
Sep 04 22:06:09.637156 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:09.637043    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40e25751-a369-4ffc-93b6-5246cfdf7bb7-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94\" (UID: \"40e25751-a369-4ffc-93b6-5246cfdf7bb7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94"
Sep 04 22:06:09.637490 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:09.637474    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d245566-9664-459c-beac-a51fcbc60bae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f81019dc-4ll74\" (UID: \"5d245566-9664-459c-beac-a51fcbc60bae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f81019dc-4ll74"
Sep 04 22:06:09.738037 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:09.737931    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40e25751-a369-4ffc-93b6-5246cfdf7bb7-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94\" (UID: \"40e25751-a369-4ffc-93b6-5246cfdf7bb7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94"
Sep 04 22:06:09.738211 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:09.738104    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2vb7\" (UniqueName: \"kubernetes.io/projected/40e25751-a369-4ffc-93b6-5246cfdf7bb7-kube-api-access-b2vb7\") pod \"payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94\" (UID: \"40e25751-a369-4ffc-93b6-5246cfdf7bb7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94"
Sep 04 22:06:09.738682 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:09.738659    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40e25751-a369-4ffc-93b6-5246cfdf7bb7-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94\" (UID: \"40e25751-a369-4ffc-93b6-5246cfdf7bb7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94"
Sep 04 22:06:09.752308 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:09.752275    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f81019dc-4ll74"
Sep 04 22:06:09.753293 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:09.753270    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2vb7\" (UniqueName: \"kubernetes.io/projected/40e25751-a369-4ffc-93b6-5246cfdf7bb7-kube-api-access-b2vb7\") pod \"payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94\" (UID: \"40e25751-a369-4ffc-93b6-5246cfdf7bb7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94"
Sep 04 22:06:09.808805 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:09.808771    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94"
Sep 04 22:06:10.030489 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:10.030453    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f81019dc-4ll74"]
Sep 04 22:06:10.031725 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:06:10.031690    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d245566_9664_459c_beac_a51fcbc60bae.slice/crio-db0c5aa647e593dfb11bed7c72bdc89f55cf4b32fecb511d65d360b9b437b94e WatchSource:0}: Error finding container db0c5aa647e593dfb11bed7c72bdc89f55cf4b32fecb511d65d360b9b437b94e: Status 404 returned error can't find the container with id db0c5aa647e593dfb11bed7c72bdc89f55cf4b32fecb511d65d360b9b437b94e
Sep 04 22:06:10.033540 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:10.033522    2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 04 22:06:10.092382 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:10.092354    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94"]
Sep 04 22:06:10.093994 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:06:10.093964    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40e25751_a369_4ffc_93b6_5246cfdf7bb7.slice/crio-4fd0d2f4f0a797f8c728c238ada320a5edc05daf9eae096278274d6afbe81aec WatchSource:0}: Error finding container 4fd0d2f4f0a797f8c728c238ada320a5edc05daf9eae096278274d6afbe81aec: Status 404 returned error can't find the container with id 4fd0d2f4f0a797f8c728c238ada320a5edc05daf9eae096278274d6afbe81aec
Sep 04 22:06:11.008844 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:11.008803    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f81019dc-4ll74" event={"ID":"5d245566-9664-459c-beac-a51fcbc60bae","Type":"ContainerStarted","Data":"db0c5aa647e593dfb11bed7c72bdc89f55cf4b32fecb511d65d360b9b437b94e"}
Sep 04 22:06:11.010952 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:11.010894    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94" event={"ID":"40e25751-a369-4ffc-93b6-5246cfdf7bb7","Type":"ContainerStarted","Data":"4fd0d2f4f0a797f8c728c238ada320a5edc05daf9eae096278274d6afbe81aec"}
Sep 04 22:06:13.028640 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:13.028591    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f81019dc-4ll74" event={"ID":"5d245566-9664-459c-beac-a51fcbc60bae","Type":"ContainerStarted","Data":"806e24937e0c41b411d2b7e738e38e9750bb77187f99003f3b65ceea8e3b2f46"}
Sep 04 22:06:13.030362 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:13.030328    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94" event={"ID":"40e25751-a369-4ffc-93b6-5246cfdf7bb7","Type":"ContainerStarted","Data":"e752c6bc13d8bf37befcbb51ee3ab403342e5b8b8696279dcd57a42220b46941"}
Sep 04 22:06:13.030486 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:13.030386    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94"
Sep 04 22:06:13.044932 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:13.044805    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f81019dc-4ll74" podStartSLOduration=1.570623638 podStartE2EDuration="4.044787313s" totalImagesPullingTime="2.474163675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:06:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:06:13.043991292 +0000 UTC m=+747.360558030" watchObservedRunningTime="2026-09-04 22:06:13.044787313 +0000 UTC m=+747.361354051"
Sep 04 22:06:13.053054 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:13.053015    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:06:13.062674 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:13.062614    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94" podStartSLOduration=1.645056722 podStartE2EDuration="4.062593356s" totalImagesPullingTime="2.417536634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:06:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:06:13.062000964 +0000 UTC m=+747.378567702" watchObservedRunningTime="2026-09-04 22:06:13.062593356 +0000 UTC m=+747.379160097"
Sep 04 22:06:14.039790 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:14.039753    2557 generic.go:356] "Generic (PLEG): container finished" podID="5d245566-9664-459c-beac-a51fcbc60bae" containerID="806e24937e0c41b411d2b7e738e38e9750bb77187f99003f3b65ceea8e3b2f46" exitCode=0
Sep 04 22:06:14.040266 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:14.039846    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f81019dc-4ll74" event={"ID":"5d245566-9664-459c-beac-a51fcbc60bae","Type":"ContainerDied","Data":"806e24937e0c41b411d2b7e738e38e9750bb77187f99003f3b65ceea8e3b2f46"}
Sep 04 22:06:15.179437 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:15.179385    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f81019dc-4ll74"
Sep 04 22:06:15.182501 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:15.182477    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d245566-9664-459c-beac-a51fcbc60bae-maas-api-service-ca\") pod \"5d245566-9664-459c-beac-a51fcbc60bae\" (UID: \"5d245566-9664-459c-beac-a51fcbc60bae\") "
Sep 04 22:06:15.182828 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:15.182804    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d245566-9664-459c-beac-a51fcbc60bae-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5d245566-9664-459c-beac-a51fcbc60bae" (UID: "5d245566-9664-459c-beac-a51fcbc60bae"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:06:15.283262 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:15.283223    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d245566-9664-459c-beac-a51fcbc60bae-maas-api-service-ca\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:06:16.051934 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:16.051903    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f81019dc-4ll74"
Sep 04 22:06:16.052110 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:16.051903    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f81019dc-4ll74" event={"ID":"5d245566-9664-459c-beac-a51fcbc60bae","Type":"ContainerDied","Data":"db0c5aa647e593dfb11bed7c72bdc89f55cf4b32fecb511d65d360b9b437b94e"}
Sep 04 22:06:16.052110 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:16.052032    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="db0c5aa647e593dfb11bed7c72bdc89f55cf4b32fecb511d65d360b9b437b94e"
Sep 04 22:06:24.042989 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:24.042956    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94"
Sep 04 22:06:26.136319 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:26.136280    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f81019dc-4ll74"]
Sep 04 22:06:26.138316 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:26.138289    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f81019dc-4ll74"]
Sep 04 22:06:26.353164 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:26.353128    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d245566-9664-459c-beac-a51fcbc60bae" path="/var/lib/kubelet/pods/5d245566-9664-459c-beac-a51fcbc60bae/volumes"
Sep 04 22:06:29.051956 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:29.051916    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:06:31.343448 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.343401    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv"]
Sep 04 22:06:31.343864 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.343762    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d245566-9664-459c-beac-a51fcbc60bae" containerName="revoke-keys"
Sep 04 22:06:31.343864 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.343774    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d245566-9664-459c-beac-a51fcbc60bae" containerName="revoke-keys"
Sep 04 22:06:31.343864 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.343835    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d245566-9664-459c-beac-a51fcbc60bae" containerName="revoke-keys"
Sep 04 22:06:31.347231 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.347206    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv"
Sep 04 22:06:31.349910 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.349884    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-6fda89ac-dockercfg-sdhqz\""
Sep 04 22:06:31.350081 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.350058    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-6fda89ac\""
Sep 04 22:06:31.359385 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.359352    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv"]
Sep 04 22:06:31.386535 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.386497    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59"]
Sep 04 22:06:31.390038 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.390015    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59"
Sep 04 22:06:31.399617 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.399586    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59"]
Sep 04 22:06:31.439558 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.439522    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ht585\" (UniqueName: \"kubernetes.io/projected/fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1-kube-api-access-ht585\") pod \"payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv\" (UID: \"fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv"
Sep 04 22:06:31.439732 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.439573    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkptp\" (UniqueName: \"kubernetes.io/projected/7f61a89f-9de9-4610-838a-1967647f8139-kube-api-access-tkptp\") pod \"payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59\" (UID: \"7f61a89f-9de9-4610-838a-1967647f8139\") " pod="openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59"
Sep 04 22:06:31.439732 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.439630    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f61a89f-9de9-4610-838a-1967647f8139-plugins-config-volume\") pod \"payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59\" (UID: \"7f61a89f-9de9-4610-838a-1967647f8139\") " pod="openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59"
Sep 04 22:06:31.439732 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.439668    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv\" (UID: \"fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv"
Sep 04 22:06:31.540704 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.540643    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv\" (UID: \"fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv"
Sep 04 22:06:31.540896 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.540731    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ht585\" (UniqueName: \"kubernetes.io/projected/fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1-kube-api-access-ht585\") pod \"payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv\" (UID: \"fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv"
Sep 04 22:06:31.540896 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.540767    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkptp\" (UniqueName: \"kubernetes.io/projected/7f61a89f-9de9-4610-838a-1967647f8139-kube-api-access-tkptp\") pod \"payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59\" (UID: \"7f61a89f-9de9-4610-838a-1967647f8139\") " pod="openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59"
Sep 04 22:06:31.540896 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.540812    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f61a89f-9de9-4610-838a-1967647f8139-plugins-config-volume\") pod \"payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59\" (UID: \"7f61a89f-9de9-4610-838a-1967647f8139\") " pod="openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59"
Sep 04 22:06:31.541521 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.541496    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv\" (UID: \"fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv"
Sep 04 22:06:31.541521 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.541513    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f61a89f-9de9-4610-838a-1967647f8139-plugins-config-volume\") pod \"payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59\" (UID: \"7f61a89f-9de9-4610-838a-1967647f8139\") " pod="openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59"
Sep 04 22:06:31.552333 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.552298    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ht585\" (UniqueName: \"kubernetes.io/projected/fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1-kube-api-access-ht585\") pod \"payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv\" (UID: \"fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv"
Sep 04 22:06:31.557107 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.557076    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkptp\" (UniqueName: \"kubernetes.io/projected/7f61a89f-9de9-4610-838a-1967647f8139-kube-api-access-tkptp\") pod \"payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59\" (UID: \"7f61a89f-9de9-4610-838a-1967647f8139\") " pod="openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59"
Sep 04 22:06:31.658796 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.658700    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv"
Sep 04 22:06:31.702166 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.702129    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59"
Sep 04 22:06:31.935675 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.935650    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv"]
Sep 04 22:06:31.937249 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:06:31.937226    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe35d4b6_cc8f_4f12_9453_ed7bacc3e0e1.slice/crio-3c5b09447c3c4eb3313bf5515a933810b792d5a5a014f3c9df7289e8526dfcc7 WatchSource:0}: Error finding container 3c5b09447c3c4eb3313bf5515a933810b792d5a5a014f3c9df7289e8526dfcc7: Status 404 returned error can't find the container with id 3c5b09447c3c4eb3313bf5515a933810b792d5a5a014f3c9df7289e8526dfcc7
Sep 04 22:06:31.991727 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:31.991603    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59"]
Sep 04 22:06:31.997249 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:06:31.997209    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f61a89f_9de9_4610_838a_1967647f8139.slice/crio-36724ee91ede295dcb746f0b4ff5c9cfef089cc7a6c55dd50252b6723c1fa483 WatchSource:0}: Error finding container 36724ee91ede295dcb746f0b4ff5c9cfef089cc7a6c55dd50252b6723c1fa483: Status 404 returned error can't find the container with id 36724ee91ede295dcb746f0b4ff5c9cfef089cc7a6c55dd50252b6723c1fa483
Sep 04 22:06:32.143360 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:32.143320    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59" event={"ID":"7f61a89f-9de9-4610-838a-1967647f8139","Type":"ContainerStarted","Data":"94c586e92b8a0fe67e99c170a7a7d9357e243b984db8793c2398bc9d2a812072"}
Sep 04 22:06:32.143557 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:32.143385    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59" event={"ID":"7f61a89f-9de9-4610-838a-1967647f8139","Type":"ContainerStarted","Data":"36724ee91ede295dcb746f0b4ff5c9cfef089cc7a6c55dd50252b6723c1fa483"}
Sep 04 22:06:32.143557 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:32.143530    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59"
Sep 04 22:06:32.145092 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:32.145067    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv" event={"ID":"fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1","Type":"ContainerStarted","Data":"d0ebf23f06fe35717f751249de61bdd98ae72115f6ba58500dbd9000b23bc467"}
Sep 04 22:06:32.145467 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:32.145109    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv" event={"ID":"fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1","Type":"ContainerStarted","Data":"3c5b09447c3c4eb3313bf5515a933810b792d5a5a014f3c9df7289e8526dfcc7"}
Sep 04 22:06:32.145467 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:32.145164    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv"
Sep 04 22:06:32.162075 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:32.162014    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59" podStartSLOduration=1.1619948390000001 podStartE2EDuration="1.161994839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:06:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:06:32.159326901 +0000 UTC m=+766.475893678" watchObservedRunningTime="2026-09-04 22:06:32.161994839 +0000 UTC m=+766.478561575"
Sep 04 22:06:32.176349 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:32.176298    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv" podStartSLOduration=1.1762825590000001 podStartE2EDuration="1.176282559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:06:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:06:32.175371626 +0000 UTC m=+766.491938364" watchObservedRunningTime="2026-09-04 22:06:32.176282559 +0000 UTC m=+766.492849297"
Sep 04 22:06:42.760748 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:42.760708    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:06:43.155204 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:43.155118    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv"
Sep 04 22:06:43.155204 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:43.155188    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59"
Sep 04 22:06:53.454296 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:53.454257    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:06:54.034969 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.034926    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"]
Sep 04 22:06:54.043911 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.043883    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:06:54.046756 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.046727    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a5e1c2\"/\"kube-root-ca.crt\""
Sep 04 22:06:54.047836 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.047710    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-a5e1c2\"/\"openshift-service-ca.crt\""
Sep 04 22:06:54.047836 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.047779    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a5e1c2\"/\"default-dockercfg-6vkj8\""
Sep 04 22:06:54.047836 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.047721    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-a5e1c2\"/\"test-model-a5e1c2-kserve-self-signed-certs\""
Sep 04 22:06:54.048561 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.048536    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"]
Sep 04 22:06:54.147559 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.147520    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k876k\" (UniqueName: \"kubernetes.io/projected/c316e698-4d33-47ac-9494-fa22b213a8d6-kube-api-access-k876k\") pod \"test-model-a5e1c2-kserve-598bfc6875-bcmn8\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") " pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:06:54.147770 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.147584    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-tmp-dir\") pod \"test-model-a5e1c2-kserve-598bfc6875-bcmn8\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") " pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:06:54.147770 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.147608    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-home\") pod \"test-model-a5e1c2-kserve-598bfc6875-bcmn8\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") " pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:06:54.147770 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.147674    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-model-cache\") pod \"test-model-a5e1c2-kserve-598bfc6875-bcmn8\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") " pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:06:54.147770 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.147714    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c316e698-4d33-47ac-9494-fa22b213a8d6-tls-certs\") pod \"test-model-a5e1c2-kserve-598bfc6875-bcmn8\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") " pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:06:54.147770 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.147759    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-dshm\") pod \"test-model-a5e1c2-kserve-598bfc6875-bcmn8\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") " pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:06:54.249003 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.248961    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k876k\" (UniqueName: \"kubernetes.io/projected/c316e698-4d33-47ac-9494-fa22b213a8d6-kube-api-access-k876k\") pod \"test-model-a5e1c2-kserve-598bfc6875-bcmn8\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") " pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:06:54.249210 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.249106    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-tmp-dir\") pod \"test-model-a5e1c2-kserve-598bfc6875-bcmn8\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") " pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:06:54.249210 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.249137    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-home\") pod \"test-model-a5e1c2-kserve-598bfc6875-bcmn8\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") " pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:06:54.249210 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.249163    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-model-cache\") pod \"test-model-a5e1c2-kserve-598bfc6875-bcmn8\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") " pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:06:54.249210 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.249186    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c316e698-4d33-47ac-9494-fa22b213a8d6-tls-certs\") pod \"test-model-a5e1c2-kserve-598bfc6875-bcmn8\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") " pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:06:54.249410 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.249222    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-dshm\") pod \"test-model-a5e1c2-kserve-598bfc6875-bcmn8\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") " pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:06:54.249592 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.249568    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-tmp-dir\") pod \"test-model-a5e1c2-kserve-598bfc6875-bcmn8\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") " pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:06:54.249632 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.249580    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-home\") pod \"test-model-a5e1c2-kserve-598bfc6875-bcmn8\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") " pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:06:54.249677 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.249626    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-model-cache\") pod \"test-model-a5e1c2-kserve-598bfc6875-bcmn8\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") " pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:06:54.251495 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.251463    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-dshm\") pod \"test-model-a5e1c2-kserve-598bfc6875-bcmn8\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") " pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:06:54.251774 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.251757    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c316e698-4d33-47ac-9494-fa22b213a8d6-tls-certs\") pod \"test-model-a5e1c2-kserve-598bfc6875-bcmn8\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") " pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:06:54.257659 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.257631    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k876k\" (UniqueName: \"kubernetes.io/projected/c316e698-4d33-47ac-9494-fa22b213a8d6-kube-api-access-k876k\") pod \"test-model-a5e1c2-kserve-598bfc6875-bcmn8\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") " pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:06:54.354958 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.354875    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:06:54.629153 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:54.629125    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"]
Sep 04 22:06:54.630455 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:06:54.630430    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc316e698_4d33_47ac_9494_fa22b213a8d6.slice/crio-e73368ddbcb1a1d251d9292aa8cd2f4f432e435d0ebcba05129fd9681a9313ac WatchSource:0}: Error finding container e73368ddbcb1a1d251d9292aa8cd2f4f432e435d0ebcba05129fd9681a9313ac: Status 404 returned error can't find the container with id e73368ddbcb1a1d251d9292aa8cd2f4f432e435d0ebcba05129fd9681a9313ac
Sep 04 22:06:55.270702 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:55.270655    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8" event={"ID":"c316e698-4d33-47ac-9494-fa22b213a8d6","Type":"ContainerStarted","Data":"8c815138ecd5da7e325fea4c689e09af7e32eb7ab93d536add2988f464c8aee7"}
Sep 04 22:06:55.270864 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:55.270712    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8" event={"ID":"c316e698-4d33-47ac-9494-fa22b213a8d6","Type":"ContainerStarted","Data":"e73368ddbcb1a1d251d9292aa8cd2f4f432e435d0ebcba05129fd9681a9313ac"}
Sep 04 22:06:55.289470 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:06:55.289399    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8" podStartSLOduration=1.28938035 podStartE2EDuration="1.28938035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:06:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:06:55.28654147 +0000 UTC m=+789.603108211" watchObservedRunningTime="2026-09-04 22:06:55.28938035 +0000 UTC m=+789.605947087"
Sep 04 22:07:01.748462 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:01.748405    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:07:02.007184 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.007075    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59"]
Sep 04 22:07:02.007366 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.007337    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59" podUID="7f61a89f-9de9-4610-838a-1967647f8139" containerName="payload-processing" containerID="cri-o://94c586e92b8a0fe67e99c170a7a7d9357e243b984db8793c2398bc9d2a812072" gracePeriod=30
Sep 04 22:07:02.015608 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.015344    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv"]
Sep 04 22:07:02.016577 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.016278    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv" podUID="fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1" containerName="payload-pre-processing" containerID="cri-o://d0ebf23f06fe35717f751249de61bdd98ae72115f6ba58500dbd9000b23bc467" gracePeriod=30
Sep 04 22:07:02.260273 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.260200    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59"
Sep 04 22:07:02.288288 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.288256    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv"
Sep 04 22:07:02.309641 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.309582    2557 generic.go:356] "Generic (PLEG): container finished" podID="fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1" containerID="d0ebf23f06fe35717f751249de61bdd98ae72115f6ba58500dbd9000b23bc467" exitCode=0
Sep 04 22:07:02.309641 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.309630    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv"
Sep 04 22:07:02.309885 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.309672    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv" event={"ID":"fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1","Type":"ContainerDied","Data":"d0ebf23f06fe35717f751249de61bdd98ae72115f6ba58500dbd9000b23bc467"}
Sep 04 22:07:02.309885 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.309747    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv" event={"ID":"fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1","Type":"ContainerDied","Data":"3c5b09447c3c4eb3313bf5515a933810b792d5a5a014f3c9df7289e8526dfcc7"}
Sep 04 22:07:02.309885 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.309775    2557 scope.go:117] "RemoveContainer" containerID="d0ebf23f06fe35717f751249de61bdd98ae72115f6ba58500dbd9000b23bc467"
Sep 04 22:07:02.312393 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.312130    2557 generic.go:356] "Generic (PLEG): container finished" podID="7f61a89f-9de9-4610-838a-1967647f8139" containerID="94c586e92b8a0fe67e99c170a7a7d9357e243b984db8793c2398bc9d2a812072" exitCode=0
Sep 04 22:07:02.312393 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.312200    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59"
Sep 04 22:07:02.312393 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.312208    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59" event={"ID":"7f61a89f-9de9-4610-838a-1967647f8139","Type":"ContainerDied","Data":"94c586e92b8a0fe67e99c170a7a7d9357e243b984db8793c2398bc9d2a812072"}
Sep 04 22:07:02.312393 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.312254    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59" event={"ID":"7f61a89f-9de9-4610-838a-1967647f8139","Type":"ContainerDied","Data":"36724ee91ede295dcb746f0b4ff5c9cfef089cc7a6c55dd50252b6723c1fa483"}
Sep 04 22:07:02.325567 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.325525    2557 scope.go:117] "RemoveContainer" containerID="d0ebf23f06fe35717f751249de61bdd98ae72115f6ba58500dbd9000b23bc467"
Sep 04 22:07:02.326077 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:07:02.325985    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0ebf23f06fe35717f751249de61bdd98ae72115f6ba58500dbd9000b23bc467\": container with ID starting with d0ebf23f06fe35717f751249de61bdd98ae72115f6ba58500dbd9000b23bc467 not found: ID does not exist" containerID="d0ebf23f06fe35717f751249de61bdd98ae72115f6ba58500dbd9000b23bc467"
Sep 04 22:07:02.326077 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.326041    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0ebf23f06fe35717f751249de61bdd98ae72115f6ba58500dbd9000b23bc467"} err="failed to get container status \"d0ebf23f06fe35717f751249de61bdd98ae72115f6ba58500dbd9000b23bc467\": rpc error: code = NotFound desc = could not find container \"d0ebf23f06fe35717f751249de61bdd98ae72115f6ba58500dbd9000b23bc467\": container with ID starting with d0ebf23f06fe35717f751249de61bdd98ae72115f6ba58500dbd9000b23bc467 not found: ID does not exist"
Sep 04 22:07:02.326077 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.326068    2557 scope.go:117] "RemoveContainer" containerID="94c586e92b8a0fe67e99c170a7a7d9357e243b984db8793c2398bc9d2a812072"
Sep 04 22:07:02.328524 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.327773    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f61a89f-9de9-4610-838a-1967647f8139-plugins-config-volume\") pod \"7f61a89f-9de9-4610-838a-1967647f8139\" (UID: \"7f61a89f-9de9-4610-838a-1967647f8139\") "
Sep 04 22:07:02.328524 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.327816    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ht585\" (UniqueName: \"kubernetes.io/projected/fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1-kube-api-access-ht585\") pod \"fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1\" (UID: \"fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1\") "
Sep 04 22:07:02.328524 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.327870    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkptp\" (UniqueName: \"kubernetes.io/projected/7f61a89f-9de9-4610-838a-1967647f8139-kube-api-access-tkptp\") pod \"7f61a89f-9de9-4610-838a-1967647f8139\" (UID: \"7f61a89f-9de9-4610-838a-1967647f8139\") "
Sep 04 22:07:02.328524 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.327943    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1-plugins-config-volume\") pod \"fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1\" (UID: \"fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1\") "
Sep 04 22:07:02.328931 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.328904    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f61a89f-9de9-4610-838a-1967647f8139-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7f61a89f-9de9-4610-838a-1967647f8139" (UID: "7f61a89f-9de9-4610-838a-1967647f8139"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:07:02.329453 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.329385    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1" (UID: "fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:07:02.331662 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.331625    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1-kube-api-access-ht585" (OuterVolumeSpecName: "kube-api-access-ht585") pod "fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1" (UID: "fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1"). InnerVolumeSpecName "kube-api-access-ht585". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:07:02.331863 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.331839    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f61a89f-9de9-4610-838a-1967647f8139-kube-api-access-tkptp" (OuterVolumeSpecName: "kube-api-access-tkptp") pod "7f61a89f-9de9-4610-838a-1967647f8139" (UID: "7f61a89f-9de9-4610-838a-1967647f8139"). InnerVolumeSpecName "kube-api-access-tkptp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:07:02.339678 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.339644    2557 scope.go:117] "RemoveContainer" containerID="94c586e92b8a0fe67e99c170a7a7d9357e243b984db8793c2398bc9d2a812072"
Sep 04 22:07:02.340129 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:07:02.340104    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94c586e92b8a0fe67e99c170a7a7d9357e243b984db8793c2398bc9d2a812072\": container with ID starting with 94c586e92b8a0fe67e99c170a7a7d9357e243b984db8793c2398bc9d2a812072 not found: ID does not exist" containerID="94c586e92b8a0fe67e99c170a7a7d9357e243b984db8793c2398bc9d2a812072"
Sep 04 22:07:02.340241 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.340136    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94c586e92b8a0fe67e99c170a7a7d9357e243b984db8793c2398bc9d2a812072"} err="failed to get container status \"94c586e92b8a0fe67e99c170a7a7d9357e243b984db8793c2398bc9d2a812072\": rpc error: code = NotFound desc = could not find container \"94c586e92b8a0fe67e99c170a7a7d9357e243b984db8793c2398bc9d2a812072\": container with ID starting with 94c586e92b8a0fe67e99c170a7a7d9357e243b984db8793c2398bc9d2a812072 not found: ID does not exist"
Sep 04 22:07:02.429573 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.429531    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7f61a89f-9de9-4610-838a-1967647f8139-plugins-config-volume\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:07:02.429573 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.429570    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ht585\" (UniqueName: \"kubernetes.io/projected/fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1-kube-api-access-ht585\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:07:02.429782 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.429585    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkptp\" (UniqueName: \"kubernetes.io/projected/7f61a89f-9de9-4610-838a-1967647f8139-kube-api-access-tkptp\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:07:02.429782 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.429596    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1-plugins-config-volume\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:07:02.629190 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.628918    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv"]
Sep 04 22:07:02.631290 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.631262    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-6fda89ac-6899cf7fd6-h6tfv"]
Sep 04 22:07:02.643468 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.643432    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59"]
Sep 04 22:07:02.646684 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:02.646648    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6fda89ac-6ccbc895f8-kkq59"]
Sep 04 22:07:04.352434 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:04.352390    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f61a89f-9de9-4610-838a-1967647f8139" path="/var/lib/kubelet/pods/7f61a89f-9de9-4610-838a-1967647f8139/volumes"
Sep 04 22:07:04.352954 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:04.352932    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1" path="/var/lib/kubelet/pods/fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1/volumes"
Sep 04 22:07:04.355778 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:04.355751    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:07:04.355936 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:04.355797    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:07:04.368740 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:04.368711    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:07:05.342302 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:05.342268    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:07:07.368172 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.368130    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp"]
Sep 04 22:07:07.368630 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.368523    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f61a89f-9de9-4610-838a-1967647f8139" containerName="payload-processing"
Sep 04 22:07:07.368630 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.368538    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f61a89f-9de9-4610-838a-1967647f8139" containerName="payload-processing"
Sep 04 22:07:07.368630 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.368552    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1" containerName="payload-pre-processing"
Sep 04 22:07:07.368630 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.368558    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1" containerName="payload-pre-processing"
Sep 04 22:07:07.368630 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.368624    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe35d4b6-cc8f-4f12-9453-ed7bacc3e0e1" containerName="payload-pre-processing"
Sep 04 22:07:07.368831 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.368637    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f61a89f-9de9-4610-838a-1967647f8139" containerName="payload-processing"
Sep 04 22:07:07.373464 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.373441    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp"
Sep 04 22:07:07.376497 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.376469    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-a43b7e35\""
Sep 04 22:07:07.376647 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.376577    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-a43b7e35-dockercfg-96t82\""
Sep 04 22:07:07.382640 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.382611    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp"]
Sep 04 22:07:07.388362 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.388330    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg"]
Sep 04 22:07:07.391726 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.391701    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg"
Sep 04 22:07:07.405177 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.405148    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg"]
Sep 04 22:07:07.476807 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.476771    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5r7d\" (UniqueName: \"kubernetes.io/projected/aff77438-c850-49cc-995e-a3632d0c19f9-kube-api-access-w5r7d\") pod \"payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp\" (UID: \"aff77438-c850-49cc-995e-a3632d0c19f9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp"
Sep 04 22:07:07.476988 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.476841    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da3f6085-ad68-4d5d-9e33-2f511d1d451d-plugins-config-volume\") pod \"payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg\" (UID: \"da3f6085-ad68-4d5d-9e33-2f511d1d451d\") " pod="openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg"
Sep 04 22:07:07.476988 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.476903    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aff77438-c850-49cc-995e-a3632d0c19f9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp\" (UID: \"aff77438-c850-49cc-995e-a3632d0c19f9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp"
Sep 04 22:07:07.476988 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.476962    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4jxr\" (UniqueName: \"kubernetes.io/projected/da3f6085-ad68-4d5d-9e33-2f511d1d451d-kube-api-access-g4jxr\") pod \"payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg\" (UID: \"da3f6085-ad68-4d5d-9e33-2f511d1d451d\") " pod="openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg"
Sep 04 22:07:07.577815 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.577767    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4jxr\" (UniqueName: \"kubernetes.io/projected/da3f6085-ad68-4d5d-9e33-2f511d1d451d-kube-api-access-g4jxr\") pod \"payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg\" (UID: \"da3f6085-ad68-4d5d-9e33-2f511d1d451d\") " pod="openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg"
Sep 04 22:07:07.577994 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.577829    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5r7d\" (UniqueName: \"kubernetes.io/projected/aff77438-c850-49cc-995e-a3632d0c19f9-kube-api-access-w5r7d\") pod \"payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp\" (UID: \"aff77438-c850-49cc-995e-a3632d0c19f9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp"
Sep 04 22:07:07.577994 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.577864    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da3f6085-ad68-4d5d-9e33-2f511d1d451d-plugins-config-volume\") pod \"payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg\" (UID: \"da3f6085-ad68-4d5d-9e33-2f511d1d451d\") " pod="openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg"
Sep 04 22:07:07.577994 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.577908    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aff77438-c850-49cc-995e-a3632d0c19f9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp\" (UID: \"aff77438-c850-49cc-995e-a3632d0c19f9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp"
Sep 04 22:07:07.578883 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.578858    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aff77438-c850-49cc-995e-a3632d0c19f9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp\" (UID: \"aff77438-c850-49cc-995e-a3632d0c19f9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp"
Sep 04 22:07:07.578935 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.578858    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da3f6085-ad68-4d5d-9e33-2f511d1d451d-plugins-config-volume\") pod \"payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg\" (UID: \"da3f6085-ad68-4d5d-9e33-2f511d1d451d\") " pod="openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg"
Sep 04 22:07:07.586324 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.586293    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4jxr\" (UniqueName: \"kubernetes.io/projected/da3f6085-ad68-4d5d-9e33-2f511d1d451d-kube-api-access-g4jxr\") pod \"payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg\" (UID: \"da3f6085-ad68-4d5d-9e33-2f511d1d451d\") " pod="openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg"
Sep 04 22:07:07.586524 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.586335    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5r7d\" (UniqueName: \"kubernetes.io/projected/aff77438-c850-49cc-995e-a3632d0c19f9-kube-api-access-w5r7d\") pod \"payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp\" (UID: \"aff77438-c850-49cc-995e-a3632d0c19f9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp"
Sep 04 22:07:07.684949 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.684833    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp"
Sep 04 22:07:07.704828 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.704794    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg"
Sep 04 22:07:07.966539 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.966508    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp"]
Sep 04 22:07:07.968230 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:07:07.968201    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaff77438_c850_49cc_995e_a3632d0c19f9.slice/crio-d009a8acbbe8d262ec957571d32968d21e419fcfa6bba7ef885554815ceb8f84 WatchSource:0}: Error finding container d009a8acbbe8d262ec957571d32968d21e419fcfa6bba7ef885554815ceb8f84: Status 404 returned error can't find the container with id d009a8acbbe8d262ec957571d32968d21e419fcfa6bba7ef885554815ceb8f84
Sep 04 22:07:07.995782 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:07.995755    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg"]
Sep 04 22:07:08.001687 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:07:08.001644    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda3f6085_ad68_4d5d_9e33_2f511d1d451d.slice/crio-45998082af0b4f61e146b75e1b1def9f64aec7ef603be7f4008c6b9bdbd04652 WatchSource:0}: Error finding container 45998082af0b4f61e146b75e1b1def9f64aec7ef603be7f4008c6b9bdbd04652: Status 404 returned error can't find the container with id 45998082af0b4f61e146b75e1b1def9f64aec7ef603be7f4008c6b9bdbd04652
Sep 04 22:07:08.353288 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:08.353219    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg"
Sep 04 22:07:08.353288 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:08.353266    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg" event={"ID":"da3f6085-ad68-4d5d-9e33-2f511d1d451d","Type":"ContainerStarted","Data":"f8e1080beb637cc4ea4d371395227cb80ceebc157f53e0f9e128725cf90518fe"}
Sep 04 22:07:08.353491 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:08.353290    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg" event={"ID":"da3f6085-ad68-4d5d-9e33-2f511d1d451d","Type":"ContainerStarted","Data":"45998082af0b4f61e146b75e1b1def9f64aec7ef603be7f4008c6b9bdbd04652"}
Sep 04 22:07:08.353491 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:08.353306    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp" event={"ID":"aff77438-c850-49cc-995e-a3632d0c19f9","Type":"ContainerStarted","Data":"61b76c68742f6b9d9404405f277769efa148c9ddb7298d962f3afd79270ccd9a"}
Sep 04 22:07:08.353491 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:08.353323    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp"
Sep 04 22:07:08.353491 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:08.353338    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp" event={"ID":"aff77438-c850-49cc-995e-a3632d0c19f9","Type":"ContainerStarted","Data":"d009a8acbbe8d262ec957571d32968d21e419fcfa6bba7ef885554815ceb8f84"}
Sep 04 22:07:08.385620 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:08.385551    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg" podStartSLOduration=1.385531229 podStartE2EDuration="1.385531229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:07:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:07:08.368001135 +0000 UTC m=+802.684567873" watchObservedRunningTime="2026-09-04 22:07:08.385531229 +0000 UTC m=+802.702097962"
Sep 04 22:07:08.387480 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:08.387403    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp" podStartSLOduration=1.387392097 podStartE2EDuration="1.387392097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:07:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:07:08.385004225 +0000 UTC m=+802.701570964" watchObservedRunningTime="2026-09-04 22:07:08.387392097 +0000 UTC m=+802.703958835"
Sep 04 22:07:10.260784 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:10.260742    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:07:19.357792 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:19.357759    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg"
Sep 04 22:07:20.363493 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:20.363456    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp"
Sep 04 22:07:27.955283 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:27.955240    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:07:37.241842 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:37.241803    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg"]
Sep 04 22:07:37.242335 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:37.242081    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg" podUID="da3f6085-ad68-4d5d-9e33-2f511d1d451d" containerName="payload-processing" containerID="cri-o://f8e1080beb637cc4ea4d371395227cb80ceebc157f53e0f9e128725cf90518fe" gracePeriod=30
Sep 04 22:07:37.252943 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:37.252910    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp"]
Sep 04 22:07:37.253150 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:37.253128    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp" podUID="aff77438-c850-49cc-995e-a3632d0c19f9" containerName="payload-pre-processing" containerID="cri-o://61b76c68742f6b9d9404405f277769efa148c9ddb7298d962f3afd79270ccd9a" gracePeriod=30
Sep 04 22:07:37.508234 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:37.508195    2557 generic.go:356] "Generic (PLEG): container finished" podID="da3f6085-ad68-4d5d-9e33-2f511d1d451d" containerID="f8e1080beb637cc4ea4d371395227cb80ceebc157f53e0f9e128725cf90518fe" exitCode=0
Sep 04 22:07:37.508397 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:37.508260    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg" event={"ID":"da3f6085-ad68-4d5d-9e33-2f511d1d451d","Type":"ContainerDied","Data":"f8e1080beb637cc4ea4d371395227cb80ceebc157f53e0f9e128725cf90518fe"}
Sep 04 22:07:37.510333 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:37.510262    2557 generic.go:356] "Generic (PLEG): container finished" podID="aff77438-c850-49cc-995e-a3632d0c19f9" containerID="61b76c68742f6b9d9404405f277769efa148c9ddb7298d962f3afd79270ccd9a" exitCode=0
Sep 04 22:07:37.510333 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:37.510317    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp" event={"ID":"aff77438-c850-49cc-995e-a3632d0c19f9","Type":"ContainerDied","Data":"61b76c68742f6b9d9404405f277769efa148c9ddb7298d962f3afd79270ccd9a"}
Sep 04 22:07:37.532896 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:37.532869    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg"
Sep 04 22:07:37.536293 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:37.536266    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp"
Sep 04 22:07:37.564564 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:37.564526    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aff77438-c850-49cc-995e-a3632d0c19f9-plugins-config-volume\") pod \"aff77438-c850-49cc-995e-a3632d0c19f9\" (UID: \"aff77438-c850-49cc-995e-a3632d0c19f9\") "
Sep 04 22:07:37.565136 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:37.564600    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5r7d\" (UniqueName: \"kubernetes.io/projected/aff77438-c850-49cc-995e-a3632d0c19f9-kube-api-access-w5r7d\") pod \"aff77438-c850-49cc-995e-a3632d0c19f9\" (UID: \"aff77438-c850-49cc-995e-a3632d0c19f9\") "
Sep 04 22:07:37.565136 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:37.564624    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g4jxr\" (UniqueName: \"kubernetes.io/projected/da3f6085-ad68-4d5d-9e33-2f511d1d451d-kube-api-access-g4jxr\") pod \"da3f6085-ad68-4d5d-9e33-2f511d1d451d\" (UID: \"da3f6085-ad68-4d5d-9e33-2f511d1d451d\") "
Sep 04 22:07:37.565136 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:37.564659    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da3f6085-ad68-4d5d-9e33-2f511d1d451d-plugins-config-volume\") pod \"da3f6085-ad68-4d5d-9e33-2f511d1d451d\" (UID: \"da3f6085-ad68-4d5d-9e33-2f511d1d451d\") "
Sep 04 22:07:37.565446 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:37.565393    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da3f6085-ad68-4d5d-9e33-2f511d1d451d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "da3f6085-ad68-4d5d-9e33-2f511d1d451d" (UID: "da3f6085-ad68-4d5d-9e33-2f511d1d451d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:07:37.565565 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:37.565520    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aff77438-c850-49cc-995e-a3632d0c19f9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "aff77438-c850-49cc-995e-a3632d0c19f9" (UID: "aff77438-c850-49cc-995e-a3632d0c19f9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:07:37.567698 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:37.567667    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aff77438-c850-49cc-995e-a3632d0c19f9-kube-api-access-w5r7d" (OuterVolumeSpecName: "kube-api-access-w5r7d") pod "aff77438-c850-49cc-995e-a3632d0c19f9" (UID: "aff77438-c850-49cc-995e-a3632d0c19f9"). InnerVolumeSpecName "kube-api-access-w5r7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:07:37.567813 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:37.567783    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da3f6085-ad68-4d5d-9e33-2f511d1d451d-kube-api-access-g4jxr" (OuterVolumeSpecName: "kube-api-access-g4jxr") pod "da3f6085-ad68-4d5d-9e33-2f511d1d451d" (UID: "da3f6085-ad68-4d5d-9e33-2f511d1d451d"). InnerVolumeSpecName "kube-api-access-g4jxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:07:37.665636 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:37.665588    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aff77438-c850-49cc-995e-a3632d0c19f9-plugins-config-volume\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:07:37.665636 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:37.665629    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5r7d\" (UniqueName: \"kubernetes.io/projected/aff77438-c850-49cc-995e-a3632d0c19f9-kube-api-access-w5r7d\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:07:37.665636 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:37.665643    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g4jxr\" (UniqueName: \"kubernetes.io/projected/da3f6085-ad68-4d5d-9e33-2f511d1d451d-kube-api-access-g4jxr\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:07:37.665886 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:37.665656    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da3f6085-ad68-4d5d-9e33-2f511d1d451d-plugins-config-volume\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:07:38.517240 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:38.517137    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg" event={"ID":"da3f6085-ad68-4d5d-9e33-2f511d1d451d","Type":"ContainerDied","Data":"45998082af0b4f61e146b75e1b1def9f64aec7ef603be7f4008c6b9bdbd04652"}
Sep 04 22:07:38.517240 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:38.517161    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg"
Sep 04 22:07:38.517240 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:38.517241    2557 scope.go:117] "RemoveContainer" containerID="f8e1080beb637cc4ea4d371395227cb80ceebc157f53e0f9e128725cf90518fe"
Sep 04 22:07:38.518667 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:38.518626    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp" event={"ID":"aff77438-c850-49cc-995e-a3632d0c19f9","Type":"ContainerDied","Data":"d009a8acbbe8d262ec957571d32968d21e419fcfa6bba7ef885554815ceb8f84"}
Sep 04 22:07:38.518667 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:38.518659    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp"
Sep 04 22:07:38.526385 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:38.526362    2557 scope.go:117] "RemoveContainer" containerID="61b76c68742f6b9d9404405f277769efa148c9ddb7298d962f3afd79270ccd9a"
Sep 04 22:07:38.541794 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:38.541763    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp"]
Sep 04 22:07:38.552768 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:38.552729    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a43b7e35-76d84c6c8b-pshjp"]
Sep 04 22:07:38.564039 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:38.564006    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg"]
Sep 04 22:07:38.573231 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:38.573201    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a43b7e35-6d9dc7799-vhscg"]
Sep 04 22:07:40.353244 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:40.353206    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aff77438-c850-49cc-995e-a3632d0c19f9" path="/var/lib/kubelet/pods/aff77438-c850-49cc-995e-a3632d0c19f9/volumes"
Sep 04 22:07:40.353637 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:40.353604    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da3f6085-ad68-4d5d-9e33-2f511d1d451d" path="/var/lib/kubelet/pods/da3f6085-ad68-4d5d-9e33-2f511d1d451d/volumes"
Sep 04 22:07:40.768263 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:40.768224    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:07:43.782176 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:43.782141    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf"]
Sep 04 22:07:43.784663 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:43.782524    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da3f6085-ad68-4d5d-9e33-2f511d1d451d" containerName="payload-processing"
Sep 04 22:07:43.784663 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:43.782538    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="da3f6085-ad68-4d5d-9e33-2f511d1d451d" containerName="payload-processing"
Sep 04 22:07:43.784663 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:43.782550    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aff77438-c850-49cc-995e-a3632d0c19f9" containerName="payload-pre-processing"
Sep 04 22:07:43.784663 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:43.782555    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="aff77438-c850-49cc-995e-a3632d0c19f9" containerName="payload-pre-processing"
Sep 04 22:07:43.784663 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:43.782616    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="aff77438-c850-49cc-995e-a3632d0c19f9" containerName="payload-pre-processing"
Sep 04 22:07:43.784663 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:43.782626    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="da3f6085-ad68-4d5d-9e33-2f511d1d451d" containerName="payload-processing"
Sep 04 22:07:43.786073 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:43.786055    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf"
Sep 04 22:07:43.790745 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:43.790715    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-5d398f01-dockercfg-t2twx\""
Sep 04 22:07:43.791044 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:43.791028    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-5d398f01\""
Sep 04 22:07:43.808200 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:43.808167    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf"]
Sep 04 22:07:43.824708 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:43.824669    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzz5s\" (UniqueName: \"kubernetes.io/projected/83a27515-a907-4f17-9d0c-5d844152b305-kube-api-access-fzz5s\") pod \"payload-processing-e2e-derive-5d398f01-787478fd89-plszf\" (UID: \"83a27515-a907-4f17-9d0c-5d844152b305\") " pod="openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf"
Sep 04 22:07:43.824869 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:43.824778    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83a27515-a907-4f17-9d0c-5d844152b305-plugins-config-volume\") pod \"payload-processing-e2e-derive-5d398f01-787478fd89-plszf\" (UID: \"83a27515-a907-4f17-9d0c-5d844152b305\") " pod="openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf"
Sep 04 22:07:43.832329 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:43.832297    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd"]
Sep 04 22:07:43.835875 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:43.835851    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd"
Sep 04 22:07:43.852778 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:43.852742    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd"]
Sep 04 22:07:43.926041 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:43.925995    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzz5s\" (UniqueName: \"kubernetes.io/projected/83a27515-a907-4f17-9d0c-5d844152b305-kube-api-access-fzz5s\") pod \"payload-processing-e2e-derive-5d398f01-787478fd89-plszf\" (UID: \"83a27515-a907-4f17-9d0c-5d844152b305\") " pod="openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf"
Sep 04 22:07:43.926250 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:43.926062    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54b527ca-2eb0-4d37-b31a-ebb5c6d7d742-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd\" (UID: \"54b527ca-2eb0-4d37-b31a-ebb5c6d7d742\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd"
Sep 04 22:07:43.926250 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:43.926115    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83a27515-a907-4f17-9d0c-5d844152b305-plugins-config-volume\") pod \"payload-processing-e2e-derive-5d398f01-787478fd89-plszf\" (UID: \"83a27515-a907-4f17-9d0c-5d844152b305\") " pod="openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf"
Sep 04 22:07:43.926250 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:43.926221    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw7zn\" (UniqueName: \"kubernetes.io/projected/54b527ca-2eb0-4d37-b31a-ebb5c6d7d742-kube-api-access-jw7zn\") pod \"payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd\" (UID: \"54b527ca-2eb0-4d37-b31a-ebb5c6d7d742\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd"
Sep 04 22:07:43.926887 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:43.926863    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83a27515-a907-4f17-9d0c-5d844152b305-plugins-config-volume\") pod \"payload-processing-e2e-derive-5d398f01-787478fd89-plszf\" (UID: \"83a27515-a907-4f17-9d0c-5d844152b305\") " pod="openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf"
Sep 04 22:07:43.935293 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:43.935264    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzz5s\" (UniqueName: \"kubernetes.io/projected/83a27515-a907-4f17-9d0c-5d844152b305-kube-api-access-fzz5s\") pod \"payload-processing-e2e-derive-5d398f01-787478fd89-plszf\" (UID: \"83a27515-a907-4f17-9d0c-5d844152b305\") " pod="openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf"
Sep 04 22:07:44.027052 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:44.027003    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54b527ca-2eb0-4d37-b31a-ebb5c6d7d742-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd\" (UID: \"54b527ca-2eb0-4d37-b31a-ebb5c6d7d742\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd"
Sep 04 22:07:44.027233 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:44.027102    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw7zn\" (UniqueName: \"kubernetes.io/projected/54b527ca-2eb0-4d37-b31a-ebb5c6d7d742-kube-api-access-jw7zn\") pod \"payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd\" (UID: \"54b527ca-2eb0-4d37-b31a-ebb5c6d7d742\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd"
Sep 04 22:07:44.027777 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:44.027755    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54b527ca-2eb0-4d37-b31a-ebb5c6d7d742-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd\" (UID: \"54b527ca-2eb0-4d37-b31a-ebb5c6d7d742\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd"
Sep 04 22:07:44.035840 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:44.035775    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw7zn\" (UniqueName: \"kubernetes.io/projected/54b527ca-2eb0-4d37-b31a-ebb5c6d7d742-kube-api-access-jw7zn\") pod \"payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd\" (UID: \"54b527ca-2eb0-4d37-b31a-ebb5c6d7d742\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd"
Sep 04 22:07:44.096681 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:44.096637    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf"
Sep 04 22:07:44.147965 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:44.147688    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd"
Sep 04 22:07:44.371537 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:44.371502    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf"]
Sep 04 22:07:44.372924 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:07:44.372894    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83a27515_a907_4f17_9d0c_5d844152b305.slice/crio-3ae12203b8b4ddcc51b4d46eedcb7379fe2c3d3b0030af9a78383581327404f3 WatchSource:0}: Error finding container 3ae12203b8b4ddcc51b4d46eedcb7379fe2c3d3b0030af9a78383581327404f3: Status 404 returned error can't find the container with id 3ae12203b8b4ddcc51b4d46eedcb7379fe2c3d3b0030af9a78383581327404f3
Sep 04 22:07:44.434767 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:44.434737    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd"]
Sep 04 22:07:44.436298 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:07:44.436272    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54b527ca_2eb0_4d37_b31a_ebb5c6d7d742.slice/crio-9b2c10d94f9569125e662b5210d7beb2016c487eeecc552c3b14f8ec60f547db WatchSource:0}: Error finding container 9b2c10d94f9569125e662b5210d7beb2016c487eeecc552c3b14f8ec60f547db: Status 404 returned error can't find the container with id 9b2c10d94f9569125e662b5210d7beb2016c487eeecc552c3b14f8ec60f547db
Sep 04 22:07:44.553358 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:44.553320    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd" event={"ID":"54b527ca-2eb0-4d37-b31a-ebb5c6d7d742","Type":"ContainerStarted","Data":"b822c310edf410992b9874850ef298567bc160ac0ce6f428cb0fa5afded762af"}
Sep 04 22:07:44.553571 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:44.553374    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd" event={"ID":"54b527ca-2eb0-4d37-b31a-ebb5c6d7d742","Type":"ContainerStarted","Data":"9b2c10d94f9569125e662b5210d7beb2016c487eeecc552c3b14f8ec60f547db"}
Sep 04 22:07:44.553571 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:44.553460    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd"
Sep 04 22:07:44.555229 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:44.555195    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf" event={"ID":"83a27515-a907-4f17-9d0c-5d844152b305","Type":"ContainerStarted","Data":"1cd0fef0c3b9cb5801f61c2e3e6aef87d42eb8397bc038db0c884a26d3ffb880"}
Sep 04 22:07:44.555363 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:44.555257    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf"
Sep 04 22:07:44.555363 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:44.555281    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf" event={"ID":"83a27515-a907-4f17-9d0c-5d844152b305","Type":"ContainerStarted","Data":"3ae12203b8b4ddcc51b4d46eedcb7379fe2c3d3b0030af9a78383581327404f3"}
Sep 04 22:07:44.571367 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:44.571308    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd" podStartSLOduration=1.571285351 podStartE2EDuration="1.571285351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:07:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:07:44.569355665 +0000 UTC m=+838.885922404" watchObservedRunningTime="2026-09-04 22:07:44.571285351 +0000 UTC m=+838.887852091"
Sep 04 22:07:44.586015 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:44.585953    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf" podStartSLOduration=1.585930579 podStartE2EDuration="1.585930579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:07:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:07:44.585718073 +0000 UTC m=+838.902284811" watchObservedRunningTime="2026-09-04 22:07:44.585930579 +0000 UTC m=+838.902497322"
Sep 04 22:07:53.751627 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:53.751587    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:07:55.562124 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:55.562089    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd"
Sep 04 22:07:55.562514 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:07:55.562167    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf"
Sep 04 22:08:13.705015 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:13.704973    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf"]
Sep 04 22:08:13.705499 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:13.705241    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf" podUID="83a27515-a907-4f17-9d0c-5d844152b305" containerName="payload-processing" containerID="cri-o://1cd0fef0c3b9cb5801f61c2e3e6aef87d42eb8397bc038db0c884a26d3ffb880" gracePeriod=30
Sep 04 22:08:13.716739 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:13.716705    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd"]
Sep 04 22:08:13.716982 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:13.716956    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd" podUID="54b527ca-2eb0-4d37-b31a-ebb5c6d7d742" containerName="payload-pre-processing" containerID="cri-o://b822c310edf410992b9874850ef298567bc160ac0ce6f428cb0fa5afded762af" gracePeriod=30
Sep 04 22:08:13.970314 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:13.970241    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf"
Sep 04 22:08:13.999088 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:13.999051    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd"
Sep 04 22:08:14.125186 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.125149    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54b527ca-2eb0-4d37-b31a-ebb5c6d7d742-plugins-config-volume\") pod \"54b527ca-2eb0-4d37-b31a-ebb5c6d7d742\" (UID: \"54b527ca-2eb0-4d37-b31a-ebb5c6d7d742\") "
Sep 04 22:08:14.125186 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.125200    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jw7zn\" (UniqueName: \"kubernetes.io/projected/54b527ca-2eb0-4d37-b31a-ebb5c6d7d742-kube-api-access-jw7zn\") pod \"54b527ca-2eb0-4d37-b31a-ebb5c6d7d742\" (UID: \"54b527ca-2eb0-4d37-b31a-ebb5c6d7d742\") "
Sep 04 22:08:14.125483 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.125224    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzz5s\" (UniqueName: \"kubernetes.io/projected/83a27515-a907-4f17-9d0c-5d844152b305-kube-api-access-fzz5s\") pod \"83a27515-a907-4f17-9d0c-5d844152b305\" (UID: \"83a27515-a907-4f17-9d0c-5d844152b305\") "
Sep 04 22:08:14.125483 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.125319    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83a27515-a907-4f17-9d0c-5d844152b305-plugins-config-volume\") pod \"83a27515-a907-4f17-9d0c-5d844152b305\" (UID: \"83a27515-a907-4f17-9d0c-5d844152b305\") "
Sep 04 22:08:14.125726 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.125596    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54b527ca-2eb0-4d37-b31a-ebb5c6d7d742-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "54b527ca-2eb0-4d37-b31a-ebb5c6d7d742" (UID: "54b527ca-2eb0-4d37-b31a-ebb5c6d7d742"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:08:14.125726 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.125668    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54b527ca-2eb0-4d37-b31a-ebb5c6d7d742-plugins-config-volume\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:08:14.125834 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.125747    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/83a27515-a907-4f17-9d0c-5d844152b305-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "83a27515-a907-4f17-9d0c-5d844152b305" (UID: "83a27515-a907-4f17-9d0c-5d844152b305"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:08:14.127491 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.127459    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83a27515-a907-4f17-9d0c-5d844152b305-kube-api-access-fzz5s" (OuterVolumeSpecName: "kube-api-access-fzz5s") pod "83a27515-a907-4f17-9d0c-5d844152b305" (UID: "83a27515-a907-4f17-9d0c-5d844152b305"). InnerVolumeSpecName "kube-api-access-fzz5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:08:14.127552 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.127504    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54b527ca-2eb0-4d37-b31a-ebb5c6d7d742-kube-api-access-jw7zn" (OuterVolumeSpecName: "kube-api-access-jw7zn") pod "54b527ca-2eb0-4d37-b31a-ebb5c6d7d742" (UID: "54b527ca-2eb0-4d37-b31a-ebb5c6d7d742"). InnerVolumeSpecName "kube-api-access-jw7zn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:08:14.226316 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.226220    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/83a27515-a907-4f17-9d0c-5d844152b305-plugins-config-volume\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:08:14.226316 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.226252    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jw7zn\" (UniqueName: \"kubernetes.io/projected/54b527ca-2eb0-4d37-b31a-ebb5c6d7d742-kube-api-access-jw7zn\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:08:14.226316 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.226262    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzz5s\" (UniqueName: \"kubernetes.io/projected/83a27515-a907-4f17-9d0c-5d844152b305-kube-api-access-fzz5s\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:08:14.727227 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.727192    2557 generic.go:356] "Generic (PLEG): container finished" podID="83a27515-a907-4f17-9d0c-5d844152b305" containerID="1cd0fef0c3b9cb5801f61c2e3e6aef87d42eb8397bc038db0c884a26d3ffb880" exitCode=0
Sep 04 22:08:14.727740 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.727234    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf"
Sep 04 22:08:14.727740 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.727274    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf" event={"ID":"83a27515-a907-4f17-9d0c-5d844152b305","Type":"ContainerDied","Data":"1cd0fef0c3b9cb5801f61c2e3e6aef87d42eb8397bc038db0c884a26d3ffb880"}
Sep 04 22:08:14.727740 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.727326    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf" event={"ID":"83a27515-a907-4f17-9d0c-5d844152b305","Type":"ContainerDied","Data":"3ae12203b8b4ddcc51b4d46eedcb7379fe2c3d3b0030af9a78383581327404f3"}
Sep 04 22:08:14.727740 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.727351    2557 scope.go:117] "RemoveContainer" containerID="1cd0fef0c3b9cb5801f61c2e3e6aef87d42eb8397bc038db0c884a26d3ffb880"
Sep 04 22:08:14.729147 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.729106    2557 generic.go:356] "Generic (PLEG): container finished" podID="54b527ca-2eb0-4d37-b31a-ebb5c6d7d742" containerID="b822c310edf410992b9874850ef298567bc160ac0ce6f428cb0fa5afded762af" exitCode=0
Sep 04 22:08:14.729291 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.729237    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd"
Sep 04 22:08:14.729457 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.729242    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd" event={"ID":"54b527ca-2eb0-4d37-b31a-ebb5c6d7d742","Type":"ContainerDied","Data":"b822c310edf410992b9874850ef298567bc160ac0ce6f428cb0fa5afded762af"}
Sep 04 22:08:14.729546 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.729522    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd" event={"ID":"54b527ca-2eb0-4d37-b31a-ebb5c6d7d742","Type":"ContainerDied","Data":"9b2c10d94f9569125e662b5210d7beb2016c487eeecc552c3b14f8ec60f547db"}
Sep 04 22:08:14.737160 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.737137    2557 scope.go:117] "RemoveContainer" containerID="1cd0fef0c3b9cb5801f61c2e3e6aef87d42eb8397bc038db0c884a26d3ffb880"
Sep 04 22:08:14.737500 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:08:14.737482    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cd0fef0c3b9cb5801f61c2e3e6aef87d42eb8397bc038db0c884a26d3ffb880\": container with ID starting with 1cd0fef0c3b9cb5801f61c2e3e6aef87d42eb8397bc038db0c884a26d3ffb880 not found: ID does not exist" containerID="1cd0fef0c3b9cb5801f61c2e3e6aef87d42eb8397bc038db0c884a26d3ffb880"
Sep 04 22:08:14.737558 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.737510    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cd0fef0c3b9cb5801f61c2e3e6aef87d42eb8397bc038db0c884a26d3ffb880"} err="failed to get container status \"1cd0fef0c3b9cb5801f61c2e3e6aef87d42eb8397bc038db0c884a26d3ffb880\": rpc error: code = NotFound desc = could not find container \"1cd0fef0c3b9cb5801f61c2e3e6aef87d42eb8397bc038db0c884a26d3ffb880\": container with ID starting with 1cd0fef0c3b9cb5801f61c2e3e6aef87d42eb8397bc038db0c884a26d3ffb880 not found: ID does not exist"
Sep 04 22:08:14.737558 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.737526    2557 scope.go:117] "RemoveContainer" containerID="b822c310edf410992b9874850ef298567bc160ac0ce6f428cb0fa5afded762af"
Sep 04 22:08:14.747103 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.747075    2557 scope.go:117] "RemoveContainer" containerID="b822c310edf410992b9874850ef298567bc160ac0ce6f428cb0fa5afded762af"
Sep 04 22:08:14.747448 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:08:14.747424    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b822c310edf410992b9874850ef298567bc160ac0ce6f428cb0fa5afded762af\": container with ID starting with b822c310edf410992b9874850ef298567bc160ac0ce6f428cb0fa5afded762af not found: ID does not exist" containerID="b822c310edf410992b9874850ef298567bc160ac0ce6f428cb0fa5afded762af"
Sep 04 22:08:14.747510 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.747458    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b822c310edf410992b9874850ef298567bc160ac0ce6f428cb0fa5afded762af"} err="failed to get container status \"b822c310edf410992b9874850ef298567bc160ac0ce6f428cb0fa5afded762af\": rpc error: code = NotFound desc = could not find container \"b822c310edf410992b9874850ef298567bc160ac0ce6f428cb0fa5afded762af\": container with ID starting with b822c310edf410992b9874850ef298567bc160ac0ce6f428cb0fa5afded762af not found: ID does not exist"
Sep 04 22:08:14.759179 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.759143    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf"]
Sep 04 22:08:14.767074 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.767044    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-5d398f01-787478fd89-plszf"]
Sep 04 22:08:14.804435 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.802383    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd"]
Sep 04 22:08:14.810345 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:14.810307    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-5d398f01-7475ddf6db-mpghd"]
Sep 04 22:08:16.352166 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:16.352120    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54b527ca-2eb0-4d37-b31a-ebb5c6d7d742" path="/var/lib/kubelet/pods/54b527ca-2eb0-4d37-b31a-ebb5c6d7d742/volumes"
Sep 04 22:08:16.352709 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:16.352685    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83a27515-a907-4f17-9d0c-5d844152b305" path="/var/lib/kubelet/pods/83a27515-a907-4f17-9d0c-5d844152b305/volumes"
Sep 04 22:08:18.166498 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:18.166454    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:08:52.093696 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:52.093656    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"]
Sep 04 22:08:54.556504 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:08:54.556464    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:09:05.965432 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:05.965387    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:09:07.113333 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:07.113282    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8" podUID="c316e698-4d33-47ac-9494-fa22b213a8d6" containerName="main" containerID="cri-o://8c815138ecd5da7e325fea4c689e09af7e32eb7ab93d536add2988f464c8aee7" gracePeriod=45
Sep 04 22:09:07.374395 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:07.374320    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:09:07.403684 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:07.403647    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-model-cache\") pod \"c316e698-4d33-47ac-9494-fa22b213a8d6\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") "
Sep 04 22:09:07.403873 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:07.403699    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k876k\" (UniqueName: \"kubernetes.io/projected/c316e698-4d33-47ac-9494-fa22b213a8d6-kube-api-access-k876k\") pod \"c316e698-4d33-47ac-9494-fa22b213a8d6\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") "
Sep 04 22:09:07.403873 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:07.403731    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-tmp-dir\") pod \"c316e698-4d33-47ac-9494-fa22b213a8d6\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") "
Sep 04 22:09:07.403873 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:07.403750    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-home\") pod \"c316e698-4d33-47ac-9494-fa22b213a8d6\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") "
Sep 04 22:09:07.403873 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:07.403780    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-dshm\") pod \"c316e698-4d33-47ac-9494-fa22b213a8d6\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") "
Sep 04 22:09:07.403873 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:07.403817    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c316e698-4d33-47ac-9494-fa22b213a8d6-tls-certs\") pod \"c316e698-4d33-47ac-9494-fa22b213a8d6\" (UID: \"c316e698-4d33-47ac-9494-fa22b213a8d6\") "
Sep 04 22:09:07.404426 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:07.404008    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c316e698-4d33-47ac-9494-fa22b213a8d6" (UID: "c316e698-4d33-47ac-9494-fa22b213a8d6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:09:07.404426 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:07.404022    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-model-cache" (OuterVolumeSpecName: "model-cache") pod "c316e698-4d33-47ac-9494-fa22b213a8d6" (UID: "c316e698-4d33-47ac-9494-fa22b213a8d6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:09:07.404426 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:07.404096    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-home" (OuterVolumeSpecName: "home") pod "c316e698-4d33-47ac-9494-fa22b213a8d6" (UID: "c316e698-4d33-47ac-9494-fa22b213a8d6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:09:07.404426 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:07.404185    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-model-cache\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:09:07.404426 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:07.404203    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-tmp-dir\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:09:07.404426 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:07.404215    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-home\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:09:07.406106 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:07.406071    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c316e698-4d33-47ac-9494-fa22b213a8d6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c316e698-4d33-47ac-9494-fa22b213a8d6" (UID: "c316e698-4d33-47ac-9494-fa22b213a8d6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:09:07.406211 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:07.406079    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-dshm" (OuterVolumeSpecName: "dshm") pod "c316e698-4d33-47ac-9494-fa22b213a8d6" (UID: "c316e698-4d33-47ac-9494-fa22b213a8d6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:09:07.406211 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:07.406131    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c316e698-4d33-47ac-9494-fa22b213a8d6-kube-api-access-k876k" (OuterVolumeSpecName: "kube-api-access-k876k") pod "c316e698-4d33-47ac-9494-fa22b213a8d6" (UID: "c316e698-4d33-47ac-9494-fa22b213a8d6"). InnerVolumeSpecName "kube-api-access-k876k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:09:07.505484 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:07.505447    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c316e698-4d33-47ac-9494-fa22b213a8d6-tls-certs\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:09:07.505484 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:07.505480    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k876k\" (UniqueName: \"kubernetes.io/projected/c316e698-4d33-47ac-9494-fa22b213a8d6-kube-api-access-k876k\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:09:07.505484 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:07.505490    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c316e698-4d33-47ac-9494-fa22b213a8d6-dshm\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:09:08.019273 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:08.019237    2557 generic.go:356] "Generic (PLEG): container finished" podID="c316e698-4d33-47ac-9494-fa22b213a8d6" containerID="8c815138ecd5da7e325fea4c689e09af7e32eb7ab93d536add2988f464c8aee7" exitCode=0
Sep 04 22:09:08.019493 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:08.019289    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"
Sep 04 22:09:08.019493 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:08.019290    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8" event={"ID":"c316e698-4d33-47ac-9494-fa22b213a8d6","Type":"ContainerDied","Data":"8c815138ecd5da7e325fea4c689e09af7e32eb7ab93d536add2988f464c8aee7"}
Sep 04 22:09:08.019493 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:08.019406    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8" event={"ID":"c316e698-4d33-47ac-9494-fa22b213a8d6","Type":"ContainerDied","Data":"e73368ddbcb1a1d251d9292aa8cd2f4f432e435d0ebcba05129fd9681a9313ac"}
Sep 04 22:09:08.019493 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:08.019452    2557 scope.go:117] "RemoveContainer" containerID="8c815138ecd5da7e325fea4c689e09af7e32eb7ab93d536add2988f464c8aee7"
Sep 04 22:09:08.029188 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:08.029166    2557 scope.go:117] "RemoveContainer" containerID="8c815138ecd5da7e325fea4c689e09af7e32eb7ab93d536add2988f464c8aee7"
Sep 04 22:09:08.029552 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:09:08.029529    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c815138ecd5da7e325fea4c689e09af7e32eb7ab93d536add2988f464c8aee7\": container with ID starting with 8c815138ecd5da7e325fea4c689e09af7e32eb7ab93d536add2988f464c8aee7 not found: ID does not exist" containerID="8c815138ecd5da7e325fea4c689e09af7e32eb7ab93d536add2988f464c8aee7"
Sep 04 22:09:08.029625 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:08.029560    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c815138ecd5da7e325fea4c689e09af7e32eb7ab93d536add2988f464c8aee7"} err="failed to get container status \"8c815138ecd5da7e325fea4c689e09af7e32eb7ab93d536add2988f464c8aee7\": rpc error: code = NotFound desc = could not find container \"8c815138ecd5da7e325fea4c689e09af7e32eb7ab93d536add2988f464c8aee7\": container with ID starting with 8c815138ecd5da7e325fea4c689e09af7e32eb7ab93d536add2988f464c8aee7 not found: ID does not exist"
Sep 04 22:09:08.040408 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:08.040372    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"]
Sep 04 22:09:08.042564 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:08.042534    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-a5e1c2/test-model-a5e1c2-kserve-598bfc6875-bcmn8"]
Sep 04 22:09:08.352939 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:08.352837    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c316e698-4d33-47ac-9494-fa22b213a8d6" path="/var/lib/kubelet/pods/c316e698-4d33-47ac-9494-fa22b213a8d6/volumes"
Sep 04 22:09:21.480868 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:21.480830    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94"]
Sep 04 22:09:21.481484 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:21.481084    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94" podUID="40e25751-a369-4ffc-93b6-5246cfdf7bb7" containerName="payload-pre-processing" containerID="cri-o://e752c6bc13d8bf37befcbb51ee3ab403342e5b8b8696279dcd57a42220b46941" gracePeriod=30
Sep 04 22:09:21.749875 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:21.749847    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94"
Sep 04 22:09:21.825439 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:21.825369    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2vb7\" (UniqueName: \"kubernetes.io/projected/40e25751-a369-4ffc-93b6-5246cfdf7bb7-kube-api-access-b2vb7\") pod \"40e25751-a369-4ffc-93b6-5246cfdf7bb7\" (UID: \"40e25751-a369-4ffc-93b6-5246cfdf7bb7\") "
Sep 04 22:09:21.825614 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:21.825512    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40e25751-a369-4ffc-93b6-5246cfdf7bb7-plugins-config-volume\") pod \"40e25751-a369-4ffc-93b6-5246cfdf7bb7\" (UID: \"40e25751-a369-4ffc-93b6-5246cfdf7bb7\") "
Sep 04 22:09:21.825914 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:21.825891    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40e25751-a369-4ffc-93b6-5246cfdf7bb7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "40e25751-a369-4ffc-93b6-5246cfdf7bb7" (UID: "40e25751-a369-4ffc-93b6-5246cfdf7bb7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:09:21.827642 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:21.827613    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40e25751-a369-4ffc-93b6-5246cfdf7bb7-kube-api-access-b2vb7" (OuterVolumeSpecName: "kube-api-access-b2vb7") pod "40e25751-a369-4ffc-93b6-5246cfdf7bb7" (UID: "40e25751-a369-4ffc-93b6-5246cfdf7bb7"). InnerVolumeSpecName "kube-api-access-b2vb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:09:21.926162 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:21.926118    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40e25751-a369-4ffc-93b6-5246cfdf7bb7-plugins-config-volume\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:09:21.926162 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:21.926153    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2vb7\" (UniqueName: \"kubernetes.io/projected/40e25751-a369-4ffc-93b6-5246cfdf7bb7-kube-api-access-b2vb7\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:09:22.097497 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:22.097453    2557 generic.go:356] "Generic (PLEG): container finished" podID="40e25751-a369-4ffc-93b6-5246cfdf7bb7" containerID="e752c6bc13d8bf37befcbb51ee3ab403342e5b8b8696279dcd57a42220b46941" exitCode=0
Sep 04 22:09:22.097649 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:22.097508    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94"
Sep 04 22:09:22.097649 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:22.097543    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94" event={"ID":"40e25751-a369-4ffc-93b6-5246cfdf7bb7","Type":"ContainerDied","Data":"e752c6bc13d8bf37befcbb51ee3ab403342e5b8b8696279dcd57a42220b46941"}
Sep 04 22:09:22.097649 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:22.097601    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94" event={"ID":"40e25751-a369-4ffc-93b6-5246cfdf7bb7","Type":"ContainerDied","Data":"4fd0d2f4f0a797f8c728c238ada320a5edc05daf9eae096278274d6afbe81aec"}
Sep 04 22:09:22.097649 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:22.097624    2557 scope.go:117] "RemoveContainer" containerID="e752c6bc13d8bf37befcbb51ee3ab403342e5b8b8696279dcd57a42220b46941"
Sep 04 22:09:22.106803 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:22.106783    2557 scope.go:117] "RemoveContainer" containerID="e752c6bc13d8bf37befcbb51ee3ab403342e5b8b8696279dcd57a42220b46941"
Sep 04 22:09:22.107122 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:09:22.107105    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e752c6bc13d8bf37befcbb51ee3ab403342e5b8b8696279dcd57a42220b46941\": container with ID starting with e752c6bc13d8bf37befcbb51ee3ab403342e5b8b8696279dcd57a42220b46941 not found: ID does not exist" containerID="e752c6bc13d8bf37befcbb51ee3ab403342e5b8b8696279dcd57a42220b46941"
Sep 04 22:09:22.107179 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:22.107130    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e752c6bc13d8bf37befcbb51ee3ab403342e5b8b8696279dcd57a42220b46941"} err="failed to get container status \"e752c6bc13d8bf37befcbb51ee3ab403342e5b8b8696279dcd57a42220b46941\": rpc error: code = NotFound desc = could not find container \"e752c6bc13d8bf37befcbb51ee3ab403342e5b8b8696279dcd57a42220b46941\": container with ID starting with e752c6bc13d8bf37befcbb51ee3ab403342e5b8b8696279dcd57a42220b46941 not found: ID does not exist"
Sep 04 22:09:22.119765 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:22.119727    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94"]
Sep 04 22:09:22.125083 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:22.125048    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-d4ad2d-6b4d78699d-cqf94"]
Sep 04 22:09:22.352170 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:22.352077    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40e25751-a369-4ffc-93b6-5246cfdf7bb7" path="/var/lib/kubelet/pods/40e25751-a369-4ffc-93b6-5246cfdf7bb7/volumes"
Sep 04 22:09:23.853673 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:23.853635    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:09:36.064975 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:36.064931    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:09:44.073359 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:44.073319    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:09:51.293871 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:51.293832    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:09:56.151678 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.151633    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx"]
Sep 04 22:09:56.152161 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.151964    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c316e698-4d33-47ac-9494-fa22b213a8d6" containerName="main"
Sep 04 22:09:56.152161 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.151976    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c316e698-4d33-47ac-9494-fa22b213a8d6" containerName="main"
Sep 04 22:09:56.152161 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.151985    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40e25751-a369-4ffc-93b6-5246cfdf7bb7" containerName="payload-pre-processing"
Sep 04 22:09:56.152161 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.151991    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="40e25751-a369-4ffc-93b6-5246cfdf7bb7" containerName="payload-pre-processing"
Sep 04 22:09:56.152161 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.152013    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83a27515-a907-4f17-9d0c-5d844152b305" containerName="payload-processing"
Sep 04 22:09:56.152161 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.152019    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="83a27515-a907-4f17-9d0c-5d844152b305" containerName="payload-processing"
Sep 04 22:09:56.152161 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.152028    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54b527ca-2eb0-4d37-b31a-ebb5c6d7d742" containerName="payload-pre-processing"
Sep 04 22:09:56.152161 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.152033    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="54b527ca-2eb0-4d37-b31a-ebb5c6d7d742" containerName="payload-pre-processing"
Sep 04 22:09:56.152161 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.152086    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="40e25751-a369-4ffc-93b6-5246cfdf7bb7" containerName="payload-pre-processing"
Sep 04 22:09:56.152161 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.152094    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="83a27515-a907-4f17-9d0c-5d844152b305" containerName="payload-processing"
Sep 04 22:09:56.152161 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.152101    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c316e698-4d33-47ac-9494-fa22b213a8d6" containerName="main"
Sep 04 22:09:56.152161 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.152107    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="54b527ca-2eb0-4d37-b31a-ebb5c6d7d742" containerName="payload-pre-processing"
Sep 04 22:09:56.154926 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.154908    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx"
Sep 04 22:09:56.157132 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.157108    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-806943a9\""
Sep 04 22:09:56.157246 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.157144    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-806943a9-dockercfg-jbdjp\""
Sep 04 22:09:56.165547 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.165522    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx"]
Sep 04 22:09:56.183812 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.183775    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r"]
Sep 04 22:09:56.187094 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.187075    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r"
Sep 04 22:09:56.196847 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.196814    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r"]
Sep 04 22:09:56.242817 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.242771    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd72d1a3-3f66-4c74-ae18-0afcd5761319-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx\" (UID: \"cd72d1a3-3f66-4c74-ae18-0afcd5761319\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx"
Sep 04 22:09:56.242817 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.242816    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz4s8\" (UniqueName: \"kubernetes.io/projected/cd72d1a3-3f66-4c74-ae18-0afcd5761319-kube-api-access-jz4s8\") pod \"payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx\" (UID: \"cd72d1a3-3f66-4c74-ae18-0afcd5761319\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx"
Sep 04 22:09:56.243058 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.242888    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b2a8fd8-31b4-4ed5-b0db-6972341e9cab-plugins-config-volume\") pod \"payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r\" (UID: \"2b2a8fd8-31b4-4ed5-b0db-6972341e9cab\") " pod="openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r"
Sep 04 22:09:56.243058 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.242991    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ppx6\" (UniqueName: \"kubernetes.io/projected/2b2a8fd8-31b4-4ed5-b0db-6972341e9cab-kube-api-access-2ppx6\") pod \"payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r\" (UID: \"2b2a8fd8-31b4-4ed5-b0db-6972341e9cab\") " pod="openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r"
Sep 04 22:09:56.344136 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.344099    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b2a8fd8-31b4-4ed5-b0db-6972341e9cab-plugins-config-volume\") pod \"payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r\" (UID: \"2b2a8fd8-31b4-4ed5-b0db-6972341e9cab\") " pod="openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r"
Sep 04 22:09:56.344356 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.344171    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ppx6\" (UniqueName: \"kubernetes.io/projected/2b2a8fd8-31b4-4ed5-b0db-6972341e9cab-kube-api-access-2ppx6\") pod \"payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r\" (UID: \"2b2a8fd8-31b4-4ed5-b0db-6972341e9cab\") " pod="openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r"
Sep 04 22:09:56.344356 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.344212    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd72d1a3-3f66-4c74-ae18-0afcd5761319-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx\" (UID: \"cd72d1a3-3f66-4c74-ae18-0afcd5761319\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx"
Sep 04 22:09:56.344356 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.344229    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz4s8\" (UniqueName: \"kubernetes.io/projected/cd72d1a3-3f66-4c74-ae18-0afcd5761319-kube-api-access-jz4s8\") pod \"payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx\" (UID: \"cd72d1a3-3f66-4c74-ae18-0afcd5761319\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx"
Sep 04 22:09:56.344879 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.344856    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b2a8fd8-31b4-4ed5-b0db-6972341e9cab-plugins-config-volume\") pod \"payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r\" (UID: \"2b2a8fd8-31b4-4ed5-b0db-6972341e9cab\") " pod="openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r"
Sep 04 22:09:56.344942 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.344922    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd72d1a3-3f66-4c74-ae18-0afcd5761319-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx\" (UID: \"cd72d1a3-3f66-4c74-ae18-0afcd5761319\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx"
Sep 04 22:09:56.352371 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.352339    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ppx6\" (UniqueName: \"kubernetes.io/projected/2b2a8fd8-31b4-4ed5-b0db-6972341e9cab-kube-api-access-2ppx6\") pod \"payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r\" (UID: \"2b2a8fd8-31b4-4ed5-b0db-6972341e9cab\") " pod="openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r"
Sep 04 22:09:56.352824 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.352801    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz4s8\" (UniqueName: \"kubernetes.io/projected/cd72d1a3-3f66-4c74-ae18-0afcd5761319-kube-api-access-jz4s8\") pod \"payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx\" (UID: \"cd72d1a3-3f66-4c74-ae18-0afcd5761319\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx"
Sep 04 22:09:56.465393 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.465303    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx"
Sep 04 22:09:56.497946 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.497911    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r"
Sep 04 22:09:56.745180 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.745150    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx"]
Sep 04 22:09:56.746869 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:09:56.746835    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd72d1a3_3f66_4c74_ae18_0afcd5761319.slice/crio-e18681f65815d71e7beb93b15e6789744c37e809173a2a6463bb22112543c8c8 WatchSource:0}: Error finding container e18681f65815d71e7beb93b15e6789744c37e809173a2a6463bb22112543c8c8: Status 404 returned error can't find the container with id e18681f65815d71e7beb93b15e6789744c37e809173a2a6463bb22112543c8c8
Sep 04 22:09:56.797877 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:56.797844    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r"]
Sep 04 22:09:56.807684 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:09:56.807653    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b2a8fd8_31b4_4ed5_b0db_6972341e9cab.slice/crio-ad88510c3c1e4f69e9cbcb3f1b604324a75d20d7d25ccfca3f7c65c4d135bef7 WatchSource:0}: Error finding container ad88510c3c1e4f69e9cbcb3f1b604324a75d20d7d25ccfca3f7c65c4d135bef7: Status 404 returned error can't find the container with id ad88510c3c1e4f69e9cbcb3f1b604324a75d20d7d25ccfca3f7c65c4d135bef7
Sep 04 22:09:57.285117 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:57.285071    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r" event={"ID":"2b2a8fd8-31b4-4ed5-b0db-6972341e9cab","Type":"ContainerStarted","Data":"314477efb867adc2cb01791c929650eed3efb46070f994d6c33edabe55c53325"}
Sep 04 22:09:57.285622 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:57.285151    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r" event={"ID":"2b2a8fd8-31b4-4ed5-b0db-6972341e9cab","Type":"ContainerStarted","Data":"ad88510c3c1e4f69e9cbcb3f1b604324a75d20d7d25ccfca3f7c65c4d135bef7"}
Sep 04 22:09:57.285622 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:57.285184    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r"
Sep 04 22:09:57.286862 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:57.286831    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx" event={"ID":"cd72d1a3-3f66-4c74-ae18-0afcd5761319","Type":"ContainerStarted","Data":"4a3839afa13b22b17c26d19b592e70330f56b408bc380d5113eec8a19c0a76ac"}
Sep 04 22:09:57.287008 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:57.286876    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx" event={"ID":"cd72d1a3-3f66-4c74-ae18-0afcd5761319","Type":"ContainerStarted","Data":"e18681f65815d71e7beb93b15e6789744c37e809173a2a6463bb22112543c8c8"}
Sep 04 22:09:57.287008 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:57.286942    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx"
Sep 04 22:09:57.302819 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:57.302765    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r" podStartSLOduration=1.3027491279999999 podStartE2EDuration="1.302749128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:09:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:09:57.301345758 +0000 UTC m=+971.617912498" watchObservedRunningTime="2026-09-04 22:09:57.302749128 +0000 UTC m=+971.619315870"
Sep 04 22:09:57.319762 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:57.319520    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx" podStartSLOduration=1.319498688 podStartE2EDuration="1.319498688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:09:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:09:57.319122624 +0000 UTC m=+971.635689361" watchObservedRunningTime="2026-09-04 22:09:57.319498688 +0000 UTC m=+971.636065435"
Sep 04 22:09:58.253866 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:09:58.253829    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:10:02.964310 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:02.964272    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx"]
Sep 04 22:10:02.967827 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:02.967804    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx"
Sep 04 22:10:02.970945 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:02.970919    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-7a26c9-dockercfg-wlbll\""
Sep 04 22:10:02.971086 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:02.970918    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-7a26c9\""
Sep 04 22:10:02.980909 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:02.980879    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx"]
Sep 04 22:10:02.984891 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:02.984863    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4"]
Sep 04 22:10:02.988372 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:02.988350    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4"
Sep 04 22:10:02.998080 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:02.998051    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4"]
Sep 04 22:10:03.005898 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:03.005862    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d3f722b-6821-4b1e-9404-9a07af50a378-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx\" (UID: \"1d3f722b-6821-4b1e-9404-9a07af50a378\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx"
Sep 04 22:10:03.006059 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:03.005933    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll67d\" (UniqueName: \"kubernetes.io/projected/1d3f722b-6821-4b1e-9404-9a07af50a378-kube-api-access-ll67d\") pod \"payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx\" (UID: \"1d3f722b-6821-4b1e-9404-9a07af50a378\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx"
Sep 04 22:10:03.107031 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:03.106984    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll67d\" (UniqueName: \"kubernetes.io/projected/1d3f722b-6821-4b1e-9404-9a07af50a378-kube-api-access-ll67d\") pod \"payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx\" (UID: \"1d3f722b-6821-4b1e-9404-9a07af50a378\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx"
Sep 04 22:10:03.107242 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:03.107044    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkz8q\" (UniqueName: \"kubernetes.io/projected/4c1b06e8-98cb-4c9b-badf-6dc8e846e184-kube-api-access-tkz8q\") pod \"payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4\" (UID: \"4c1b06e8-98cb-4c9b-badf-6dc8e846e184\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4"
Sep 04 22:10:03.107242 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:03.107125    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c1b06e8-98cb-4c9b-badf-6dc8e846e184-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4\" (UID: \"4c1b06e8-98cb-4c9b-badf-6dc8e846e184\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4"
Sep 04 22:10:03.107242 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:03.107204    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d3f722b-6821-4b1e-9404-9a07af50a378-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx\" (UID: \"1d3f722b-6821-4b1e-9404-9a07af50a378\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx"
Sep 04 22:10:03.107954 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:03.107928    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d3f722b-6821-4b1e-9404-9a07af50a378-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx\" (UID: \"1d3f722b-6821-4b1e-9404-9a07af50a378\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx"
Sep 04 22:10:03.116699 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:03.116663    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll67d\" (UniqueName: \"kubernetes.io/projected/1d3f722b-6821-4b1e-9404-9a07af50a378-kube-api-access-ll67d\") pod \"payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx\" (UID: \"1d3f722b-6821-4b1e-9404-9a07af50a378\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx"
Sep 04 22:10:03.208328 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:03.208282    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkz8q\" (UniqueName: \"kubernetes.io/projected/4c1b06e8-98cb-4c9b-badf-6dc8e846e184-kube-api-access-tkz8q\") pod \"payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4\" (UID: \"4c1b06e8-98cb-4c9b-badf-6dc8e846e184\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4"
Sep 04 22:10:03.208550 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:03.208367    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c1b06e8-98cb-4c9b-badf-6dc8e846e184-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4\" (UID: \"4c1b06e8-98cb-4c9b-badf-6dc8e846e184\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4"
Sep 04 22:10:03.209066 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:03.209044    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c1b06e8-98cb-4c9b-badf-6dc8e846e184-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4\" (UID: \"4c1b06e8-98cb-4c9b-badf-6dc8e846e184\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4"
Sep 04 22:10:03.217281 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:03.217203    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkz8q\" (UniqueName: \"kubernetes.io/projected/4c1b06e8-98cb-4c9b-badf-6dc8e846e184-kube-api-access-tkz8q\") pod \"payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4\" (UID: \"4c1b06e8-98cb-4c9b-badf-6dc8e846e184\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4"
Sep 04 22:10:03.280215 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:03.280175    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx"
Sep 04 22:10:03.301664 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:03.301631    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4"
Sep 04 22:10:03.563431 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:03.563391    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx"]
Sep 04 22:10:03.566149 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:10:03.566106    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d3f722b_6821_4b1e_9404_9a07af50a378.slice/crio-bceb1e0239f2f7aa6ef22cb1a8347f8485d9e99282ce0b2cde76724c1f4f48f4 WatchSource:0}: Error finding container bceb1e0239f2f7aa6ef22cb1a8347f8485d9e99282ce0b2cde76724c1f4f48f4: Status 404 returned error can't find the container with id bceb1e0239f2f7aa6ef22cb1a8347f8485d9e99282ce0b2cde76724c1f4f48f4
Sep 04 22:10:03.591560 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:03.591529    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4"]
Sep 04 22:10:03.598595 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:10:03.598562    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c1b06e8_98cb_4c9b_badf_6dc8e846e184.slice/crio-2fd85c6f61c8ef1f97f2112378b175497637c45c845b9c33e8499817e848d775 WatchSource:0}: Error finding container 2fd85c6f61c8ef1f97f2112378b175497637c45c845b9c33e8499817e848d775: Status 404 returned error can't find the container with id 2fd85c6f61c8ef1f97f2112378b175497637c45c845b9c33e8499817e848d775
Sep 04 22:10:04.331189 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:04.331146    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx" event={"ID":"1d3f722b-6821-4b1e-9404-9a07af50a378","Type":"ContainerStarted","Data":"25c9501fb4b90790235344bac6be3951dece0e2514936e1a655a5289a6cba6c8"}
Sep 04 22:10:04.331699 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:04.331211    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx" event={"ID":"1d3f722b-6821-4b1e-9404-9a07af50a378","Type":"ContainerStarted","Data":"bceb1e0239f2f7aa6ef22cb1a8347f8485d9e99282ce0b2cde76724c1f4f48f4"}
Sep 04 22:10:04.331699 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:04.331244    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx"
Sep 04 22:10:04.332906 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:04.332878    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4" event={"ID":"4c1b06e8-98cb-4c9b-badf-6dc8e846e184","Type":"ContainerStarted","Data":"51f7e815e79adfc17065ae77f436254a2f971e37637524db7b7aae425b8532f6"}
Sep 04 22:10:04.333025 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:04.332927    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4" event={"ID":"4c1b06e8-98cb-4c9b-badf-6dc8e846e184","Type":"ContainerStarted","Data":"2fd85c6f61c8ef1f97f2112378b175497637c45c845b9c33e8499817e848d775"}
Sep 04 22:10:04.333025 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:04.332999    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4"
Sep 04 22:10:04.348637 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:04.348575    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx" podStartSLOduration=2.348555785 podStartE2EDuration="2.348555785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:10:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:10:04.347762929 +0000 UTC m=+978.664329668" watchObservedRunningTime="2026-09-04 22:10:04.348555785 +0000 UTC m=+978.665122523"
Sep 04 22:10:04.364889 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:04.364828    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4" podStartSLOduration=2.364809399 podStartE2EDuration="2.364809399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:10:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:10:04.363352374 +0000 UTC m=+978.679919113" watchObservedRunningTime="2026-09-04 22:10:04.364809399 +0000 UTC m=+978.681376128"
Sep 04 22:10:08.297484 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:08.297385    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx"
Sep 04 22:10:08.297869 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:08.297603    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r"
Sep 04 22:10:13.149236 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:13.149193    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:10:15.340614 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:15.340580    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4"
Sep 04 22:10:15.341063 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:15.340651    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx"
Sep 04 22:10:34.663198 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:34.663151    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:10:37.044512 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.044472    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r"]
Sep 04 22:10:37.045094 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.044757    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r" podUID="2b2a8fd8-31b4-4ed5-b0db-6972341e9cab" containerName="payload-processing" containerID="cri-o://314477efb867adc2cb01791c929650eed3efb46070f994d6c33edabe55c53325" gracePeriod=30
Sep 04 22:10:37.049546 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.049507    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx"]
Sep 04 22:10:37.049788 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.049758    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx" podUID="cd72d1a3-3f66-4c74-ae18-0afcd5761319" containerName="payload-pre-processing" containerID="cri-o://4a3839afa13b22b17c26d19b592e70330f56b408bc380d5113eec8a19c0a76ac" gracePeriod=30
Sep 04 22:10:37.336976 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.336949    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r"
Sep 04 22:10:37.339742 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.339717    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx"
Sep 04 22:10:37.432480 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.432403    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jz4s8\" (UniqueName: \"kubernetes.io/projected/cd72d1a3-3f66-4c74-ae18-0afcd5761319-kube-api-access-jz4s8\") pod \"cd72d1a3-3f66-4c74-ae18-0afcd5761319\" (UID: \"cd72d1a3-3f66-4c74-ae18-0afcd5761319\") "
Sep 04 22:10:37.432663 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.432487    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b2a8fd8-31b4-4ed5-b0db-6972341e9cab-plugins-config-volume\") pod \"2b2a8fd8-31b4-4ed5-b0db-6972341e9cab\" (UID: \"2b2a8fd8-31b4-4ed5-b0db-6972341e9cab\") "
Sep 04 22:10:37.432663 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.432525    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd72d1a3-3f66-4c74-ae18-0afcd5761319-plugins-config-volume\") pod \"cd72d1a3-3f66-4c74-ae18-0afcd5761319\" (UID: \"cd72d1a3-3f66-4c74-ae18-0afcd5761319\") "
Sep 04 22:10:37.432663 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.432589    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2ppx6\" (UniqueName: \"kubernetes.io/projected/2b2a8fd8-31b4-4ed5-b0db-6972341e9cab-kube-api-access-2ppx6\") pod \"2b2a8fd8-31b4-4ed5-b0db-6972341e9cab\" (UID: \"2b2a8fd8-31b4-4ed5-b0db-6972341e9cab\") "
Sep 04 22:10:37.432922 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.432892    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd72d1a3-3f66-4c74-ae18-0afcd5761319-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cd72d1a3-3f66-4c74-ae18-0afcd5761319" (UID: "cd72d1a3-3f66-4c74-ae18-0afcd5761319"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:10:37.433037 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.432893    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b2a8fd8-31b4-4ed5-b0db-6972341e9cab-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2b2a8fd8-31b4-4ed5-b0db-6972341e9cab" (UID: "2b2a8fd8-31b4-4ed5-b0db-6972341e9cab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:10:37.434660 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.434631    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd72d1a3-3f66-4c74-ae18-0afcd5761319-kube-api-access-jz4s8" (OuterVolumeSpecName: "kube-api-access-jz4s8") pod "cd72d1a3-3f66-4c74-ae18-0afcd5761319" (UID: "cd72d1a3-3f66-4c74-ae18-0afcd5761319"). InnerVolumeSpecName "kube-api-access-jz4s8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:10:37.434801 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.434771    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b2a8fd8-31b4-4ed5-b0db-6972341e9cab-kube-api-access-2ppx6" (OuterVolumeSpecName: "kube-api-access-2ppx6") pod "2b2a8fd8-31b4-4ed5-b0db-6972341e9cab" (UID: "2b2a8fd8-31b4-4ed5-b0db-6972341e9cab"). InnerVolumeSpecName "kube-api-access-2ppx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:10:37.514042 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.514007    2557 generic.go:356] "Generic (PLEG): container finished" podID="2b2a8fd8-31b4-4ed5-b0db-6972341e9cab" containerID="314477efb867adc2cb01791c929650eed3efb46070f994d6c33edabe55c53325" exitCode=0
Sep 04 22:10:37.514222 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.514048    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r"
Sep 04 22:10:37.514222 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.514084    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r" event={"ID":"2b2a8fd8-31b4-4ed5-b0db-6972341e9cab","Type":"ContainerDied","Data":"314477efb867adc2cb01791c929650eed3efb46070f994d6c33edabe55c53325"}
Sep 04 22:10:37.514222 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.514137    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r" event={"ID":"2b2a8fd8-31b4-4ed5-b0db-6972341e9cab","Type":"ContainerDied","Data":"ad88510c3c1e4f69e9cbcb3f1b604324a75d20d7d25ccfca3f7c65c4d135bef7"}
Sep 04 22:10:37.514222 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.514166    2557 scope.go:117] "RemoveContainer" containerID="314477efb867adc2cb01791c929650eed3efb46070f994d6c33edabe55c53325"
Sep 04 22:10:37.515489 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.515456    2557 generic.go:356] "Generic (PLEG): container finished" podID="cd72d1a3-3f66-4c74-ae18-0afcd5761319" containerID="4a3839afa13b22b17c26d19b592e70330f56b408bc380d5113eec8a19c0a76ac" exitCode=0
Sep 04 22:10:37.515608 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.515500    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx"
Sep 04 22:10:37.515608 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.515535    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx" event={"ID":"cd72d1a3-3f66-4c74-ae18-0afcd5761319","Type":"ContainerDied","Data":"4a3839afa13b22b17c26d19b592e70330f56b408bc380d5113eec8a19c0a76ac"}
Sep 04 22:10:37.515608 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.515582    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx" event={"ID":"cd72d1a3-3f66-4c74-ae18-0afcd5761319","Type":"ContainerDied","Data":"e18681f65815d71e7beb93b15e6789744c37e809173a2a6463bb22112543c8c8"}
Sep 04 22:10:37.524945 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.524921    2557 scope.go:117] "RemoveContainer" containerID="314477efb867adc2cb01791c929650eed3efb46070f994d6c33edabe55c53325"
Sep 04 22:10:37.525245 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:10:37.525225    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"314477efb867adc2cb01791c929650eed3efb46070f994d6c33edabe55c53325\": container with ID starting with 314477efb867adc2cb01791c929650eed3efb46070f994d6c33edabe55c53325 not found: ID does not exist" containerID="314477efb867adc2cb01791c929650eed3efb46070f994d6c33edabe55c53325"
Sep 04 22:10:37.525322 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.525257    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"314477efb867adc2cb01791c929650eed3efb46070f994d6c33edabe55c53325"} err="failed to get container status \"314477efb867adc2cb01791c929650eed3efb46070f994d6c33edabe55c53325\": rpc error: code = NotFound desc = could not find container \"314477efb867adc2cb01791c929650eed3efb46070f994d6c33edabe55c53325\": container with ID starting with 314477efb867adc2cb01791c929650eed3efb46070f994d6c33edabe55c53325 not found: ID does not exist"
Sep 04 22:10:37.525322 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.525283    2557 scope.go:117] "RemoveContainer" containerID="4a3839afa13b22b17c26d19b592e70330f56b408bc380d5113eec8a19c0a76ac"
Sep 04 22:10:37.533499 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.533470    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jz4s8\" (UniqueName: \"kubernetes.io/projected/cd72d1a3-3f66-4c74-ae18-0afcd5761319-kube-api-access-jz4s8\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:10:37.533623 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.533532    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2b2a8fd8-31b4-4ed5-b0db-6972341e9cab-plugins-config-volume\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:10:37.533623 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.533549    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cd72d1a3-3f66-4c74-ae18-0afcd5761319-plugins-config-volume\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:10:37.533623 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.533558    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2ppx6\" (UniqueName: \"kubernetes.io/projected/2b2a8fd8-31b4-4ed5-b0db-6972341e9cab-kube-api-access-2ppx6\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:10:37.534018 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.534005    2557 scope.go:117] "RemoveContainer" containerID="4a3839afa13b22b17c26d19b592e70330f56b408bc380d5113eec8a19c0a76ac"
Sep 04 22:10:37.534339 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:10:37.534320    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a3839afa13b22b17c26d19b592e70330f56b408bc380d5113eec8a19c0a76ac\": container with ID starting with 4a3839afa13b22b17c26d19b592e70330f56b408bc380d5113eec8a19c0a76ac not found: ID does not exist" containerID="4a3839afa13b22b17c26d19b592e70330f56b408bc380d5113eec8a19c0a76ac"
Sep 04 22:10:37.534382 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.534345    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a3839afa13b22b17c26d19b592e70330f56b408bc380d5113eec8a19c0a76ac"} err="failed to get container status \"4a3839afa13b22b17c26d19b592e70330f56b408bc380d5113eec8a19c0a76ac\": rpc error: code = NotFound desc = could not find container \"4a3839afa13b22b17c26d19b592e70330f56b408bc380d5113eec8a19c0a76ac\": container with ID starting with 4a3839afa13b22b17c26d19b592e70330f56b408bc380d5113eec8a19c0a76ac not found: ID does not exist"
Sep 04 22:10:37.538022 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.537996    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r"]
Sep 04 22:10:37.544229 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.544200    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-806943a9-6c744fcd97-9wb9r"]
Sep 04 22:10:37.554899 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.554859    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx"]
Sep 04 22:10:37.560183 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:37.560152    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-806943a9-65ddb6cb66-2ckbx"]
Sep 04 22:10:38.352825 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:38.352784    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b2a8fd8-31b4-4ed5-b0db-6972341e9cab" path="/var/lib/kubelet/pods/2b2a8fd8-31b4-4ed5-b0db-6972341e9cab/volumes"
Sep 04 22:10:38.353206 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:38.353139    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd72d1a3-3f66-4c74-ae18-0afcd5761319" path="/var/lib/kubelet/pods/cd72d1a3-3f66-4c74-ae18-0afcd5761319/volumes"
Sep 04 22:10:46.245789 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:10:46.245747    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:11:03.281519 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.281478    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r"]
Sep 04 22:11:03.281901 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.281834    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd72d1a3-3f66-4c74-ae18-0afcd5761319" containerName="payload-pre-processing"
Sep 04 22:11:03.281901 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.281845    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd72d1a3-3f66-4c74-ae18-0afcd5761319" containerName="payload-pre-processing"
Sep 04 22:11:03.281901 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.281859    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b2a8fd8-31b4-4ed5-b0db-6972341e9cab" containerName="payload-processing"
Sep 04 22:11:03.281901 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.281864    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b2a8fd8-31b4-4ed5-b0db-6972341e9cab" containerName="payload-processing"
Sep 04 22:11:03.282038 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.281922    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b2a8fd8-31b4-4ed5-b0db-6972341e9cab" containerName="payload-processing"
Sep 04 22:11:03.282038 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.281931    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd72d1a3-3f66-4c74-ae18-0afcd5761319" containerName="payload-pre-processing"
Sep 04 22:11:03.286305 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.286275    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r"
Sep 04 22:11:03.290018 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.289988    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-fc3156\""
Sep 04 22:11:03.290185 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.290045    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-fc3156-dockercfg-4pq8n\""
Sep 04 22:11:03.306479 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.302470    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq"]
Sep 04 22:11:03.311459 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.311399    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r"]
Sep 04 22:11:03.311664 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.311526    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq"
Sep 04 22:11:03.323755 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.323720    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq"]
Sep 04 22:11:03.355483 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.355436    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvzwc\" (UniqueName: \"kubernetes.io/projected/8a2aae30-4b34-46b2-b0dd-595eb60f39fa-kube-api-access-pvzwc\") pod \"payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq\" (UID: \"8a2aae30-4b34-46b2-b0dd-595eb60f39fa\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq"
Sep 04 22:11:03.355689 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.355588    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cd50664-2acb-4745-956a-21f990edaed7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r\" (UID: \"0cd50664-2acb-4745-956a-21f990edaed7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r"
Sep 04 22:11:03.355689 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.355637    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a2aae30-4b34-46b2-b0dd-595eb60f39fa-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq\" (UID: \"8a2aae30-4b34-46b2-b0dd-595eb60f39fa\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq"
Sep 04 22:11:03.355689 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.355653    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jfkp\" (UniqueName: \"kubernetes.io/projected/0cd50664-2acb-4745-956a-21f990edaed7-kube-api-access-7jfkp\") pod \"payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r\" (UID: \"0cd50664-2acb-4745-956a-21f990edaed7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r"
Sep 04 22:11:03.456985 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.456933    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvzwc\" (UniqueName: \"kubernetes.io/projected/8a2aae30-4b34-46b2-b0dd-595eb60f39fa-kube-api-access-pvzwc\") pod \"payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq\" (UID: \"8a2aae30-4b34-46b2-b0dd-595eb60f39fa\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq"
Sep 04 22:11:03.457204 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.457033    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cd50664-2acb-4745-956a-21f990edaed7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r\" (UID: \"0cd50664-2acb-4745-956a-21f990edaed7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r"
Sep 04 22:11:03.457204 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.457065    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a2aae30-4b34-46b2-b0dd-595eb60f39fa-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq\" (UID: \"8a2aae30-4b34-46b2-b0dd-595eb60f39fa\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq"
Sep 04 22:11:03.457204 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.457088    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jfkp\" (UniqueName: \"kubernetes.io/projected/0cd50664-2acb-4745-956a-21f990edaed7-kube-api-access-7jfkp\") pod \"payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r\" (UID: \"0cd50664-2acb-4745-956a-21f990edaed7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r"
Sep 04 22:11:03.457862 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.457840    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cd50664-2acb-4745-956a-21f990edaed7-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r\" (UID: \"0cd50664-2acb-4745-956a-21f990edaed7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r"
Sep 04 22:11:03.457921 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.457879    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a2aae30-4b34-46b2-b0dd-595eb60f39fa-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq\" (UID: \"8a2aae30-4b34-46b2-b0dd-595eb60f39fa\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq"
Sep 04 22:11:03.465638 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.465607    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvzwc\" (UniqueName: \"kubernetes.io/projected/8a2aae30-4b34-46b2-b0dd-595eb60f39fa-kube-api-access-pvzwc\") pod \"payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq\" (UID: \"8a2aae30-4b34-46b2-b0dd-595eb60f39fa\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq"
Sep 04 22:11:03.465794 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.465655    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jfkp\" (UniqueName: \"kubernetes.io/projected/0cd50664-2acb-4745-956a-21f990edaed7-kube-api-access-7jfkp\") pod \"payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r\" (UID: \"0cd50664-2acb-4745-956a-21f990edaed7\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r"
Sep 04 22:11:03.598296 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.598196    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r"
Sep 04 22:11:03.622920 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.622881    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq"
Sep 04 22:11:03.879394 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.879366    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r"]
Sep 04 22:11:03.882074 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:11:03.882042    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cd50664_2acb_4745_956a_21f990edaed7.slice/crio-ec875c801b17cee570b0bf91814a8470dbd1d6b3b0ca391c9531d49db428dc4f WatchSource:0}: Error finding container ec875c801b17cee570b0bf91814a8470dbd1d6b3b0ca391c9531d49db428dc4f: Status 404 returned error can't find the container with id ec875c801b17cee570b0bf91814a8470dbd1d6b3b0ca391c9531d49db428dc4f
Sep 04 22:11:03.907764 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:03.907736    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq"]
Sep 04 22:11:03.910620 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:11:03.910591    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a2aae30_4b34_46b2_b0dd_595eb60f39fa.slice/crio-47273b80b6a66ae23499e53bcd817deb8b2d06cdfe085efca292897d267c97e2 WatchSource:0}: Error finding container 47273b80b6a66ae23499e53bcd817deb8b2d06cdfe085efca292897d267c97e2: Status 404 returned error can't find the container with id 47273b80b6a66ae23499e53bcd817deb8b2d06cdfe085efca292897d267c97e2
Sep 04 22:11:04.664803 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:04.664747    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r" event={"ID":"0cd50664-2acb-4745-956a-21f990edaed7","Type":"ContainerStarted","Data":"2753f3c57643a9883f512703caef51b41fa069c58e98c60b8343f3eea178099d"}
Sep 04 22:11:04.665297 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:04.664819    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r" event={"ID":"0cd50664-2acb-4745-956a-21f990edaed7","Type":"ContainerStarted","Data":"ec875c801b17cee570b0bf91814a8470dbd1d6b3b0ca391c9531d49db428dc4f"}
Sep 04 22:11:04.665297 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:04.664846    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r"
Sep 04 22:11:04.666528 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:04.666491    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq" event={"ID":"8a2aae30-4b34-46b2-b0dd-595eb60f39fa","Type":"ContainerStarted","Data":"35427fb44bb813bd7765048a55ae9b3ef557a6b344967672529752acae925e34"}
Sep 04 22:11:04.666675 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:04.666552    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq" event={"ID":"8a2aae30-4b34-46b2-b0dd-595eb60f39fa","Type":"ContainerStarted","Data":"47273b80b6a66ae23499e53bcd817deb8b2d06cdfe085efca292897d267c97e2"}
Sep 04 22:11:04.666675 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:04.666580    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq"
Sep 04 22:11:04.683956 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:04.683879    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r" podStartSLOduration=1.683857344 podStartE2EDuration="1.683857344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:11:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:11:04.680083974 +0000 UTC m=+1038.996650723" watchObservedRunningTime="2026-09-04 22:11:04.683857344 +0000 UTC m=+1039.000424082"
Sep 04 22:11:04.700104 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:04.700039    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq" podStartSLOduration=1.700015608 podStartE2EDuration="1.700015608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:11:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:11:04.698965179 +0000 UTC m=+1039.015531917" watchObservedRunningTime="2026-09-04 22:11:04.700015608 +0000 UTC m=+1039.016582347"
Sep 04 22:11:06.654758 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:06.654716    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:11:15.674648 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:15.674607    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r"
Sep 04 22:11:15.675095 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:15.674734    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq"
Sep 04 22:11:22.657864 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:22.657820    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:11:32.485480 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:32.485437    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78"]
Sep 04 22:11:32.488967 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:32.488942    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78"
Sep 04 22:11:32.491448 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:32.491399    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-3abe72\""
Sep 04 22:11:32.491610 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:32.491592    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-3abe72-dockercfg-ftrmt\""
Sep 04 22:11:32.504012 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:32.503977    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78"]
Sep 04 22:11:32.517270 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:32.517235    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl"]
Sep 04 22:11:32.520955 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:32.520925    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl"
Sep 04 22:11:32.525231 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:32.525199    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nm6m8\" (UniqueName: \"kubernetes.io/projected/4466206b-3c94-4dfe-b1d5-8c689298684b-kube-api-access-nm6m8\") pod \"payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl\" (UID: \"4466206b-3c94-4dfe-b1d5-8c689298684b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl"
Sep 04 22:11:32.525368 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:32.525253    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k67tk\" (UniqueName: \"kubernetes.io/projected/8ab87507-bdff-439a-8ece-8902dffe7dfd-kube-api-access-k67tk\") pod \"payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78\" (UID: \"8ab87507-bdff-439a-8ece-8902dffe7dfd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78"
Sep 04 22:11:32.525368 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:32.525312    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ab87507-bdff-439a-8ece-8902dffe7dfd-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78\" (UID: \"8ab87507-bdff-439a-8ece-8902dffe7dfd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78"
Sep 04 22:11:32.525368 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:32.525340    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4466206b-3c94-4dfe-b1d5-8c689298684b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl\" (UID: \"4466206b-3c94-4dfe-b1d5-8c689298684b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl"
Sep 04 22:11:32.533552 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:32.533521    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl"]
Sep 04 22:11:32.626509 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:32.626468    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nm6m8\" (UniqueName: \"kubernetes.io/projected/4466206b-3c94-4dfe-b1d5-8c689298684b-kube-api-access-nm6m8\") pod \"payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl\" (UID: \"4466206b-3c94-4dfe-b1d5-8c689298684b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl"
Sep 04 22:11:32.626702 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:32.626568    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k67tk\" (UniqueName: \"kubernetes.io/projected/8ab87507-bdff-439a-8ece-8902dffe7dfd-kube-api-access-k67tk\") pod \"payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78\" (UID: \"8ab87507-bdff-439a-8ece-8902dffe7dfd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78"
Sep 04 22:11:32.626702 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:32.626620    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ab87507-bdff-439a-8ece-8902dffe7dfd-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78\" (UID: \"8ab87507-bdff-439a-8ece-8902dffe7dfd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78"
Sep 04 22:11:32.626702 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:32.626660    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4466206b-3c94-4dfe-b1d5-8c689298684b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl\" (UID: \"4466206b-3c94-4dfe-b1d5-8c689298684b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl"
Sep 04 22:11:32.627382 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:32.627356    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ab87507-bdff-439a-8ece-8902dffe7dfd-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78\" (UID: \"8ab87507-bdff-439a-8ece-8902dffe7dfd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78"
Sep 04 22:11:32.627525 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:32.627401    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4466206b-3c94-4dfe-b1d5-8c689298684b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl\" (UID: \"4466206b-3c94-4dfe-b1d5-8c689298684b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl"
Sep 04 22:11:32.635392 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:32.635362    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k67tk\" (UniqueName: \"kubernetes.io/projected/8ab87507-bdff-439a-8ece-8902dffe7dfd-kube-api-access-k67tk\") pod \"payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78\" (UID: \"8ab87507-bdff-439a-8ece-8902dffe7dfd\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78"
Sep 04 22:11:32.635543 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:32.635369    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nm6m8\" (UniqueName: \"kubernetes.io/projected/4466206b-3c94-4dfe-b1d5-8c689298684b-kube-api-access-nm6m8\") pod \"payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl\" (UID: \"4466206b-3c94-4dfe-b1d5-8c689298684b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl"
Sep 04 22:11:32.800391 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:32.800347    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78"
Sep 04 22:11:32.834106 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:32.834068    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl"
Sep 04 22:11:33.079732 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:33.079705    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78"]
Sep 04 22:11:33.080756 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:11:33.080730    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ab87507_bdff_439a_8ece_8902dffe7dfd.slice/crio-a35f64706e585465ac63833f5a65bf415951dcfeb5fd0077c523e360405c55ce WatchSource:0}: Error finding container a35f64706e585465ac63833f5a65bf415951dcfeb5fd0077c523e360405c55ce: Status 404 returned error can't find the container with id a35f64706e585465ac63833f5a65bf415951dcfeb5fd0077c523e360405c55ce
Sep 04 22:11:33.117355 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:33.117330    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl"]
Sep 04 22:11:33.118859 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:11:33.118828    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4466206b_3c94_4dfe_b1d5_8c689298684b.slice/crio-89fb8d45ae846a6edd9b70846881e8635049c033b064d5c845ced1d57cdb0ed2 WatchSource:0}: Error finding container 89fb8d45ae846a6edd9b70846881e8635049c033b064d5c845ced1d57cdb0ed2: Status 404 returned error can't find the container with id 89fb8d45ae846a6edd9b70846881e8635049c033b064d5c845ced1d57cdb0ed2
Sep 04 22:11:33.828595 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:33.828555    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78" event={"ID":"8ab87507-bdff-439a-8ece-8902dffe7dfd","Type":"ContainerStarted","Data":"35854cc0c119afd8f5e6dedf09db29278ce33f86fd971b32083cbbb5e025737b"}
Sep 04 22:11:33.829094 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:33.828620    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78" event={"ID":"8ab87507-bdff-439a-8ece-8902dffe7dfd","Type":"ContainerStarted","Data":"a35f64706e585465ac63833f5a65bf415951dcfeb5fd0077c523e360405c55ce"}
Sep 04 22:11:33.829094 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:33.828779    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78"
Sep 04 22:11:33.830324 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:33.830294    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl" event={"ID":"4466206b-3c94-4dfe-b1d5-8c689298684b","Type":"ContainerStarted","Data":"113728004bf9f5dbd9d7377d1ca848af9b18e35050c5e0511dafe48b00376ec0"}
Sep 04 22:11:33.830444 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:33.830349    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl" event={"ID":"4466206b-3c94-4dfe-b1d5-8c689298684b","Type":"ContainerStarted","Data":"89fb8d45ae846a6edd9b70846881e8635049c033b064d5c845ced1d57cdb0ed2"}
Sep 04 22:11:33.830444 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:33.830385    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl"
Sep 04 22:11:33.849254 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:33.849196    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78" podStartSLOduration=1.849178016 podStartE2EDuration="1.849178016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:11:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:11:33.847950974 +0000 UTC m=+1068.164517713" watchObservedRunningTime="2026-09-04 22:11:33.849178016 +0000 UTC m=+1068.165744778"
Sep 04 22:11:33.866809 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:33.866742    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl" podStartSLOduration=1.866721317 podStartE2EDuration="1.866721317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:11:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:11:33.864907715 +0000 UTC m=+1068.181474479" watchObservedRunningTime="2026-09-04 22:11:33.866721317 +0000 UTC m=+1068.183288055"
Sep 04 22:11:38.057304 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:38.057260    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:11:44.838399 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:44.838365    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl"
Sep 04 22:11:44.838788 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:44.838467    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78"
Sep 04 22:11:50.354877 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:11:50.354838    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:12:01.749884 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:01.749842    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:12:13.728656 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:13.728614    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9"]
Sep 04 22:12:13.733321 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:13.733297    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9"
Sep 04 22:12:13.735832 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:13.735797    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-bba28942\""
Sep 04 22:12:13.735991 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:13.735879    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-bba28942-dockercfg-qjrnx\""
Sep 04 22:12:13.742293 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:13.742263    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9"]
Sep 04 22:12:13.908559 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:13.908502    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa63c716-c2fc-4f3f-8d6e-145452326e5c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9\" (UID: \"aa63c716-c2fc-4f3f-8d6e-145452326e5c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9"
Sep 04 22:12:13.908754 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:13.908634    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mb5g\" (UniqueName: \"kubernetes.io/projected/aa63c716-c2fc-4f3f-8d6e-145452326e5c-kube-api-access-9mb5g\") pod \"payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9\" (UID: \"aa63c716-c2fc-4f3f-8d6e-145452326e5c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9"
Sep 04 22:12:14.009344 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:14.009303    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mb5g\" (UniqueName: \"kubernetes.io/projected/aa63c716-c2fc-4f3f-8d6e-145452326e5c-kube-api-access-9mb5g\") pod \"payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9\" (UID: \"aa63c716-c2fc-4f3f-8d6e-145452326e5c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9"
Sep 04 22:12:14.009539 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:14.009457    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa63c716-c2fc-4f3f-8d6e-145452326e5c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9\" (UID: \"aa63c716-c2fc-4f3f-8d6e-145452326e5c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9"
Sep 04 22:12:14.010177 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:14.010152    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa63c716-c2fc-4f3f-8d6e-145452326e5c-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9\" (UID: \"aa63c716-c2fc-4f3f-8d6e-145452326e5c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9"
Sep 04 22:12:14.023989 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:14.023954    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mb5g\" (UniqueName: \"kubernetes.io/projected/aa63c716-c2fc-4f3f-8d6e-145452326e5c-kube-api-access-9mb5g\") pod \"payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9\" (UID: \"aa63c716-c2fc-4f3f-8d6e-145452326e5c\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9"
Sep 04 22:12:14.044958 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:14.044922    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9"
Sep 04 22:12:14.149597 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:14.149550    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:12:14.321725 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:14.321693    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9"]
Sep 04 22:12:14.324091 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:12:14.324059    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa63c716_c2fc_4f3f_8d6e_145452326e5c.slice/crio-6752ed65575c05926c1b77ebee11f2d000c8db2ca93f716c788c545d7e86bf87 WatchSource:0}: Error finding container 6752ed65575c05926c1b77ebee11f2d000c8db2ca93f716c788c545d7e86bf87: Status 404 returned error can't find the container with id 6752ed65575c05926c1b77ebee11f2d000c8db2ca93f716c788c545d7e86bf87
Sep 04 22:12:15.056565 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:15.056525    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9" event={"ID":"aa63c716-c2fc-4f3f-8d6e-145452326e5c","Type":"ContainerStarted","Data":"8f8e4e84ce67b435e53dbe0c186842c13c58cdf7bf33d369a2e0f59b1d1a1a5d"}
Sep 04 22:12:15.056964 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:15.056591    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9" event={"ID":"aa63c716-c2fc-4f3f-8d6e-145452326e5c","Type":"ContainerStarted","Data":"6752ed65575c05926c1b77ebee11f2d000c8db2ca93f716c788c545d7e86bf87"}
Sep 04 22:12:15.056964 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:15.056620    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9"
Sep 04 22:12:15.078399 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:15.078334    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9" podStartSLOduration=2.078315102 podStartE2EDuration="2.078315102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:12:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:12:15.075272291 +0000 UTC m=+1109.391839031" watchObservedRunningTime="2026-09-04 22:12:15.078315102 +0000 UTC m=+1109.394881918"
Sep 04 22:12:26.065268 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:26.065237    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9"
Sep 04 22:12:27.354678 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:27.354630    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:12:40.045015 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.044975    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:12:40.635732 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.635698    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"]
Sep 04 22:12:40.639461 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.639437    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:40.642503 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.642470    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-bba28942\"/\"e2e-lifecycle-model-bba28942-kserve-self-signed-certs\""
Sep 04 22:12:40.642503 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.642473    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-bba28942\"/\"openshift-service-ca.crt\""
Sep 04 22:12:40.642687 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.642521    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-bba28942\"/\"kube-root-ca.crt\""
Sep 04 22:12:40.642687 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.642638    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-bba28942\"/\"default-dockercfg-hfm6z\""
Sep 04 22:12:40.647699 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.647676    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"]
Sep 04 22:12:40.760387 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.760345    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-dshm\") pod \"e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") " pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:40.760629 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.760408    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfnxp\" (UniqueName: \"kubernetes.io/projected/2b41afb0-00ee-4f85-8624-6e6c90631410-kube-api-access-dfnxp\") pod \"e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") " pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:40.760629 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.760557    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b41afb0-00ee-4f85-8624-6e6c90631410-tls-certs\") pod \"e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") " pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:40.760629 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.760620    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-model-cache\") pod \"e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") " pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:40.760787 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.760653    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-home\") pod \"e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") " pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:40.760787 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.760681    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-tmp-dir\") pod \"e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") " pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:40.861979 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.861937    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b41afb0-00ee-4f85-8624-6e6c90631410-tls-certs\") pod \"e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") " pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:40.862803 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.862777    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-model-cache\") pod \"e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") " pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:40.863471 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.863435    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-home\") pod \"e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") " pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:40.864127 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.864104    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-tmp-dir\") pod \"e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") " pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:40.864241 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.864201    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-dshm\") pod \"e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") " pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:40.864314 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.864294    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfnxp\" (UniqueName: \"kubernetes.io/projected/2b41afb0-00ee-4f85-8624-6e6c90631410-kube-api-access-dfnxp\") pod \"e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") " pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:40.864692 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.864670    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-tmp-dir\") pod \"e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") " pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:40.864824 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.864056    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-home\") pod \"e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") " pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:40.864920 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.863346    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-model-cache\") pod \"e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") " pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:40.867672 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.867639    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-dshm\") pod \"e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") " pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:40.867807 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.867785    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b41afb0-00ee-4f85-8624-6e6c90631410-tls-certs\") pod \"e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") " pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:40.875798 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.875770    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfnxp\" (UniqueName: \"kubernetes.io/projected/2b41afb0-00ee-4f85-8624-6e6c90631410-kube-api-access-dfnxp\") pod \"e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") " pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:40.949651 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:40.949558    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:41.233858 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:41.233832    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"]
Sep 04 22:12:41.236692 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:12:41.236663    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b41afb0_00ee_4f85_8624_6e6c90631410.slice/crio-673eab2d1b79e9825fc6d01e4d98041c0e3bce57ba12360afa01b27ee9e23229 WatchSource:0}: Error finding container 673eab2d1b79e9825fc6d01e4d98041c0e3bce57ba12360afa01b27ee9e23229: Status 404 returned error can't find the container with id 673eab2d1b79e9825fc6d01e4d98041c0e3bce57ba12360afa01b27ee9e23229
Sep 04 22:12:42.209226 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:42.209190    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2" event={"ID":"2b41afb0-00ee-4f85-8624-6e6c90631410","Type":"ContainerStarted","Data":"f92d0159d06e80cb35dd3b2eb407bf021a0262031101a8f0eecb537dc8c2cf97"}
Sep 04 22:12:42.209226 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:42.209234    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2" event={"ID":"2b41afb0-00ee-4f85-8624-6e6c90631410","Type":"ContainerStarted","Data":"673eab2d1b79e9825fc6d01e4d98041c0e3bce57ba12360afa01b27ee9e23229"}
Sep 04 22:12:42.226899 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:42.226843    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2" podStartSLOduration=2.22682549 podStartE2EDuration="2.22682549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:12:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:12:42.224536631 +0000 UTC m=+1136.541103370" watchObservedRunningTime="2026-09-04 22:12:42.22682549 +0000 UTC m=+1136.543392228"
Sep 04 22:12:46.417832 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:46.417798    2557 scope.go:117] "RemoveContainer" containerID="806e24937e0c41b411d2b7e738e38e9750bb77187f99003f3b65ceea8e3b2f46"
Sep 04 22:12:50.950577 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:50.950522    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:50.950978 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:50.950594    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:50.963640 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:50.963597    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:51.270099 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:51.270067    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:12:55.246126 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:12:55.246089    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:13:10.446485 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:10.446381    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:13:11.228237 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:11.228193    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bba28942-t72ql"]
Sep 04 22:13:11.233916 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:11.233883    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bba28942-t72ql"
Sep 04 22:13:11.236764 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:11.236725    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 04 22:13:11.236764 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:11.236732    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dftg6\""
Sep 04 22:13:11.247262 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:11.247228    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bba28942-t72ql"]
Sep 04 22:13:11.247555 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:11.247227    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fa95dce-01ec-46e0-92e6-aeff556495dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bba28942-t72ql\" (UID: \"1fa95dce-01ec-46e0-92e6-aeff556495dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bba28942-t72ql"
Sep 04 22:13:11.348857 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:11.348817    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fa95dce-01ec-46e0-92e6-aeff556495dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bba28942-t72ql\" (UID: \"1fa95dce-01ec-46e0-92e6-aeff556495dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bba28942-t72ql"
Sep 04 22:13:11.349548 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:11.349528    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fa95dce-01ec-46e0-92e6-aeff556495dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-bba28942-t72ql\" (UID: \"1fa95dce-01ec-46e0-92e6-aeff556495dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bba28942-t72ql"
Sep 04 22:13:11.548844 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:11.548800    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bba28942-t72ql"
Sep 04 22:13:11.819045 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:11.819019    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bba28942-t72ql"]
Sep 04 22:13:11.820971 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:13:11.820938    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fa95dce_01ec_46e0_92e6_aeff556495dc.slice/crio-dded6de324a195781d1ca80a161c6b35221970c60d70624f5d41b8efb834dd45 WatchSource:0}: Error finding container dded6de324a195781d1ca80a161c6b35221970c60d70624f5d41b8efb834dd45: Status 404 returned error can't find the container with id dded6de324a195781d1ca80a161c6b35221970c60d70624f5d41b8efb834dd45
Sep 04 22:13:12.373688 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:12.373646    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bba28942-t72ql" event={"ID":"1fa95dce-01ec-46e0-92e6-aeff556495dc","Type":"ContainerStarted","Data":"2e829cc83be95667bfb16dbf6c9bd76ea758d2323e2cc43d3be2af293fcca8d2"}
Sep 04 22:13:12.373876 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:12.373711    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bba28942-t72ql" event={"ID":"1fa95dce-01ec-46e0-92e6-aeff556495dc","Type":"ContainerStarted","Data":"dded6de324a195781d1ca80a161c6b35221970c60d70624f5d41b8efb834dd45"}
Sep 04 22:13:12.389304 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:12.389251    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bba28942-t72ql" podStartSLOduration=1.389231165 podStartE2EDuration="1.389231165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:13:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:13:12.387426921 +0000 UTC m=+1166.703993650" watchObservedRunningTime="2026-09-04 22:13:12.389231165 +0000 UTC m=+1166.705797903"
Sep 04 22:13:13.380808 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:13.380771    2557 generic.go:356] "Generic (PLEG): container finished" podID="1fa95dce-01ec-46e0-92e6-aeff556495dc" containerID="2e829cc83be95667bfb16dbf6c9bd76ea758d2323e2cc43d3be2af293fcca8d2" exitCode=0
Sep 04 22:13:13.381192 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:13.380856    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bba28942-t72ql" event={"ID":"1fa95dce-01ec-46e0-92e6-aeff556495dc","Type":"ContainerDied","Data":"2e829cc83be95667bfb16dbf6c9bd76ea758d2323e2cc43d3be2af293fcca8d2"}
Sep 04 22:13:14.521265 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:14.521232    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bba28942-t72ql"
Sep 04 22:13:14.580326 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:14.580283    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fa95dce-01ec-46e0-92e6-aeff556495dc-maas-api-service-ca\") pod \"1fa95dce-01ec-46e0-92e6-aeff556495dc\" (UID: \"1fa95dce-01ec-46e0-92e6-aeff556495dc\") "
Sep 04 22:13:14.580709 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:14.580683    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fa95dce-01ec-46e0-92e6-aeff556495dc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1fa95dce-01ec-46e0-92e6-aeff556495dc" (UID: "1fa95dce-01ec-46e0-92e6-aeff556495dc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:13:14.681222 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:14.681120    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fa95dce-01ec-46e0-92e6-aeff556495dc-maas-api-service-ca\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:13:15.394099 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:15.394061    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bba28942-t72ql"
Sep 04 22:13:15.394285 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:15.394062    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bba28942-t72ql" event={"ID":"1fa95dce-01ec-46e0-92e6-aeff556495dc","Type":"ContainerDied","Data":"dded6de324a195781d1ca80a161c6b35221970c60d70624f5d41b8efb834dd45"}
Sep 04 22:13:15.394285 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:15.394190    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dded6de324a195781d1ca80a161c6b35221970c60d70624f5d41b8efb834dd45"
Sep 04 22:13:26.560556 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:26.560502    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:13:26.767019 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:26.766981    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9"]
Sep 04 22:13:26.767528 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:26.767457    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9" podUID="aa63c716-c2fc-4f3f-8d6e-145452326e5c" containerName="payload-pre-processing" containerID="cri-o://8f8e4e84ce67b435e53dbe0c186842c13c58cdf7bf33d369a2e0f59b1d1a1a5d" gracePeriod=30
Sep 04 22:13:26.945788 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:26.945721    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bba28942-t72ql"]
Sep 04 22:13:26.950462 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:26.950405    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-bba28942-t72ql"]
Sep 04 22:13:27.078435 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:27.077192    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9"
Sep 04 22:13:27.206950 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:27.206819    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mb5g\" (UniqueName: \"kubernetes.io/projected/aa63c716-c2fc-4f3f-8d6e-145452326e5c-kube-api-access-9mb5g\") pod \"aa63c716-c2fc-4f3f-8d6e-145452326e5c\" (UID: \"aa63c716-c2fc-4f3f-8d6e-145452326e5c\") "
Sep 04 22:13:27.206950 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:27.206908    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa63c716-c2fc-4f3f-8d6e-145452326e5c-plugins-config-volume\") pod \"aa63c716-c2fc-4f3f-8d6e-145452326e5c\" (UID: \"aa63c716-c2fc-4f3f-8d6e-145452326e5c\") "
Sep 04 22:13:27.207378 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:27.207346    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa63c716-c2fc-4f3f-8d6e-145452326e5c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "aa63c716-c2fc-4f3f-8d6e-145452326e5c" (UID: "aa63c716-c2fc-4f3f-8d6e-145452326e5c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:13:27.209283 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:27.209250    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa63c716-c2fc-4f3f-8d6e-145452326e5c-kube-api-access-9mb5g" (OuterVolumeSpecName: "kube-api-access-9mb5g") pod "aa63c716-c2fc-4f3f-8d6e-145452326e5c" (UID: "aa63c716-c2fc-4f3f-8d6e-145452326e5c"). InnerVolumeSpecName "kube-api-access-9mb5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:13:27.309054 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:27.309000    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mb5g\" (UniqueName: \"kubernetes.io/projected/aa63c716-c2fc-4f3f-8d6e-145452326e5c-kube-api-access-9mb5g\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:13:27.309054 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:27.309056    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/aa63c716-c2fc-4f3f-8d6e-145452326e5c-plugins-config-volume\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:13:27.466875 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:27.466779    2557 generic.go:356] "Generic (PLEG): container finished" podID="aa63c716-c2fc-4f3f-8d6e-145452326e5c" containerID="8f8e4e84ce67b435e53dbe0c186842c13c58cdf7bf33d369a2e0f59b1d1a1a5d" exitCode=0
Sep 04 22:13:27.466875 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:27.466833    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9" event={"ID":"aa63c716-c2fc-4f3f-8d6e-145452326e5c","Type":"ContainerDied","Data":"8f8e4e84ce67b435e53dbe0c186842c13c58cdf7bf33d369a2e0f59b1d1a1a5d"}
Sep 04 22:13:27.466875 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:27.466827    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9"
Sep 04 22:13:27.466875 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:27.466875    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9" event={"ID":"aa63c716-c2fc-4f3f-8d6e-145452326e5c","Type":"ContainerDied","Data":"6752ed65575c05926c1b77ebee11f2d000c8db2ca93f716c788c545d7e86bf87"}
Sep 04 22:13:27.467203 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:27.466897    2557 scope.go:117] "RemoveContainer" containerID="8f8e4e84ce67b435e53dbe0c186842c13c58cdf7bf33d369a2e0f59b1d1a1a5d"
Sep 04 22:13:27.476724 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:27.476698    2557 scope.go:117] "RemoveContainer" containerID="8f8e4e84ce67b435e53dbe0c186842c13c58cdf7bf33d369a2e0f59b1d1a1a5d"
Sep 04 22:13:27.477049 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:13:27.477028    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f8e4e84ce67b435e53dbe0c186842c13c58cdf7bf33d369a2e0f59b1d1a1a5d\": container with ID starting with 8f8e4e84ce67b435e53dbe0c186842c13c58cdf7bf33d369a2e0f59b1d1a1a5d not found: ID does not exist" containerID="8f8e4e84ce67b435e53dbe0c186842c13c58cdf7bf33d369a2e0f59b1d1a1a5d"
Sep 04 22:13:27.477104 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:27.477066    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f8e4e84ce67b435e53dbe0c186842c13c58cdf7bf33d369a2e0f59b1d1a1a5d"} err="failed to get container status \"8f8e4e84ce67b435e53dbe0c186842c13c58cdf7bf33d369a2e0f59b1d1a1a5d\": rpc error: code = NotFound desc = could not find container \"8f8e4e84ce67b435e53dbe0c186842c13c58cdf7bf33d369a2e0f59b1d1a1a5d\": container with ID starting with 8f8e4e84ce67b435e53dbe0c186842c13c58cdf7bf33d369a2e0f59b1d1a1a5d not found: ID does not exist"
Sep 04 22:13:27.488771 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:27.488733    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9"]
Sep 04 22:13:27.494935 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:27.494898    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-bba28942-7bc6b77994-gzlc9"]
Sep 04 22:13:28.352920 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:28.352882    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fa95dce-01ec-46e0-92e6-aeff556495dc" path="/var/lib/kubelet/pods/1fa95dce-01ec-46e0-92e6-aeff556495dc/volumes"
Sep 04 22:13:28.353300 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:28.353246    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa63c716-c2fc-4f3f-8d6e-145452326e5c" path="/var/lib/kubelet/pods/aa63c716-c2fc-4f3f-8d6e-145452326e5c/volumes"
Sep 04 22:13:35.194974 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:35.194928    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"]
Sep 04 22:13:37.151452 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:37.151391    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:13:49.950567 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:49.950527    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:13:50.215566 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.215438    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2" podUID="2b41afb0-00ee-4f85-8624-6e6c90631410" containerName="main" containerID="cri-o://f92d0159d06e80cb35dd3b2eb407bf021a0262031101a8f0eecb537dc8c2cf97" gracePeriod=45
Sep 04 22:13:50.471731 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.471659    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:13:50.516740 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.516706    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-home\") pod \"2b41afb0-00ee-4f85-8624-6e6c90631410\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") "
Sep 04 22:13:50.516944 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.516750    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-model-cache\") pod \"2b41afb0-00ee-4f85-8624-6e6c90631410\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") "
Sep 04 22:13:50.516944 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.516765    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-tmp-dir\") pod \"2b41afb0-00ee-4f85-8624-6e6c90631410\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") "
Sep 04 22:13:50.516944 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.516809    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfnxp\" (UniqueName: \"kubernetes.io/projected/2b41afb0-00ee-4f85-8624-6e6c90631410-kube-api-access-dfnxp\") pod \"2b41afb0-00ee-4f85-8624-6e6c90631410\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") "
Sep 04 22:13:50.516944 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.516880    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-dshm\") pod \"2b41afb0-00ee-4f85-8624-6e6c90631410\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") "
Sep 04 22:13:50.517160 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.517042    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-model-cache" (OuterVolumeSpecName: "model-cache") pod "2b41afb0-00ee-4f85-8624-6e6c90631410" (UID: "2b41afb0-00ee-4f85-8624-6e6c90631410"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:13:50.517160 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.517057    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-home" (OuterVolumeSpecName: "home") pod "2b41afb0-00ee-4f85-8624-6e6c90631410" (UID: "2b41afb0-00ee-4f85-8624-6e6c90631410"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:13:50.517160 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.517107    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2b41afb0-00ee-4f85-8624-6e6c90631410" (UID: "2b41afb0-00ee-4f85-8624-6e6c90631410"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:13:50.517271 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.517209    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-home\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:13:50.517271 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.517226    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-model-cache\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:13:50.517271 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.517242    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-tmp-dir\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:13:50.519102 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.519067    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-dshm" (OuterVolumeSpecName: "dshm") pod "2b41afb0-00ee-4f85-8624-6e6c90631410" (UID: "2b41afb0-00ee-4f85-8624-6e6c90631410"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:13:50.519220 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.519192    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b41afb0-00ee-4f85-8624-6e6c90631410-kube-api-access-dfnxp" (OuterVolumeSpecName: "kube-api-access-dfnxp") pod "2b41afb0-00ee-4f85-8624-6e6c90631410" (UID: "2b41afb0-00ee-4f85-8624-6e6c90631410"). InnerVolumeSpecName "kube-api-access-dfnxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:13:50.597495 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.597461    2557 generic.go:356] "Generic (PLEG): container finished" podID="2b41afb0-00ee-4f85-8624-6e6c90631410" containerID="f92d0159d06e80cb35dd3b2eb407bf021a0262031101a8f0eecb537dc8c2cf97" exitCode=0
Sep 04 22:13:50.597495 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.597506    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"
Sep 04 22:13:50.597495 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.597512    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2" event={"ID":"2b41afb0-00ee-4f85-8624-6e6c90631410","Type":"ContainerDied","Data":"f92d0159d06e80cb35dd3b2eb407bf021a0262031101a8f0eecb537dc8c2cf97"}
Sep 04 22:13:50.597796 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.597544    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2" event={"ID":"2b41afb0-00ee-4f85-8624-6e6c90631410","Type":"ContainerDied","Data":"673eab2d1b79e9825fc6d01e4d98041c0e3bce57ba12360afa01b27ee9e23229"}
Sep 04 22:13:50.597796 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.597560    2557 scope.go:117] "RemoveContainer" containerID="f92d0159d06e80cb35dd3b2eb407bf021a0262031101a8f0eecb537dc8c2cf97"
Sep 04 22:13:50.608978 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.608952    2557 scope.go:117] "RemoveContainer" containerID="f92d0159d06e80cb35dd3b2eb407bf021a0262031101a8f0eecb537dc8c2cf97"
Sep 04 22:13:50.609339 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:13:50.609316    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f92d0159d06e80cb35dd3b2eb407bf021a0262031101a8f0eecb537dc8c2cf97\": container with ID starting with f92d0159d06e80cb35dd3b2eb407bf021a0262031101a8f0eecb537dc8c2cf97 not found: ID does not exist" containerID="f92d0159d06e80cb35dd3b2eb407bf021a0262031101a8f0eecb537dc8c2cf97"
Sep 04 22:13:50.609398 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.609349    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f92d0159d06e80cb35dd3b2eb407bf021a0262031101a8f0eecb537dc8c2cf97"} err="failed to get container status \"f92d0159d06e80cb35dd3b2eb407bf021a0262031101a8f0eecb537dc8c2cf97\": rpc error: code = NotFound desc = could not find container \"f92d0159d06e80cb35dd3b2eb407bf021a0262031101a8f0eecb537dc8c2cf97\": container with ID starting with f92d0159d06e80cb35dd3b2eb407bf021a0262031101a8f0eecb537dc8c2cf97 not found: ID does not exist"
Sep 04 22:13:50.617686 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.617653    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b41afb0-00ee-4f85-8624-6e6c90631410-tls-certs\") pod \"2b41afb0-00ee-4f85-8624-6e6c90631410\" (UID: \"2b41afb0-00ee-4f85-8624-6e6c90631410\") "
Sep 04 22:13:50.618030 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.618007    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfnxp\" (UniqueName: \"kubernetes.io/projected/2b41afb0-00ee-4f85-8624-6e6c90631410-kube-api-access-dfnxp\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:13:50.618114 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.618034    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2b41afb0-00ee-4f85-8624-6e6c90631410-dshm\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:13:50.623744 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.623700    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b41afb0-00ee-4f85-8624-6e6c90631410-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2b41afb0-00ee-4f85-8624-6e6c90631410" (UID: "2b41afb0-00ee-4f85-8624-6e6c90631410"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:13:50.718545 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.718508    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2b41afb0-00ee-4f85-8624-6e6c90631410-tls-certs\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:13:50.921037 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.920998    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"]
Sep 04 22:13:50.931152 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:50.931112    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-bba28942/e2e-lifecycle-model-bba28942-kserve-68bcb644d7-bvst2"]
Sep 04 22:13:52.353329 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:13:52.353293    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b41afb0-00ee-4f85-8624-6e6c90631410" path="/var/lib/kubelet/pods/2b41afb0-00ee-4f85-8624-6e6c90631410/volumes"
Sep 04 22:14:10.160206 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:10.160170    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:14:24.453514 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:24.453472    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:14:33.641835 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.641795    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"]
Sep 04 22:14:33.642466 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.642444    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b41afb0-00ee-4f85-8624-6e6c90631410" containerName="main"
Sep 04 22:14:33.642466 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.642468    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b41afb0-00ee-4f85-8624-6e6c90631410" containerName="main"
Sep 04 22:14:33.642635 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.642507    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fa95dce-01ec-46e0-92e6-aeff556495dc" containerName="revoke-keys"
Sep 04 22:14:33.642635 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.642516    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fa95dce-01ec-46e0-92e6-aeff556495dc" containerName="revoke-keys"
Sep 04 22:14:33.642635 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.642531    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa63c716-c2fc-4f3f-8d6e-145452326e5c" containerName="payload-pre-processing"
Sep 04 22:14:33.642635 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.642540    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa63c716-c2fc-4f3f-8d6e-145452326e5c" containerName="payload-pre-processing"
Sep 04 22:14:33.642834 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.642650    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa63c716-c2fc-4f3f-8d6e-145452326e5c" containerName="payload-pre-processing"
Sep 04 22:14:33.642834 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.642664    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b41afb0-00ee-4f85-8624-6e6c90631410" containerName="main"
Sep 04 22:14:33.642834 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.642677    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fa95dce-01ec-46e0-92e6-aeff556495dc" containerName="revoke-keys"
Sep 04 22:14:33.646228 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.646200    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:33.649491 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.649456    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-6680c7\"/\"openshift-service-ca.crt\""
Sep 04 22:14:33.649491 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.649476    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-6680c7\"/\"rate-test-model-6680c7-kserve-self-signed-certs\""
Sep 04 22:14:33.649751 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.649460    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-6680c7\"/\"default-dockercfg-lxjlt\""
Sep 04 22:14:33.649751 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.649476    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-6680c7\"/\"kube-root-ca.crt\""
Sep 04 22:14:33.660085 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.660045    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"]
Sep 04 22:14:33.717495 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.717454    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-tmp-dir\") pod \"rate-test-model-6680c7-kserve-54f4fb9899-rgqgf\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:33.717495 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.717500    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-dshm\") pod \"rate-test-model-6680c7-kserve-54f4fb9899-rgqgf\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:33.717709 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.717556    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-home\") pod \"rate-test-model-6680c7-kserve-54f4fb9899-rgqgf\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:33.717709 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.717628    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-model-cache\") pod \"rate-test-model-6680c7-kserve-54f4fb9899-rgqgf\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:33.717709 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.717662    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e7a0497-a55c-4300-8b67-2334041f5e4c-tls-certs\") pod \"rate-test-model-6680c7-kserve-54f4fb9899-rgqgf\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:33.717813 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.717721    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvhcn\" (UniqueName: \"kubernetes.io/projected/9e7a0497-a55c-4300-8b67-2334041f5e4c-kube-api-access-pvhcn\") pod \"rate-test-model-6680c7-kserve-54f4fb9899-rgqgf\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:33.818448 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.818373    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvhcn\" (UniqueName: \"kubernetes.io/projected/9e7a0497-a55c-4300-8b67-2334041f5e4c-kube-api-access-pvhcn\") pod \"rate-test-model-6680c7-kserve-54f4fb9899-rgqgf\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:33.818664 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.818485    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-tmp-dir\") pod \"rate-test-model-6680c7-kserve-54f4fb9899-rgqgf\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:33.818664 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.818505    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-dshm\") pod \"rate-test-model-6680c7-kserve-54f4fb9899-rgqgf\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:33.818664 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.818543    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-home\") pod \"rate-test-model-6680c7-kserve-54f4fb9899-rgqgf\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:33.818851 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.818675    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-model-cache\") pod \"rate-test-model-6680c7-kserve-54f4fb9899-rgqgf\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:33.818851 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.818724    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e7a0497-a55c-4300-8b67-2334041f5e4c-tls-certs\") pod \"rate-test-model-6680c7-kserve-54f4fb9899-rgqgf\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:33.818937 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.818913    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-tmp-dir\") pod \"rate-test-model-6680c7-kserve-54f4fb9899-rgqgf\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:33.818981 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.818929    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-home\") pod \"rate-test-model-6680c7-kserve-54f4fb9899-rgqgf\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:33.819075 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.819060    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-model-cache\") pod \"rate-test-model-6680c7-kserve-54f4fb9899-rgqgf\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:33.821022 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.820999    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-dshm\") pod \"rate-test-model-6680c7-kserve-54f4fb9899-rgqgf\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:33.821300 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.821279    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e7a0497-a55c-4300-8b67-2334041f5e4c-tls-certs\") pod \"rate-test-model-6680c7-kserve-54f4fb9899-rgqgf\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:33.827389 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.827355    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvhcn\" (UniqueName: \"kubernetes.io/projected/9e7a0497-a55c-4300-8b67-2334041f5e4c-kube-api-access-pvhcn\") pod \"rate-test-model-6680c7-kserve-54f4fb9899-rgqgf\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") " pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:33.956953 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:33.956854    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:34.232878 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:34.232848    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"]
Sep 04 22:14:34.234463 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:14:34.234405    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e7a0497_a55c_4300_8b67_2334041f5e4c.slice/crio-34181cd9775d167b2ab252a0fa4882c97e77cae45284c0b48ceae36f0e1e7dd1 WatchSource:0}: Error finding container 34181cd9775d167b2ab252a0fa4882c97e77cae45284c0b48ceae36f0e1e7dd1: Status 404 returned error can't find the container with id 34181cd9775d167b2ab252a0fa4882c97e77cae45284c0b48ceae36f0e1e7dd1
Sep 04 22:14:34.452663 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:34.452630    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:14:34.849967 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:34.849922    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf" event={"ID":"9e7a0497-a55c-4300-8b67-2334041f5e4c","Type":"ContainerStarted","Data":"52a45653f58a829baa93200227e57da188bcadbac454d8d3c1988cd2026e1f23"}
Sep 04 22:14:34.849967 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:34.849975    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf" event={"ID":"9e7a0497-a55c-4300-8b67-2334041f5e4c","Type":"ContainerStarted","Data":"34181cd9775d167b2ab252a0fa4882c97e77cae45284c0b48ceae36f0e1e7dd1"}
Sep 04 22:14:34.869102 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:34.869044    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf" podStartSLOduration=1.8690238909999999 podStartE2EDuration="1.869023891s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:14:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:14:34.866157512 +0000 UTC m=+1249.182724250" watchObservedRunningTime="2026-09-04 22:14:34.869023891 +0000 UTC m=+1249.185590632"
Sep 04 22:14:43.543956 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:43.543873    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:14:43.957694 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:43.957595    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:43.957694 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:43.957651    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:43.970443 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:43.970392    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:44.915781 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:44.915743    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:14:58.254558 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:14:58.254516    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:15:00.141117 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:00.141079    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29809335-cxzkw"]
Sep 04 22:15:00.146068 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:00.146043    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7a26c9-29809335-tgggm"]
Sep 04 22:15:00.146212 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:00.146159    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809335-cxzkw"
Sep 04 22:15:00.148982 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:00.148960    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dftg6\""
Sep 04 22:15:00.149575 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:00.149553    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7a26c9-29809335-tgggm"
Sep 04 22:15:00.154681 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:00.154654    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29809335-cxzkw"]
Sep 04 22:15:00.162310 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:00.162277    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7a26c9-29809335-tgggm"]
Sep 04 22:15:00.163035 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:00.163008    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809335-cxzkw"
Sep 04 22:15:00.170333 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:00.170306    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7a26c9-29809335-tgggm"
Sep 04 22:15:00.458641 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:00.458610    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29809335-cxzkw"]
Sep 04 22:15:00.460732 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:15:00.460697    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda91b6bd3_6244_4b0c_ba43_ec22e4aaf1b5.slice/crio-d607d9d59f45fa8f85f7fb23bd98bad26de0b416441fbcd43c83b3761e9a5090 WatchSource:0}: Error finding container d607d9d59f45fa8f85f7fb23bd98bad26de0b416441fbcd43c83b3761e9a5090: Status 404 returned error can't find the container with id d607d9d59f45fa8f85f7fb23bd98bad26de0b416441fbcd43c83b3761e9a5090
Sep 04 22:15:00.480828 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:00.480795    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7a26c9-29809335-tgggm"]
Sep 04 22:15:00.481607 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:15:00.481577    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7db8920_fded_486e_90ff_d1f1bc6078e8.slice/crio-d0c7cdb696b7037b14e6849b5033fa1e9ce2dc0c69bfef0040a8b2153d6832d7 WatchSource:0}: Error finding container d0c7cdb696b7037b14e6849b5033fa1e9ce2dc0c69bfef0040a8b2153d6832d7: Status 404 returned error can't find the container with id d0c7cdb696b7037b14e6849b5033fa1e9ce2dc0c69bfef0040a8b2153d6832d7
Sep 04 22:15:01.000832 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:01.000790    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7a26c9-29809335-tgggm" event={"ID":"b7db8920-fded-486e-90ff-d1f1bc6078e8","Type":"ContainerStarted","Data":"8088bbd78e97ea5e1e51350fef3be9ea32d76c5a1158bed5cc05458f646bd360"}
Sep 04 22:15:01.001036 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:01.000856    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7a26c9-29809335-tgggm" event={"ID":"b7db8920-fded-486e-90ff-d1f1bc6078e8","Type":"ContainerStarted","Data":"d0c7cdb696b7037b14e6849b5033fa1e9ce2dc0c69bfef0040a8b2153d6832d7"}
Sep 04 22:15:01.002372 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:01.002344    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809335-cxzkw" event={"ID":"a91b6bd3-6244-4b0c-ba43-ec22e4aaf1b5","Type":"ContainerStarted","Data":"b4e8d5ca8df4733e75f6a00ee451feb81c73ea9c141d191edec643469c0b5add"}
Sep 04 22:15:01.002508 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:01.002389    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809335-cxzkw" event={"ID":"a91b6bd3-6244-4b0c-ba43-ec22e4aaf1b5","Type":"ContainerStarted","Data":"d607d9d59f45fa8f85f7fb23bd98bad26de0b416441fbcd43c83b3761e9a5090"}
Sep 04 22:15:01.018768 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:01.018711    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7a26c9-29809335-tgggm" podStartSLOduration=1.018693324 podStartE2EDuration="1.018693324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:15:01.017398471 +0000 UTC m=+1275.333965209" watchObservedRunningTime="2026-09-04 22:15:01.018693324 +0000 UTC m=+1275.335260062"
Sep 04 22:15:01.032753 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:01.032690    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809335-cxzkw" podStartSLOduration=1.032674113 podStartE2EDuration="1.032674113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:15:01.031402646 +0000 UTC m=+1275.347969383" watchObservedRunningTime="2026-09-04 22:15:01.032674113 +0000 UTC m=+1275.349240852"
Sep 04 22:15:02.009736 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:02.009696    2557 generic.go:356] "Generic (PLEG): container finished" podID="b7db8920-fded-486e-90ff-d1f1bc6078e8" containerID="8088bbd78e97ea5e1e51350fef3be9ea32d76c5a1158bed5cc05458f646bd360" exitCode=0
Sep 04 22:15:02.010196 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:02.009788    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7a26c9-29809335-tgggm" event={"ID":"b7db8920-fded-486e-90ff-d1f1bc6078e8","Type":"ContainerDied","Data":"8088bbd78e97ea5e1e51350fef3be9ea32d76c5a1158bed5cc05458f646bd360"}
Sep 04 22:15:02.011402 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:02.011373    2557 generic.go:356] "Generic (PLEG): container finished" podID="a91b6bd3-6244-4b0c-ba43-ec22e4aaf1b5" containerID="b4e8d5ca8df4733e75f6a00ee451feb81c73ea9c141d191edec643469c0b5add" exitCode=0
Sep 04 22:15:02.011556 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:02.011476    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809335-cxzkw" event={"ID":"a91b6bd3-6244-4b0c-ba43-ec22e4aaf1b5","Type":"ContainerDied","Data":"b4e8d5ca8df4733e75f6a00ee451feb81c73ea9c141d191edec643469c0b5add"}
Sep 04 22:15:03.171517 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:03.171489    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7a26c9-29809335-tgggm"
Sep 04 22:15:03.174649 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:03.174622    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809335-cxzkw"
Sep 04 22:15:04.028120 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:04.028086    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7a26c9-29809335-tgggm"
Sep 04 22:15:04.028300 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:04.028090    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7a26c9-29809335-tgggm" event={"ID":"b7db8920-fded-486e-90ff-d1f1bc6078e8","Type":"ContainerDied","Data":"d0c7cdb696b7037b14e6849b5033fa1e9ce2dc0c69bfef0040a8b2153d6832d7"}
Sep 04 22:15:04.028300 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:04.028213    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0c7cdb696b7037b14e6849b5033fa1e9ce2dc0c69bfef0040a8b2153d6832d7"
Sep 04 22:15:04.029572 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:04.029549    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809335-cxzkw"
Sep 04 22:15:04.029684 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:04.029587    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29809335-cxzkw" event={"ID":"a91b6bd3-6244-4b0c-ba43-ec22e4aaf1b5","Type":"ContainerDied","Data":"d607d9d59f45fa8f85f7fb23bd98bad26de0b416441fbcd43c83b3761e9a5090"}
Sep 04 22:15:04.029684 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:04.029626    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d607d9d59f45fa8f85f7fb23bd98bad26de0b416441fbcd43c83b3761e9a5090"
Sep 04 22:15:24.152700 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:24.152593    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:15:39.250059 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:39.250010    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:15:47.017797 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.017759    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27"]
Sep 04 22:15:47.018194 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.018146    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a91b6bd3-6244-4b0c-ba43-ec22e4aaf1b5" containerName="cleanup"
Sep 04 22:15:47.018194 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.018157    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="a91b6bd3-6244-4b0c-ba43-ec22e4aaf1b5" containerName="cleanup"
Sep 04 22:15:47.018194 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.018171    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7db8920-fded-486e-90ff-d1f1bc6078e8" containerName="cleanup"
Sep 04 22:15:47.018194 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.018178    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7db8920-fded-486e-90ff-d1f1bc6078e8" containerName="cleanup"
Sep 04 22:15:47.018323 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.018234    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="a91b6bd3-6244-4b0c-ba43-ec22e4aaf1b5" containerName="cleanup"
Sep 04 22:15:47.018323 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.018243    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7db8920-fded-486e-90ff-d1f1bc6078e8" containerName="cleanup"
Sep 04 22:15:47.021301 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.021273    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27"
Sep 04 22:15:47.023672 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.023650    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-dac33e-dockercfg-phlj8\""
Sep 04 22:15:47.023760 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.023684    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-dac33e\""
Sep 04 22:15:47.033942 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.033913    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27"]
Sep 04 22:15:47.066841 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.066800    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c"]
Sep 04 22:15:47.070220 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.070196    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c"
Sep 04 22:15:47.078825 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.078796    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c"]
Sep 04 22:15:47.105284 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.105243    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrzwk\" (UniqueName: \"kubernetes.io/projected/2ddbe8f6-3d38-4271-9cef-d230cc6bf531-kube-api-access-hrzwk\") pod \"payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27\" (UID: \"2ddbe8f6-3d38-4271-9cef-d230cc6bf531\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27"
Sep 04 22:15:47.105464 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.105403    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ddbe8f6-3d38-4271-9cef-d230cc6bf531-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27\" (UID: \"2ddbe8f6-3d38-4271-9cef-d230cc6bf531\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27"
Sep 04 22:15:47.206203 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.206157    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdj2n\" (UniqueName: \"kubernetes.io/projected/782d444a-e8b9-4c2f-a76f-fce9e99bbabe-kube-api-access-wdj2n\") pod \"payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c\" (UID: \"782d444a-e8b9-4c2f-a76f-fce9e99bbabe\") " pod="openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c"
Sep 04 22:15:47.206404 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.206231    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ddbe8f6-3d38-4271-9cef-d230cc6bf531-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27\" (UID: \"2ddbe8f6-3d38-4271-9cef-d230cc6bf531\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27"
Sep 04 22:15:47.212443 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.206701    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrzwk\" (UniqueName: \"kubernetes.io/projected/2ddbe8f6-3d38-4271-9cef-d230cc6bf531-kube-api-access-hrzwk\") pod \"payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27\" (UID: \"2ddbe8f6-3d38-4271-9cef-d230cc6bf531\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27"
Sep 04 22:15:47.212443 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.207262    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/782d444a-e8b9-4c2f-a76f-fce9e99bbabe-plugins-config-volume\") pod \"payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c\" (UID: \"782d444a-e8b9-4c2f-a76f-fce9e99bbabe\") " pod="openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c"
Sep 04 22:15:47.212443 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.207756    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ddbe8f6-3d38-4271-9cef-d230cc6bf531-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27\" (UID: \"2ddbe8f6-3d38-4271-9cef-d230cc6bf531\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27"
Sep 04 22:15:47.216169 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.216134    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrzwk\" (UniqueName: \"kubernetes.io/projected/2ddbe8f6-3d38-4271-9cef-d230cc6bf531-kube-api-access-hrzwk\") pod \"payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27\" (UID: \"2ddbe8f6-3d38-4271-9cef-d230cc6bf531\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27"
Sep 04 22:15:47.308821 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.308721    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdj2n\" (UniqueName: \"kubernetes.io/projected/782d444a-e8b9-4c2f-a76f-fce9e99bbabe-kube-api-access-wdj2n\") pod \"payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c\" (UID: \"782d444a-e8b9-4c2f-a76f-fce9e99bbabe\") " pod="openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c"
Sep 04 22:15:47.308821 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.308793    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/782d444a-e8b9-4c2f-a76f-fce9e99bbabe-plugins-config-volume\") pod \"payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c\" (UID: \"782d444a-e8b9-4c2f-a76f-fce9e99bbabe\") " pod="openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c"
Sep 04 22:15:47.309489 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.309467    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/782d444a-e8b9-4c2f-a76f-fce9e99bbabe-plugins-config-volume\") pod \"payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c\" (UID: \"782d444a-e8b9-4c2f-a76f-fce9e99bbabe\") " pod="openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c"
Sep 04 22:15:47.316635 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.316608    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdj2n\" (UniqueName: \"kubernetes.io/projected/782d444a-e8b9-4c2f-a76f-fce9e99bbabe-kube-api-access-wdj2n\") pod \"payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c\" (UID: \"782d444a-e8b9-4c2f-a76f-fce9e99bbabe\") " pod="openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c"
Sep 04 22:15:47.333489 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.333438    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27"
Sep 04 22:15:47.383013 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.382969    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c"
Sep 04 22:15:47.608253 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.608216    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27"]
Sep 04 22:15:47.609478 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:15:47.609446    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ddbe8f6_3d38_4271_9cef_d230cc6bf531.slice/crio-9efc8ffd1ebbc243882c88ac27bfa5d9ca3e65738dcd4db45e05fa10070d616e WatchSource:0}: Error finding container 9efc8ffd1ebbc243882c88ac27bfa5d9ca3e65738dcd4db45e05fa10070d616e: Status 404 returned error can't find the container with id 9efc8ffd1ebbc243882c88ac27bfa5d9ca3e65738dcd4db45e05fa10070d616e
Sep 04 22:15:47.669596 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:47.669569    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c"]
Sep 04 22:15:47.671153 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:15:47.671124    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod782d444a_e8b9_4c2f_a76f_fce9e99bbabe.slice/crio-c357e62be6c1bcb31ca4d120b157da68a1423284fbd9c5653f9854c48c70f5ba WatchSource:0}: Error finding container c357e62be6c1bcb31ca4d120b157da68a1423284fbd9c5653f9854c48c70f5ba: Status 404 returned error can't find the container with id c357e62be6c1bcb31ca4d120b157da68a1423284fbd9c5653f9854c48c70f5ba
Sep 04 22:15:48.289737 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:48.289696    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c" event={"ID":"782d444a-e8b9-4c2f-a76f-fce9e99bbabe","Type":"ContainerStarted","Data":"49953a2bf123ef1f0882e85f215339922ffeb5e56f7744cf7a2ae3a3855d282f"}
Sep 04 22:15:48.290228 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:48.289768    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c"
Sep 04 22:15:48.290228 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:48.289792    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c" event={"ID":"782d444a-e8b9-4c2f-a76f-fce9e99bbabe","Type":"ContainerStarted","Data":"c357e62be6c1bcb31ca4d120b157da68a1423284fbd9c5653f9854c48c70f5ba"}
Sep 04 22:15:48.291468 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:48.291436    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27" event={"ID":"2ddbe8f6-3d38-4271-9cef-d230cc6bf531","Type":"ContainerStarted","Data":"428d97cfe1f3052711dae4345bfff085355514c67f6360d9ad53d8c221010c7e"}
Sep 04 22:15:48.291605 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:48.291502    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27" event={"ID":"2ddbe8f6-3d38-4271-9cef-d230cc6bf531","Type":"ContainerStarted","Data":"9efc8ffd1ebbc243882c88ac27bfa5d9ca3e65738dcd4db45e05fa10070d616e"}
Sep 04 22:15:48.291605 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:48.291532    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27"
Sep 04 22:15:48.312108 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:48.312043    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c" podStartSLOduration=1.312021002 podStartE2EDuration="1.312021002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:15:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:15:48.309587967 +0000 UTC m=+1322.626154703" watchObservedRunningTime="2026-09-04 22:15:48.312021002 +0000 UTC m=+1322.628587742"
Sep 04 22:15:48.328942 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:48.328876    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27" podStartSLOduration=1.328857025 podStartE2EDuration="1.328857025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:15:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:15:48.325931686 +0000 UTC m=+1322.642498426" watchObservedRunningTime="2026-09-04 22:15:48.328857025 +0000 UTC m=+1322.645423780"
Sep 04 22:15:53.356620 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:53.356584    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:15:57.120251 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:57.120216    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"]
Sep 04 22:15:58.792630 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:58.792591    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"]
Sep 04 22:15:58.799780 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:58.799745    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:58.802696 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:58.802656    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-cc92e2-openshift-default-dockercfg-ds7hx\""
Sep 04 22:15:58.809733 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:58.809691    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"]
Sep 04 22:15:58.923261 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:58.923218    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-podinfo\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:58.923261 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:58.923267    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-credential-socket\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:58.923516 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:58.923290    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2efcc572-0661-4dd9-98b1-b5816bd4b037-istiod-ca-cert\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:58.923516 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:58.923313    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-token\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:58.923516 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:58.923394    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8gcv\" (UniqueName: \"kubernetes.io/projected/2efcc572-0661-4dd9-98b1-b5816bd4b037-kube-api-access-g8gcv\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:58.923516 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:58.923457    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-data\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:58.923654 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:58.923534    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-envoy\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:58.923654 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:58.923574    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-workload-certs\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:58.923654 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:58.923611    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-workload-socket\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:59.024533 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.024494    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-envoy\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:59.024711 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.024544    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-workload-certs\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:59.024711 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.024659    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-workload-socket\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:59.024790 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.024750    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-podinfo\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:59.024842 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.024791    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-credential-socket\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:59.024842 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.024818    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2efcc572-0661-4dd9-98b1-b5816bd4b037-istiod-ca-cert\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:59.024944 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.024844    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-token\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:59.024944 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.024911    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8gcv\" (UniqueName: \"kubernetes.io/projected/2efcc572-0661-4dd9-98b1-b5816bd4b037-kube-api-access-g8gcv\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:59.025057 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.024943    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-data\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:59.025057 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.025010    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-workload-certs\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:59.025189 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.025151    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-credential-socket\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:59.025381 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.025362    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-data\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:59.025501 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.025478    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-workload-socket\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:59.025565 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.025530    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2efcc572-0661-4dd9-98b1-b5816bd4b037-istiod-ca-cert\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:59.026866 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.026845    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-envoy\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:59.027267 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.027244    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-podinfo\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:59.033060 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.033033    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-token\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:59.033311 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.033286    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8gcv\" (UniqueName: \"kubernetes.io/projected/2efcc572-0661-4dd9-98b1-b5816bd4b037-kube-api-access-g8gcv\") pod \"e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") " pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:59.115251 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.115156    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:15:59.300135 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.300101    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27"
Sep 04 22:15:59.300303 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.300258    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c"
Sep 04 22:15:59.393865 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.393831    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"]
Sep 04 22:15:59.395492 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:15:59.395400    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2efcc572_0661_4dd9_98b1_b5816bd4b037.slice/crio-2278a90f7dda0d44fce7215338cc84a7f88487970d9d038c971604c6511d74be WatchSource:0}: Error finding container 2278a90f7dda0d44fce7215338cc84a7f88487970d9d038c971604c6511d74be: Status 404 returned error can't find the container with id 2278a90f7dda0d44fce7215338cc84a7f88487970d9d038c971604c6511d74be
Sep 04 22:15:59.398255 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.398217    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:15:59.398348 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.398292    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:15:59.398348 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:15:59.398333    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:16:00.359775 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:00.359735    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j" event={"ID":"2efcc572-0661-4dd9-98b1-b5816bd4b037","Type":"ContainerStarted","Data":"8c02e29a1e4330ef9a790785b9104f53c5702b32cd4124a82bde8433194b010c"}
Sep 04 22:16:00.360195 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:00.359793    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j" event={"ID":"2efcc572-0661-4dd9-98b1-b5816bd4b037","Type":"ContainerStarted","Data":"2278a90f7dda0d44fce7215338cc84a7f88487970d9d038c971604c6511d74be"}
Sep 04 22:16:00.381472 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:00.381403    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j" podStartSLOduration=2.381386076 podStartE2EDuration="2.381386076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:15:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:16:00.378559916 +0000 UTC m=+1334.695126656" watchObservedRunningTime="2026-09-04 22:16:00.381386076 +0000 UTC m=+1334.697952847"
Sep 04 22:16:01.115757 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:01.115716    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:16:01.121295 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:01.121263    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:16:01.365675 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:01.365643    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:16:01.366663 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:01.366596    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:16:12.056315 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.056229    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:16:12.140603 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.140558    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf" podUID="9e7a0497-a55c-4300-8b67-2334041f5e4c" containerName="main" containerID="cri-o://52a45653f58a829baa93200227e57da188bcadbac454d8d3c1988cd2026e1f23" gracePeriod=45
Sep 04 22:16:12.404237 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.404203    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:16:12.434246 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.434212    2557 generic.go:356] "Generic (PLEG): container finished" podID="9e7a0497-a55c-4300-8b67-2334041f5e4c" containerID="52a45653f58a829baa93200227e57da188bcadbac454d8d3c1988cd2026e1f23" exitCode=0
Sep 04 22:16:12.434451 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.434250    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"
Sep 04 22:16:12.434451 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.434254    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf" event={"ID":"9e7a0497-a55c-4300-8b67-2334041f5e4c","Type":"ContainerDied","Data":"52a45653f58a829baa93200227e57da188bcadbac454d8d3c1988cd2026e1f23"}
Sep 04 22:16:12.434451 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.434295    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf" event={"ID":"9e7a0497-a55c-4300-8b67-2334041f5e4c","Type":"ContainerDied","Data":"34181cd9775d167b2ab252a0fa4882c97e77cae45284c0b48ceae36f0e1e7dd1"}
Sep 04 22:16:12.434451 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.434315    2557 scope.go:117] "RemoveContainer" containerID="52a45653f58a829baa93200227e57da188bcadbac454d8d3c1988cd2026e1f23"
Sep 04 22:16:12.443780 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.443753    2557 scope.go:117] "RemoveContainer" containerID="52a45653f58a829baa93200227e57da188bcadbac454d8d3c1988cd2026e1f23"
Sep 04 22:16:12.444155 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:16:12.444133    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52a45653f58a829baa93200227e57da188bcadbac454d8d3c1988cd2026e1f23\": container with ID starting with 52a45653f58a829baa93200227e57da188bcadbac454d8d3c1988cd2026e1f23 not found: ID does not exist" containerID="52a45653f58a829baa93200227e57da188bcadbac454d8d3c1988cd2026e1f23"
Sep 04 22:16:12.444258 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.444162    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52a45653f58a829baa93200227e57da188bcadbac454d8d3c1988cd2026e1f23"} err="failed to get container status \"52a45653f58a829baa93200227e57da188bcadbac454d8d3c1988cd2026e1f23\": rpc error: code = NotFound desc = could not find container \"52a45653f58a829baa93200227e57da188bcadbac454d8d3c1988cd2026e1f23\": container with ID starting with 52a45653f58a829baa93200227e57da188bcadbac454d8d3c1988cd2026e1f23 not found: ID does not exist"
Sep 04 22:16:12.450019 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.449992    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-tmp-dir\") pod \"9e7a0497-a55c-4300-8b67-2334041f5e4c\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") "
Sep 04 22:16:12.450145 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.450069    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-model-cache\") pod \"9e7a0497-a55c-4300-8b67-2334041f5e4c\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") "
Sep 04 22:16:12.450145 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.450112    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-home\") pod \"9e7a0497-a55c-4300-8b67-2334041f5e4c\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") "
Sep 04 22:16:12.450145 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.450131    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e7a0497-a55c-4300-8b67-2334041f5e4c-tls-certs\") pod \"9e7a0497-a55c-4300-8b67-2334041f5e4c\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") "
Sep 04 22:16:12.450355 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.450167    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvhcn\" (UniqueName: \"kubernetes.io/projected/9e7a0497-a55c-4300-8b67-2334041f5e4c-kube-api-access-pvhcn\") pod \"9e7a0497-a55c-4300-8b67-2334041f5e4c\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") "
Sep 04 22:16:12.450355 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.450272    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9e7a0497-a55c-4300-8b67-2334041f5e4c" (UID: "9e7a0497-a55c-4300-8b67-2334041f5e4c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:16:12.450355 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.450302    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-model-cache" (OuterVolumeSpecName: "model-cache") pod "9e7a0497-a55c-4300-8b67-2334041f5e4c" (UID: "9e7a0497-a55c-4300-8b67-2334041f5e4c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:16:12.450355 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.450316    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-dshm\") pod \"9e7a0497-a55c-4300-8b67-2334041f5e4c\" (UID: \"9e7a0497-a55c-4300-8b67-2334041f5e4c\") "
Sep 04 22:16:12.450542 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.450392    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-home" (OuterVolumeSpecName: "home") pod "9e7a0497-a55c-4300-8b67-2334041f5e4c" (UID: "9e7a0497-a55c-4300-8b67-2334041f5e4c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:16:12.450757 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.450730    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-model-cache\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:16:12.450757 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.450758    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-home\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:16:12.450949 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.450772    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-tmp-dir\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:16:12.452644 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.452613    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-dshm" (OuterVolumeSpecName: "dshm") pod "9e7a0497-a55c-4300-8b67-2334041f5e4c" (UID: "9e7a0497-a55c-4300-8b67-2334041f5e4c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:16:12.452644 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.452629    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e7a0497-a55c-4300-8b67-2334041f5e4c-kube-api-access-pvhcn" (OuterVolumeSpecName: "kube-api-access-pvhcn") pod "9e7a0497-a55c-4300-8b67-2334041f5e4c" (UID: "9e7a0497-a55c-4300-8b67-2334041f5e4c"). InnerVolumeSpecName "kube-api-access-pvhcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:16:12.452782 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.452646    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e7a0497-a55c-4300-8b67-2334041f5e4c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9e7a0497-a55c-4300-8b67-2334041f5e4c" (UID: "9e7a0497-a55c-4300-8b67-2334041f5e4c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:16:12.552272 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.552225    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e7a0497-a55c-4300-8b67-2334041f5e4c-tls-certs\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:16:12.552272 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.552266    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvhcn\" (UniqueName: \"kubernetes.io/projected/9e7a0497-a55c-4300-8b67-2334041f5e4c-kube-api-access-pvhcn\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:16:12.552272 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.552277    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e7a0497-a55c-4300-8b67-2334041f5e4c-dshm\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:16:12.758778 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.758729    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"]
Sep 04 22:16:12.763056 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:12.762983    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6680c7/rate-test-model-6680c7-kserve-54f4fb9899-rgqgf"]
Sep 04 22:16:13.144179 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:13.144137    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-25d966-8wl2n"]
Sep 04 22:16:13.144592 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:13.144579    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e7a0497-a55c-4300-8b67-2334041f5e4c" containerName="main"
Sep 04 22:16:13.144636 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:13.144593    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e7a0497-a55c-4300-8b67-2334041f5e4c" containerName="main"
Sep 04 22:16:13.144671 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:13.144666    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e7a0497-a55c-4300-8b67-2334041f5e4c" containerName="main"
Sep 04 22:16:13.149358 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:13.149337    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-25d966-8wl2n"
Sep 04 22:16:13.151888 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:13.151856    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 04 22:16:13.152065 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:13.151857    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dftg6\""
Sep 04 22:16:13.154396 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:13.154369    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-25d966-8wl2n"]
Sep 04 22:16:13.260344 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:13.260298    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4c3b9f6-f562-4b7b-a56d-be2b283db67d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-25d966-8wl2n\" (UID: \"b4c3b9f6-f562-4b7b-a56d-be2b283db67d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-25d966-8wl2n"
Sep 04 22:16:13.361182 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:13.361137    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4c3b9f6-f562-4b7b-a56d-be2b283db67d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-25d966-8wl2n\" (UID: \"b4c3b9f6-f562-4b7b-a56d-be2b283db67d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-25d966-8wl2n"
Sep 04 22:16:13.361842 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:13.361813    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4c3b9f6-f562-4b7b-a56d-be2b283db67d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-25d966-8wl2n\" (UID: \"b4c3b9f6-f562-4b7b-a56d-be2b283db67d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-25d966-8wl2n"
Sep 04 22:16:13.462286 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:13.462189    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-25d966-8wl2n"
Sep 04 22:16:13.739954 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:13.739924    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-25d966-8wl2n"]
Sep 04 22:16:13.741714 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:16:13.741688    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4c3b9f6_f562_4b7b_a56d_be2b283db67d.slice/crio-25214d349961b7d4ae919d390c5ba73ca4e33cc37959a021e3834377dcfe15df WatchSource:0}: Error finding container 25214d349961b7d4ae919d390c5ba73ca4e33cc37959a021e3834377dcfe15df: Status 404 returned error can't find the container with id 25214d349961b7d4ae919d390c5ba73ca4e33cc37959a021e3834377dcfe15df
Sep 04 22:16:14.353487 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:14.353451    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e7a0497-a55c-4300-8b67-2334041f5e4c" path="/var/lib/kubelet/pods/9e7a0497-a55c-4300-8b67-2334041f5e4c/volumes"
Sep 04 22:16:14.455909 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:14.455869    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-25d966-8wl2n" event={"ID":"b4c3b9f6-f562-4b7b-a56d-be2b283db67d","Type":"ContainerStarted","Data":"71441e6b929ecc27224a27bc5fab79c2e30a672ae30de4024a90ac2fd30ee77b"}
Sep 04 22:16:14.456086 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:14.455951    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-25d966-8wl2n" event={"ID":"b4c3b9f6-f562-4b7b-a56d-be2b283db67d","Type":"ContainerStarted","Data":"25214d349961b7d4ae919d390c5ba73ca4e33cc37959a021e3834377dcfe15df"}
Sep 04 22:16:14.475006 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:14.474949    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-25d966-8wl2n" podStartSLOduration=1.474932981 podStartE2EDuration="1.474932981s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:16:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:16:14.473445636 +0000 UTC m=+1348.790012373" watchObservedRunningTime="2026-09-04 22:16:14.474932981 +0000 UTC m=+1348.791499718"
Sep 04 22:16:15.463088 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:15.463049    2557 generic.go:356] "Generic (PLEG): container finished" podID="b4c3b9f6-f562-4b7b-a56d-be2b283db67d" containerID="71441e6b929ecc27224a27bc5fab79c2e30a672ae30de4024a90ac2fd30ee77b" exitCode=0
Sep 04 22:16:15.463088 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:15.463091    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-25d966-8wl2n" event={"ID":"b4c3b9f6-f562-4b7b-a56d-be2b283db67d","Type":"ContainerDied","Data":"71441e6b929ecc27224a27bc5fab79c2e30a672ae30de4024a90ac2fd30ee77b"}
Sep 04 22:16:16.640497 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:16.640463    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-25d966-8wl2n"
Sep 04 22:16:16.709567 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:16.709526    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4c3b9f6-f562-4b7b-a56d-be2b283db67d-maas-api-service-ca\") pod \"b4c3b9f6-f562-4b7b-a56d-be2b283db67d\" (UID: \"b4c3b9f6-f562-4b7b-a56d-be2b283db67d\") "
Sep 04 22:16:16.709924 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:16.709898    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4c3b9f6-f562-4b7b-a56d-be2b283db67d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b4c3b9f6-f562-4b7b-a56d-be2b283db67d" (UID: "b4c3b9f6-f562-4b7b-a56d-be2b283db67d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:16:16.710006 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:16.709991    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4c3b9f6-f562-4b7b-a56d-be2b283db67d-maas-api-service-ca\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:16:17.475793 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:17.475760    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-25d966-8wl2n"
Sep 04 22:16:17.476013 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:17.475805    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-25d966-8wl2n" event={"ID":"b4c3b9f6-f562-4b7b-a56d-be2b283db67d","Type":"ContainerDied","Data":"25214d349961b7d4ae919d390c5ba73ca4e33cc37959a021e3834377dcfe15df"}
Sep 04 22:16:17.476013 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:17.475857    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="25214d349961b7d4ae919d390c5ba73ca4e33cc37959a021e3834377dcfe15df"
Sep 04 22:16:22.508750 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:22.508714    2557 generic.go:356] "Generic (PLEG): container finished" podID="2efcc572-0661-4dd9-98b1-b5816bd4b037" containerID="8c02e29a1e4330ef9a790785b9104f53c5702b32cd4124a82bde8433194b010c" exitCode=0
Sep 04 22:16:22.509167 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:22.508786    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j" event={"ID":"2efcc572-0661-4dd9-98b1-b5816bd4b037","Type":"ContainerDied","Data":"8c02e29a1e4330ef9a790785b9104f53c5702b32cd4124a82bde8433194b010c"}
Sep 04 22:16:22.509210 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:22.509192    2557 scope.go:117] "RemoveContainer" containerID="8c02e29a1e4330ef9a790785b9104f53c5702b32cd4124a82bde8433194b010c"
Sep 04 22:16:22.510435 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:22.510391    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:16:22.510503 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:22.510485    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:16:22.510540 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:22.510520    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:16:23.516485 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:23.516447    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j" event={"ID":"2efcc572-0661-4dd9-98b1-b5816bd4b037","Type":"ContainerStarted","Data":"0f6b69adf3d1256cdc402b50f5cd9d76ca6c55bd7c66ddd0c57d4ae908dc45fb"}
Sep 04 22:16:24.116267 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:24.116158    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:16:24.116713 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:24.116586    2557 patch_prober.go:28] interesting pod/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 04 22:16:24.116968 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:24.116885    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j" podUID="2efcc572-0661-4dd9-98b1-b5816bd4b037" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 04 22:16:25.115824 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:25.115780    2557 patch_prober.go:28] interesting pod/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 04 22:16:25.116230 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:25.115854    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j" podUID="2efcc572-0661-4dd9-98b1-b5816bd4b037" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 04 22:16:26.116099 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:26.116052    2557 patch_prober.go:28] interesting pod/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 04 22:16:26.116553 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:26.116114    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j" podUID="2efcc572-0661-4dd9-98b1-b5816bd4b037" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 04 22:16:27.116633 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:27.116589    2557 patch_prober.go:28] interesting pod/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 04 22:16:27.117051 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:27.116655    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j" podUID="2efcc572-0661-4dd9-98b1-b5816bd4b037" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 04 22:16:28.116271 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.116228    2557 patch_prober.go:28] interesting pod/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Sep 04 22:16:28.116601 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.116293    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j" podUID="2efcc572-0661-4dd9-98b1-b5816bd4b037" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Sep 04 22:16:28.269987 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.269945    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"]
Sep 04 22:16:28.270537 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.270515    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4c3b9f6-f562-4b7b-a56d-be2b283db67d" containerName="revoke-keys"
Sep 04 22:16:28.270537 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.270537    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4c3b9f6-f562-4b7b-a56d-be2b283db67d" containerName="revoke-keys"
Sep 04 22:16:28.270721 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.270631    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4c3b9f6-f562-4b7b-a56d-be2b283db67d" containerName="revoke-keys"
Sep 04 22:16:28.272817 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.272793    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.275354 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.275329    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-f2fbdf-openshift-default-dockercfg-zs9jh\""
Sep 04 22:16:28.292974 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.292932    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"]
Sep 04 22:16:28.434662 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.434568    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c93d0e5c-af64-4138-9542-13b6c1042254-istio-token\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.434662 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.434608    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-istio-data\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.434662 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.434624    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c93d0e5c-af64-4138-9542-13b6c1042254-istio-podinfo\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.434662 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.434648    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-workload-certs\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.434922 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.434732    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c93d0e5c-af64-4138-9542-13b6c1042254-istiod-ca-cert\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.434922 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.434764    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-workload-socket\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.434922 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.434830    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-credential-socket\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.434922 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.434849    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-istio-envoy\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.435054 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.434939    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqwvw\" (UniqueName: \"kubernetes.io/projected/c93d0e5c-af64-4138-9542-13b6c1042254-kube-api-access-mqwvw\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.535680 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.535641    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqwvw\" (UniqueName: \"kubernetes.io/projected/c93d0e5c-af64-4138-9542-13b6c1042254-kube-api-access-mqwvw\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.535680 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.535692    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c93d0e5c-af64-4138-9542-13b6c1042254-istio-token\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.535951 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.535722    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-istio-data\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.535951 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.535741    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c93d0e5c-af64-4138-9542-13b6c1042254-istio-podinfo\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.535951 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.535767    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-workload-certs\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.535951 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.535800    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c93d0e5c-af64-4138-9542-13b6c1042254-istiod-ca-cert\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.535951 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.535818    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-workload-socket\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.535951 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.535871    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-credential-socket\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.535951 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.535885    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-istio-envoy\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.536796 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.536727    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-istio-data\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.537036 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.536935    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-workload-socket\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.537036 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.536990    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-credential-socket\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.537217 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.537149    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-workload-certs\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.537217 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.537184    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c93d0e5c-af64-4138-9542-13b6c1042254-istiod-ca-cert\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.539192 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.539150    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-istio-envoy\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.539581 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.539555    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c93d0e5c-af64-4138-9542-13b6c1042254-istio-podinfo\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.549717 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.548857    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c93d0e5c-af64-4138-9542-13b6c1042254-istio-token\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.549717 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.549624    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqwvw\" (UniqueName: \"kubernetes.io/projected/c93d0e5c-af64-4138-9542-13b6c1042254-kube-api-access-mqwvw\") pod \"e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") " pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.596902 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.596812    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:28.787250 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.787185    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-25d966-8wl2n"]
Sep 04 22:16:28.789995 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.789958    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-25d966-8wl2n"]
Sep 04 22:16:28.898498 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:16:28.898403    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc93d0e5c_af64_4138_9542_13b6c1042254.slice/crio-ddc1bfccb797d9a8f03fdac05272dca8443d4ce76169010ed2f5a17d1dcb4afb WatchSource:0}: Error finding container ddc1bfccb797d9a8f03fdac05272dca8443d4ce76169010ed2f5a17d1dcb4afb: Status 404 returned error can't find the container with id ddc1bfccb797d9a8f03fdac05272dca8443d4ce76169010ed2f5a17d1dcb4afb
Sep 04 22:16:28.898673 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.898545    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"]
Sep 04 22:16:28.901482 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.901439    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:16:28.901627 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.901521    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:16:28.901627 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:28.901554    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:16:29.160452 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:29.160341    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:16:29.160889 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:29.160866    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:16:29.163293 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:29.163268    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:16:29.553151 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:29.553115    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr" event={"ID":"c93d0e5c-af64-4138-9542-13b6c1042254","Type":"ContainerStarted","Data":"a985b151e803fbe1bce0897c228e41ec24a7cf4f4150170d3485e95aaf383a6e"}
Sep 04 22:16:29.553577 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:29.553180    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr" event={"ID":"c93d0e5c-af64-4138-9542-13b6c1042254","Type":"ContainerStarted","Data":"ddc1bfccb797d9a8f03fdac05272dca8443d4ce76169010ed2f5a17d1dcb4afb"}
Sep 04 22:16:29.574298 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:29.574240    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr" podStartSLOduration=1.5742198429999998 podStartE2EDuration="1.574219843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:16:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:16:29.572931186 +0000 UTC m=+1363.889497935" watchObservedRunningTime="2026-09-04 22:16:29.574219843 +0000 UTC m=+1363.890786626"
Sep 04 22:16:29.596935 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:29.596895    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:29.601821 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:29.601794    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:30.353385 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:30.353349    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4c3b9f6-f562-4b7b-a56d-be2b283db67d" path="/var/lib/kubelet/pods/b4c3b9f6-f562-4b7b-a56d-be2b283db67d/volumes"
Sep 04 22:16:30.559401 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:30.559367    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:30.560076 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:30.560055    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:33.646405 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:33.646356    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:16:35.888585 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:35.888536    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac33e-7rd82"]
Sep 04 22:16:35.891901 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:35.891873    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac33e-7rd82"
Sep 04 22:16:35.895005 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:35.894980    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 04 22:16:35.895139 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:35.895111    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dftg6\""
Sep 04 22:16:35.904459 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:35.904402    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac33e-7rd82"]
Sep 04 22:16:35.905791 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:35.905757    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e56f1b01-9b88-40a3-880a-ad2eecbaaa3d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-dac33e-7rd82\" (UID: \"e56f1b01-9b88-40a3-880a-ad2eecbaaa3d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac33e-7rd82"
Sep 04 22:16:36.006950 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:36.006902    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e56f1b01-9b88-40a3-880a-ad2eecbaaa3d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-dac33e-7rd82\" (UID: \"e56f1b01-9b88-40a3-880a-ad2eecbaaa3d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac33e-7rd82"
Sep 04 22:16:36.007601 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:36.007575    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e56f1b01-9b88-40a3-880a-ad2eecbaaa3d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-dac33e-7rd82\" (UID: \"e56f1b01-9b88-40a3-880a-ad2eecbaaa3d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac33e-7rd82"
Sep 04 22:16:36.205889 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:36.205790    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac33e-7rd82"
Sep 04 22:16:36.478338 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:36.478301    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac33e-7rd82"]
Sep 04 22:16:36.479775 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:16:36.479734    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode56f1b01_9b88_40a3_880a_ad2eecbaaa3d.slice/crio-7f6fc52f8b1c685731c1a68f96fe0d45b1cc5c78bf625da42ff585dcf789ac4b WatchSource:0}: Error finding container 7f6fc52f8b1c685731c1a68f96fe0d45b1cc5c78bf625da42ff585dcf789ac4b: Status 404 returned error can't find the container with id 7f6fc52f8b1c685731c1a68f96fe0d45b1cc5c78bf625da42ff585dcf789ac4b
Sep 04 22:16:36.595828 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:36.595790    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac33e-7rd82" event={"ID":"e56f1b01-9b88-40a3-880a-ad2eecbaaa3d","Type":"ContainerStarted","Data":"7f6fc52f8b1c685731c1a68f96fe0d45b1cc5c78bf625da42ff585dcf789ac4b"}
Sep 04 22:16:37.605529 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:37.605429    2557 generic.go:356] "Generic (PLEG): container finished" podID="e56f1b01-9b88-40a3-880a-ad2eecbaaa3d" containerID="3e3759a95b6d596cf18737c4373f5adddd7591208188cb8db2377bbcbad3efcc" exitCode=0
Sep 04 22:16:37.605877 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:37.605519    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac33e-7rd82" event={"ID":"e56f1b01-9b88-40a3-880a-ad2eecbaaa3d","Type":"ContainerDied","Data":"3e3759a95b6d596cf18737c4373f5adddd7591208188cb8db2377bbcbad3efcc"}
Sep 04 22:16:38.748741 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:38.748712    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac33e-7rd82"
Sep 04 22:16:38.835889 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:38.835845    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e56f1b01-9b88-40a3-880a-ad2eecbaaa3d-maas-api-service-ca\") pod \"e56f1b01-9b88-40a3-880a-ad2eecbaaa3d\" (UID: \"e56f1b01-9b88-40a3-880a-ad2eecbaaa3d\") "
Sep 04 22:16:38.836231 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:38.836206    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e56f1b01-9b88-40a3-880a-ad2eecbaaa3d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e56f1b01-9b88-40a3-880a-ad2eecbaaa3d" (UID: "e56f1b01-9b88-40a3-880a-ad2eecbaaa3d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:16:38.937260 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:38.937170    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e56f1b01-9b88-40a3-880a-ad2eecbaaa3d-maas-api-service-ca\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:16:39.619590 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:39.619555    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac33e-7rd82"
Sep 04 22:16:39.619769 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:39.619599    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac33e-7rd82" event={"ID":"e56f1b01-9b88-40a3-880a-ad2eecbaaa3d","Type":"ContainerDied","Data":"7f6fc52f8b1c685731c1a68f96fe0d45b1cc5c78bf625da42ff585dcf789ac4b"}
Sep 04 22:16:39.619769 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:39.619649    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f6fc52f8b1c685731c1a68f96fe0d45b1cc5c78bf625da42ff585dcf789ac4b"
Sep 04 22:16:45.848625 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:45.848584    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:16:50.694166 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:50.694049    2557 generic.go:356] "Generic (PLEG): container finished" podID="c93d0e5c-af64-4138-9542-13b6c1042254" containerID="a985b151e803fbe1bce0897c228e41ec24a7cf4f4150170d3485e95aaf383a6e" exitCode=0
Sep 04 22:16:50.694166 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:50.694127    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr" event={"ID":"c93d0e5c-af64-4138-9542-13b6c1042254","Type":"ContainerDied","Data":"a985b151e803fbe1bce0897c228e41ec24a7cf4f4150170d3485e95aaf383a6e"}
Sep 04 22:16:50.694771 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:50.694677    2557 scope.go:117] "RemoveContainer" containerID="a985b151e803fbe1bce0897c228e41ec24a7cf4f4150170d3485e95aaf383a6e"
Sep 04 22:16:50.695984 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:50.695954    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:16:50.696059 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:50.696026    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:16:50.696100 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:50.696060    2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 04 22:16:51.611927 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:51.610249    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c"]
Sep 04 22:16:51.611927 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:51.610560    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c" podUID="782d444a-e8b9-4c2f-a76f-fce9e99bbabe" containerName="payload-processing" containerID="cri-o://49953a2bf123ef1f0882e85f215339922ffeb5e56f7744cf7a2ae3a3855d282f" gracePeriod=30
Sep 04 22:16:51.616005 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:51.615970    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27"]
Sep 04 22:16:51.616479 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:51.616446    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27" podUID="2ddbe8f6-3d38-4271-9cef-d230cc6bf531" containerName="payload-pre-processing" containerID="cri-o://428d97cfe1f3052711dae4345bfff085355514c67f6360d9ad53d8c221010c7e" gracePeriod=30
Sep 04 22:16:51.712019 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:51.711984    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr" event={"ID":"c93d0e5c-af64-4138-9542-13b6c1042254","Type":"ContainerStarted","Data":"e0ebb65adab96dbd2feae6531c8a4e5b025cddb3bc05f0aba8f7c0a56795d22e"}
Sep 04 22:16:51.816585 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:51.804469    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac33e-7rd82"]
Sep 04 22:16:51.818553 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:51.818470    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-dac33e-7rd82"]
Sep 04 22:16:51.935122 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:51.935091    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27"
Sep 04 22:16:51.948220 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:51.948185    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c"
Sep 04 22:16:51.975962 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:51.975923    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ddbe8f6-3d38-4271-9cef-d230cc6bf531-plugins-config-volume\") pod \"2ddbe8f6-3d38-4271-9cef-d230cc6bf531\" (UID: \"2ddbe8f6-3d38-4271-9cef-d230cc6bf531\") "
Sep 04 22:16:51.976148 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:51.976006    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrzwk\" (UniqueName: \"kubernetes.io/projected/2ddbe8f6-3d38-4271-9cef-d230cc6bf531-kube-api-access-hrzwk\") pod \"2ddbe8f6-3d38-4271-9cef-d230cc6bf531\" (UID: \"2ddbe8f6-3d38-4271-9cef-d230cc6bf531\") "
Sep 04 22:16:51.976548 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:51.976522    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ddbe8f6-3d38-4271-9cef-d230cc6bf531-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2ddbe8f6-3d38-4271-9cef-d230cc6bf531" (UID: "2ddbe8f6-3d38-4271-9cef-d230cc6bf531"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:16:51.978432 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:51.978387    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ddbe8f6-3d38-4271-9cef-d230cc6bf531-kube-api-access-hrzwk" (OuterVolumeSpecName: "kube-api-access-hrzwk") pod "2ddbe8f6-3d38-4271-9cef-d230cc6bf531" (UID: "2ddbe8f6-3d38-4271-9cef-d230cc6bf531"). InnerVolumeSpecName "kube-api-access-hrzwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:16:52.077383 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.077342    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wdj2n\" (UniqueName: \"kubernetes.io/projected/782d444a-e8b9-4c2f-a76f-fce9e99bbabe-kube-api-access-wdj2n\") pod \"782d444a-e8b9-4c2f-a76f-fce9e99bbabe\" (UID: \"782d444a-e8b9-4c2f-a76f-fce9e99bbabe\") "
Sep 04 22:16:52.077587 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.077440    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/782d444a-e8b9-4c2f-a76f-fce9e99bbabe-plugins-config-volume\") pod \"782d444a-e8b9-4c2f-a76f-fce9e99bbabe\" (UID: \"782d444a-e8b9-4c2f-a76f-fce9e99bbabe\") "
Sep 04 22:16:52.077748 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.077729    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrzwk\" (UniqueName: \"kubernetes.io/projected/2ddbe8f6-3d38-4271-9cef-d230cc6bf531-kube-api-access-hrzwk\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:16:52.077812 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.077752    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ddbe8f6-3d38-4271-9cef-d230cc6bf531-plugins-config-volume\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:16:52.077812 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.077763    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/782d444a-e8b9-4c2f-a76f-fce9e99bbabe-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "782d444a-e8b9-4c2f-a76f-fce9e99bbabe" (UID: "782d444a-e8b9-4c2f-a76f-fce9e99bbabe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:16:52.089672 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.089591    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/782d444a-e8b9-4c2f-a76f-fce9e99bbabe-kube-api-access-wdj2n" (OuterVolumeSpecName: "kube-api-access-wdj2n") pod "782d444a-e8b9-4c2f-a76f-fce9e99bbabe" (UID: "782d444a-e8b9-4c2f-a76f-fce9e99bbabe"). InnerVolumeSpecName "kube-api-access-wdj2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:16:52.179241 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.179123    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wdj2n\" (UniqueName: \"kubernetes.io/projected/782d444a-e8b9-4c2f-a76f-fce9e99bbabe-kube-api-access-wdj2n\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:16:52.179241 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.179173    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/782d444a-e8b9-4c2f-a76f-fce9e99bbabe-plugins-config-volume\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:16:52.352678 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.352642    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e56f1b01-9b88-40a3-880a-ad2eecbaaa3d" path="/var/lib/kubelet/pods/e56f1b01-9b88-40a3-880a-ad2eecbaaa3d/volumes"
Sep 04 22:16:52.597015 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.596910    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:52.597242 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.597214    2557 patch_prober.go:28] interesting pod/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 04 22:16:52.597391 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.597274    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr" podUID="c93d0e5c-af64-4138-9542-13b6c1042254" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 04 22:16:52.720555 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.720520    2557 generic.go:356] "Generic (PLEG): container finished" podID="782d444a-e8b9-4c2f-a76f-fce9e99bbabe" containerID="49953a2bf123ef1f0882e85f215339922ffeb5e56f7744cf7a2ae3a3855d282f" exitCode=0
Sep 04 22:16:52.720999 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.720561    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c"
Sep 04 22:16:52.720999 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.720581    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c" event={"ID":"782d444a-e8b9-4c2f-a76f-fce9e99bbabe","Type":"ContainerDied","Data":"49953a2bf123ef1f0882e85f215339922ffeb5e56f7744cf7a2ae3a3855d282f"}
Sep 04 22:16:52.720999 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.720626    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c" event={"ID":"782d444a-e8b9-4c2f-a76f-fce9e99bbabe","Type":"ContainerDied","Data":"c357e62be6c1bcb31ca4d120b157da68a1423284fbd9c5653f9854c48c70f5ba"}
Sep 04 22:16:52.720999 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.720649    2557 scope.go:117] "RemoveContainer" containerID="49953a2bf123ef1f0882e85f215339922ffeb5e56f7744cf7a2ae3a3855d282f"
Sep 04 22:16:52.722464 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.722437    2557 generic.go:356] "Generic (PLEG): container finished" podID="2ddbe8f6-3d38-4271-9cef-d230cc6bf531" containerID="428d97cfe1f3052711dae4345bfff085355514c67f6360d9ad53d8c221010c7e" exitCode=0
Sep 04 22:16:52.723028 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.723012    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27"
Sep 04 22:16:52.723293 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.723258    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27" event={"ID":"2ddbe8f6-3d38-4271-9cef-d230cc6bf531","Type":"ContainerDied","Data":"428d97cfe1f3052711dae4345bfff085355514c67f6360d9ad53d8c221010c7e"}
Sep 04 22:16:52.723369 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.723341    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27" event={"ID":"2ddbe8f6-3d38-4271-9cef-d230cc6bf531","Type":"ContainerDied","Data":"9efc8ffd1ebbc243882c88ac27bfa5d9ca3e65738dcd4db45e05fa10070d616e"}
Sep 04 22:16:52.733112 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.732825    2557 scope.go:117] "RemoveContainer" containerID="49953a2bf123ef1f0882e85f215339922ffeb5e56f7744cf7a2ae3a3855d282f"
Sep 04 22:16:52.733224 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:16:52.733196    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49953a2bf123ef1f0882e85f215339922ffeb5e56f7744cf7a2ae3a3855d282f\": container with ID starting with 49953a2bf123ef1f0882e85f215339922ffeb5e56f7744cf7a2ae3a3855d282f not found: ID does not exist" containerID="49953a2bf123ef1f0882e85f215339922ffeb5e56f7744cf7a2ae3a3855d282f"
Sep 04 22:16:52.733265 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.733235    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49953a2bf123ef1f0882e85f215339922ffeb5e56f7744cf7a2ae3a3855d282f"} err="failed to get container status \"49953a2bf123ef1f0882e85f215339922ffeb5e56f7744cf7a2ae3a3855d282f\": rpc error: code = NotFound desc = could not find container \"49953a2bf123ef1f0882e85f215339922ffeb5e56f7744cf7a2ae3a3855d282f\": container with ID starting with 49953a2bf123ef1f0882e85f215339922ffeb5e56f7744cf7a2ae3a3855d282f not found: ID does not exist"
Sep 04 22:16:52.733265 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.733253    2557 scope.go:117] "RemoveContainer" containerID="428d97cfe1f3052711dae4345bfff085355514c67f6360d9ad53d8c221010c7e"
Sep 04 22:16:52.743653 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.743610    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27"]
Sep 04 22:16:52.744322 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.744300    2557 scope.go:117] "RemoveContainer" containerID="428d97cfe1f3052711dae4345bfff085355514c67f6360d9ad53d8c221010c7e"
Sep 04 22:16:52.744643 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:16:52.744621    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"428d97cfe1f3052711dae4345bfff085355514c67f6360d9ad53d8c221010c7e\": container with ID starting with 428d97cfe1f3052711dae4345bfff085355514c67f6360d9ad53d8c221010c7e not found: ID does not exist" containerID="428d97cfe1f3052711dae4345bfff085355514c67f6360d9ad53d8c221010c7e"
Sep 04 22:16:52.744719 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.744651    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"428d97cfe1f3052711dae4345bfff085355514c67f6360d9ad53d8c221010c7e"} err="failed to get container status \"428d97cfe1f3052711dae4345bfff085355514c67f6360d9ad53d8c221010c7e\": rpc error: code = NotFound desc = could not find container \"428d97cfe1f3052711dae4345bfff085355514c67f6360d9ad53d8c221010c7e\": container with ID starting with 428d97cfe1f3052711dae4345bfff085355514c67f6360d9ad53d8c221010c7e not found: ID does not exist"
Sep 04 22:16:52.746150 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.746126    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-dac33e-8685769f94-nmp27"]
Sep 04 22:16:52.759447 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.759038    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c"]
Sep 04 22:16:52.765765 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:52.765728    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-dac33e-5b9f789887-bk65c"]
Sep 04 22:16:53.598055 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:53.598010    2557 patch_prober.go:28] interesting pod/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 04 22:16:53.598271 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:53.598093    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr" podUID="c93d0e5c-af64-4138-9542-13b6c1042254" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 04 22:16:54.353058 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:54.353020    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ddbe8f6-3d38-4271-9cef-d230cc6bf531" path="/var/lib/kubelet/pods/2ddbe8f6-3d38-4271-9cef-d230cc6bf531/volumes"
Sep 04 22:16:54.353823 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:54.353785    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="782d444a-e8b9-4c2f-a76f-fce9e99bbabe" path="/var/lib/kubelet/pods/782d444a-e8b9-4c2f-a76f-fce9e99bbabe/volumes"
Sep 04 22:16:54.597716 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:54.597666    2557 patch_prober.go:28] interesting pod/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 04 22:16:54.597930 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:54.597740    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr" podUID="c93d0e5c-af64-4138-9542-13b6c1042254" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 04 22:16:55.597458 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:55.597391    2557 patch_prober.go:28] interesting pod/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 04 22:16:55.597963 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:55.597489    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr" podUID="c93d0e5c-af64-4138-9542-13b6c1042254" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 04 22:16:56.597619 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:56.597568    2557 patch_prober.go:28] interesting pod/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Sep 04 22:16:56.598127 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:56.597644    2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr" podUID="c93d0e5c-af64-4138-9542-13b6c1042254" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Sep 04 22:16:57.601499 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:57.601467    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:57.601903 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:57.601820    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:57.602524 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:57.602504    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:16:58.055184 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:58.055145    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:16:59.188323 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:59.188274    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-fc3156-f4dh5"]
Sep 04 22:16:59.188862 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:59.188704    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e56f1b01-9b88-40a3-880a-ad2eecbaaa3d" containerName="revoke-keys"
Sep 04 22:16:59.188862 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:59.188716    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e56f1b01-9b88-40a3-880a-ad2eecbaaa3d" containerName="revoke-keys"
Sep 04 22:16:59.188862 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:59.188732    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="782d444a-e8b9-4c2f-a76f-fce9e99bbabe" containerName="payload-processing"
Sep 04 22:16:59.188862 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:59.188737    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="782d444a-e8b9-4c2f-a76f-fce9e99bbabe" containerName="payload-processing"
Sep 04 22:16:59.188862 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:59.188750    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ddbe8f6-3d38-4271-9cef-d230cc6bf531" containerName="payload-pre-processing"
Sep 04 22:16:59.188862 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:59.188757    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ddbe8f6-3d38-4271-9cef-d230cc6bf531" containerName="payload-pre-processing"
Sep 04 22:16:59.188862 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:59.188821    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ddbe8f6-3d38-4271-9cef-d230cc6bf531" containerName="payload-pre-processing"
Sep 04 22:16:59.188862 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:59.188830    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e56f1b01-9b88-40a3-880a-ad2eecbaaa3d" containerName="revoke-keys"
Sep 04 22:16:59.188862 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:59.188837    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="782d444a-e8b9-4c2f-a76f-fce9e99bbabe" containerName="payload-processing"
Sep 04 22:16:59.192940 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:59.192913    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-fc3156-f4dh5"
Sep 04 22:16:59.195504 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:59.195311    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 04 22:16:59.196075 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:59.196046    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dftg6\""
Sep 04 22:16:59.199676 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:59.199648    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-fc3156-f4dh5"]
Sep 04 22:16:59.359402 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:59.359359    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c3d85e2-5925-43bf-b8df-4a6b5e73e4f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-fc3156-f4dh5\" (UID: \"3c3d85e2-5925-43bf-b8df-4a6b5e73e4f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-fc3156-f4dh5"
Sep 04 22:16:59.460519 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:59.460388    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c3d85e2-5925-43bf-b8df-4a6b5e73e4f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-fc3156-f4dh5\" (UID: \"3c3d85e2-5925-43bf-b8df-4a6b5e73e4f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-fc3156-f4dh5"
Sep 04 22:16:59.461058 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:59.461034    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c3d85e2-5925-43bf-b8df-4a6b5e73e4f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-fc3156-f4dh5\" (UID: \"3c3d85e2-5925-43bf-b8df-4a6b5e73e4f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-fc3156-f4dh5"
Sep 04 22:16:59.507217 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:59.507178    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-fc3156-f4dh5"
Sep 04 22:16:59.785890 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:16:59.783192    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-fc3156-f4dh5"]
Sep 04 22:17:00.772508 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:00.772470    2557 generic.go:356] "Generic (PLEG): container finished" podID="3c3d85e2-5925-43bf-b8df-4a6b5e73e4f6" containerID="4e07aa2d99f4da5611d416a3435d9a918267755c653a3ccf72932154e822a01d" exitCode=0
Sep 04 22:17:00.772891 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:00.772560    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-fc3156-f4dh5" event={"ID":"3c3d85e2-5925-43bf-b8df-4a6b5e73e4f6","Type":"ContainerDied","Data":"4e07aa2d99f4da5611d416a3435d9a918267755c653a3ccf72932154e822a01d"}
Sep 04 22:17:00.772891 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:00.772614    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-fc3156-f4dh5" event={"ID":"3c3d85e2-5925-43bf-b8df-4a6b5e73e4f6","Type":"ContainerStarted","Data":"5c6ef032dc8110bac1b6b0df24795b09733bdf20039751df943e6bbf29c25471"}
Sep 04 22:17:01.925183 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:01.925149    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-fc3156-f4dh5"
Sep 04 22:17:02.084335 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:02.084234    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c3d85e2-5925-43bf-b8df-4a6b5e73e4f6-maas-api-service-ca\") pod \"3c3d85e2-5925-43bf-b8df-4a6b5e73e4f6\" (UID: \"3c3d85e2-5925-43bf-b8df-4a6b5e73e4f6\") "
Sep 04 22:17:02.084652 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:02.084626    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c3d85e2-5925-43bf-b8df-4a6b5e73e4f6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3c3d85e2-5925-43bf-b8df-4a6b5e73e4f6" (UID: "3c3d85e2-5925-43bf-b8df-4a6b5e73e4f6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:17:02.185279 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:02.185234    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c3d85e2-5925-43bf-b8df-4a6b5e73e4f6-maas-api-service-ca\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:17:02.785771 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:02.785737    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-fc3156-f4dh5" event={"ID":"3c3d85e2-5925-43bf-b8df-4a6b5e73e4f6","Type":"ContainerDied","Data":"5c6ef032dc8110bac1b6b0df24795b09733bdf20039751df943e6bbf29c25471"}
Sep 04 22:17:02.785927 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:02.785785    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5c6ef032dc8110bac1b6b0df24795b09733bdf20039751df943e6bbf29c25471"
Sep 04 22:17:02.785927 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:02.785749    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-fc3156-f4dh5"
Sep 04 22:17:07.240471 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.240437    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"]
Sep 04 22:17:07.240859 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.240845    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c3d85e2-5925-43bf-b8df-4a6b5e73e4f6" containerName="revoke-keys"
Sep 04 22:17:07.240859 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.240860    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c3d85e2-5925-43bf-b8df-4a6b5e73e4f6" containerName="revoke-keys"
Sep 04 22:17:07.240946 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.240935    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c3d85e2-5925-43bf-b8df-4a6b5e73e4f6" containerName="revoke-keys"
Sep 04 22:17:07.244155 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.244130    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:07.246559 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.246534    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-cc92e2\"/\"kube-root-ca.crt\""
Sep 04 22:17:07.247336 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.247318    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-cc92e2\"/\"default-dockercfg-lxzb7\""
Sep 04 22:17:07.247492 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.247344    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-cc92e2\"/\"openshift-service-ca.crt\""
Sep 04 22:17:07.247492 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.247382    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-cc92e2\"/\"ipp-route-cc92e2-kserve-self-signed-certs\""
Sep 04 22:17:07.253515 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.253485    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"]
Sep 04 22:17:07.433035 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.432988    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-model-cache\") pod \"ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") " pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:07.433035 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.433039    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-tmp-dir\") pod \"ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") " pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:07.433292 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.433066    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-dshm\") pod \"ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") " pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:07.433292 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.433127    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-home\") pod \"ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") " pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:07.433292 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.433188    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d28zt\" (UniqueName: \"kubernetes.io/projected/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-kube-api-access-d28zt\") pod \"ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") " pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:07.433479 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.433338    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-tls-certs\") pod \"ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") " pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:07.534575 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.534528    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-model-cache\") pod \"ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") " pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:07.534575 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.534584    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-tmp-dir\") pod \"ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") " pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:07.534843 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.534724    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-dshm\") pod \"ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") " pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:07.534843 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.534802    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-home\") pod \"ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") " pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:07.534957 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.534910    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d28zt\" (UniqueName: \"kubernetes.io/projected/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-kube-api-access-d28zt\") pod \"ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") " pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:07.535038 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.535004    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-tmp-dir\") pod \"ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") " pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:07.535104 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.535030    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-model-cache\") pod \"ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") " pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:07.535104 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.535044    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-tls-certs\") pod \"ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") " pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:07.535274 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.535142    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-home\") pod \"ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") " pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:07.536984 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.536961    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-dshm\") pod \"ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") " pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:07.537336 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.537320    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-tls-certs\") pod \"ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") " pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:07.542498 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.542468    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d28zt\" (UniqueName: \"kubernetes.io/projected/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-kube-api-access-d28zt\") pod \"ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") " pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:07.553925 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.553889    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:07.831472 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:07.831439    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"]
Sep 04 22:17:07.833226 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:17:07.833197    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb4c0465a_eeb5_435f_be67_ac5f9e5c54b0.slice/crio-d3372c444a738b13d15ea0293d8c2ab28eb5941415676f299537c8d5af5ded28 WatchSource:0}: Error finding container d3372c444a738b13d15ea0293d8c2ab28eb5941415676f299537c8d5af5ded28: Status 404 returned error can't find the container with id d3372c444a738b13d15ea0293d8c2ab28eb5941415676f299537c8d5af5ded28
Sep 04 22:17:08.823265 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:08.823218    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm" event={"ID":"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0","Type":"ContainerStarted","Data":"27f5e72017512fc11008da2d5758ddf55a9b799acb9288fe5625c7f7e9846e5e"}
Sep 04 22:17:08.823265 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:08.823271    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm" event={"ID":"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0","Type":"ContainerStarted","Data":"d3372c444a738b13d15ea0293d8c2ab28eb5941415676f299537c8d5af5ded28"}
Sep 04 22:17:08.843072 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:08.843006    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm" podStartSLOduration=1.842989896 podStartE2EDuration="1.842989896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:17:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:17:08.840009803 +0000 UTC m=+1403.156576544" watchObservedRunningTime="2026-09-04 22:17:08.842989896 +0000 UTC m=+1403.159556633"
Sep 04 22:17:09.767337 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:09.767079    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq"]
Sep 04 22:17:09.768325 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:09.767740    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq" podUID="8a2aae30-4b34-46b2-b0dd-595eb60f39fa" containerName="payload-processing" containerID="cri-o://35427fb44bb813bd7765048a55ae9b3ef557a6b344967672529752acae925e34" gracePeriod=30
Sep 04 22:17:09.772133 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:09.772081    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r"]
Sep 04 22:17:09.772378 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:09.772327    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r" podUID="0cd50664-2acb-4745-956a-21f990edaed7" containerName="payload-pre-processing" containerID="cri-o://2753f3c57643a9883f512703caef51b41fa069c58e98c60b8343f3eea178099d" gracePeriod=30
Sep 04 22:17:09.922926 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:09.922811    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-fc3156-f4dh5"]
Sep 04 22:17:09.925553 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:09.925516    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-fc3156-f4dh5"]
Sep 04 22:17:10.053182 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.053149    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq"
Sep 04 22:17:10.056154 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.056127    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r"
Sep 04 22:17:10.063611 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.063577    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvzwc\" (UniqueName: \"kubernetes.io/projected/8a2aae30-4b34-46b2-b0dd-595eb60f39fa-kube-api-access-pvzwc\") pod \"8a2aae30-4b34-46b2-b0dd-595eb60f39fa\" (UID: \"8a2aae30-4b34-46b2-b0dd-595eb60f39fa\") "
Sep 04 22:17:10.063737 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.063640    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a2aae30-4b34-46b2-b0dd-595eb60f39fa-plugins-config-volume\") pod \"8a2aae30-4b34-46b2-b0dd-595eb60f39fa\" (UID: \"8a2aae30-4b34-46b2-b0dd-595eb60f39fa\") "
Sep 04 22:17:10.063877 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.063857    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cd50664-2acb-4745-956a-21f990edaed7-plugins-config-volume\") pod \"0cd50664-2acb-4745-956a-21f990edaed7\" (UID: \"0cd50664-2acb-4745-956a-21f990edaed7\") "
Sep 04 22:17:10.063939 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.063896    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jfkp\" (UniqueName: \"kubernetes.io/projected/0cd50664-2acb-4745-956a-21f990edaed7-kube-api-access-7jfkp\") pod \"0cd50664-2acb-4745-956a-21f990edaed7\" (UID: \"0cd50664-2acb-4745-956a-21f990edaed7\") "
Sep 04 22:17:10.064117 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.064091    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a2aae30-4b34-46b2-b0dd-595eb60f39fa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8a2aae30-4b34-46b2-b0dd-595eb60f39fa" (UID: "8a2aae30-4b34-46b2-b0dd-595eb60f39fa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:17:10.064266 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.064236    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cd50664-2acb-4745-956a-21f990edaed7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0cd50664-2acb-4745-956a-21f990edaed7" (UID: "0cd50664-2acb-4745-956a-21f990edaed7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:17:10.064266 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.064262    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a2aae30-4b34-46b2-b0dd-595eb60f39fa-plugins-config-volume\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:17:10.065883 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.065861    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a2aae30-4b34-46b2-b0dd-595eb60f39fa-kube-api-access-pvzwc" (OuterVolumeSpecName: "kube-api-access-pvzwc") pod "8a2aae30-4b34-46b2-b0dd-595eb60f39fa" (UID: "8a2aae30-4b34-46b2-b0dd-595eb60f39fa"). InnerVolumeSpecName "kube-api-access-pvzwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:17:10.065957 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.065899    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cd50664-2acb-4745-956a-21f990edaed7-kube-api-access-7jfkp" (OuterVolumeSpecName: "kube-api-access-7jfkp") pod "0cd50664-2acb-4745-956a-21f990edaed7" (UID: "0cd50664-2acb-4745-956a-21f990edaed7"). InnerVolumeSpecName "kube-api-access-7jfkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:17:10.165178 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.165128    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0cd50664-2acb-4745-956a-21f990edaed7-plugins-config-volume\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:17:10.165178 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.165166    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jfkp\" (UniqueName: \"kubernetes.io/projected/0cd50664-2acb-4745-956a-21f990edaed7-kube-api-access-7jfkp\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:17:10.165178 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.165176    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvzwc\" (UniqueName: \"kubernetes.io/projected/8a2aae30-4b34-46b2-b0dd-595eb60f39fa-kube-api-access-pvzwc\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:17:10.353856 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.353769    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c3d85e2-5925-43bf-b8df-4a6b5e73e4f6" path="/var/lib/kubelet/pods/3c3d85e2-5925-43bf-b8df-4a6b5e73e4f6/volumes"
Sep 04 22:17:10.836226 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.836184    2557 generic.go:356] "Generic (PLEG): container finished" podID="8a2aae30-4b34-46b2-b0dd-595eb60f39fa" containerID="35427fb44bb813bd7765048a55ae9b3ef557a6b344967672529752acae925e34" exitCode=0
Sep 04 22:17:10.836226 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.836230    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq"
Sep 04 22:17:10.836490 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.836274    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq" event={"ID":"8a2aae30-4b34-46b2-b0dd-595eb60f39fa","Type":"ContainerDied","Data":"35427fb44bb813bd7765048a55ae9b3ef557a6b344967672529752acae925e34"}
Sep 04 22:17:10.836490 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.836339    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq" event={"ID":"8a2aae30-4b34-46b2-b0dd-595eb60f39fa","Type":"ContainerDied","Data":"47273b80b6a66ae23499e53bcd817deb8b2d06cdfe085efca292897d267c97e2"}
Sep 04 22:17:10.836490 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.836356    2557 scope.go:117] "RemoveContainer" containerID="35427fb44bb813bd7765048a55ae9b3ef557a6b344967672529752acae925e34"
Sep 04 22:17:10.837764 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.837739    2557 generic.go:356] "Generic (PLEG): container finished" podID="0cd50664-2acb-4745-956a-21f990edaed7" containerID="2753f3c57643a9883f512703caef51b41fa069c58e98c60b8343f3eea178099d" exitCode=0
Sep 04 22:17:10.837874 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.837796    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r"
Sep 04 22:17:10.837874 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.837813    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r" event={"ID":"0cd50664-2acb-4745-956a-21f990edaed7","Type":"ContainerDied","Data":"2753f3c57643a9883f512703caef51b41fa069c58e98c60b8343f3eea178099d"}
Sep 04 22:17:10.837874 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.837852    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r" event={"ID":"0cd50664-2acb-4745-956a-21f990edaed7","Type":"ContainerDied","Data":"ec875c801b17cee570b0bf91814a8470dbd1d6b3b0ca391c9531d49db428dc4f"}
Sep 04 22:17:10.846463 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.846437    2557 scope.go:117] "RemoveContainer" containerID="35427fb44bb813bd7765048a55ae9b3ef557a6b344967672529752acae925e34"
Sep 04 22:17:10.846803 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:17:10.846780    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35427fb44bb813bd7765048a55ae9b3ef557a6b344967672529752acae925e34\": container with ID starting with 35427fb44bb813bd7765048a55ae9b3ef557a6b344967672529752acae925e34 not found: ID does not exist" containerID="35427fb44bb813bd7765048a55ae9b3ef557a6b344967672529752acae925e34"
Sep 04 22:17:10.846868 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.846813    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35427fb44bb813bd7765048a55ae9b3ef557a6b344967672529752acae925e34"} err="failed to get container status \"35427fb44bb813bd7765048a55ae9b3ef557a6b344967672529752acae925e34\": rpc error: code = NotFound desc = could not find container \"35427fb44bb813bd7765048a55ae9b3ef557a6b344967672529752acae925e34\": container with ID starting with 35427fb44bb813bd7765048a55ae9b3ef557a6b344967672529752acae925e34 not found: ID does not exist"
Sep 04 22:17:10.846868 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.846829    2557 scope.go:117] "RemoveContainer" containerID="2753f3c57643a9883f512703caef51b41fa069c58e98c60b8343f3eea178099d"
Sep 04 22:17:10.857265 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.857195    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq"]
Sep 04 22:17:10.857329 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.857290    2557 scope.go:117] "RemoveContainer" containerID="2753f3c57643a9883f512703caef51b41fa069c58e98c60b8343f3eea178099d"
Sep 04 22:17:10.857693 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:17:10.857669    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2753f3c57643a9883f512703caef51b41fa069c58e98c60b8343f3eea178099d\": container with ID starting with 2753f3c57643a9883f512703caef51b41fa069c58e98c60b8343f3eea178099d not found: ID does not exist" containerID="2753f3c57643a9883f512703caef51b41fa069c58e98c60b8343f3eea178099d"
Sep 04 22:17:10.857759 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.857708    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2753f3c57643a9883f512703caef51b41fa069c58e98c60b8343f3eea178099d"} err="failed to get container status \"2753f3c57643a9883f512703caef51b41fa069c58e98c60b8343f3eea178099d\": rpc error: code = NotFound desc = could not find container \"2753f3c57643a9883f512703caef51b41fa069c58e98c60b8343f3eea178099d\": container with ID starting with 2753f3c57643a9883f512703caef51b41fa069c58e98c60b8343f3eea178099d not found: ID does not exist"
Sep 04 22:17:10.867472 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.867196    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-fc3156-5bc876dc85-lrvqq"]
Sep 04 22:17:10.877706 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.877667    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r"]
Sep 04 22:17:10.883973 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:10.883935    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-fc3156-66c6b5dfcd-sb29r"]
Sep 04 22:17:12.154843 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:12.154802    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:17:12.353403 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:12.353353    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cd50664-2acb-4745-956a-21f990edaed7" path="/var/lib/kubelet/pods/0cd50664-2acb-4745-956a-21f990edaed7/volumes"
Sep 04 22:17:12.353874 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:12.353851    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a2aae30-4b34-46b2-b0dd-595eb60f39fa" path="/var/lib/kubelet/pods/8a2aae30-4b34-46b2-b0dd-595eb60f39fa/volumes"
Sep 04 22:17:16.748512 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:16.748467    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3abe72-nnsvm"]
Sep 04 22:17:16.748988 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:16.748852    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a2aae30-4b34-46b2-b0dd-595eb60f39fa" containerName="payload-processing"
Sep 04 22:17:16.748988 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:16.748864    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a2aae30-4b34-46b2-b0dd-595eb60f39fa" containerName="payload-processing"
Sep 04 22:17:16.748988 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:16.748874    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cd50664-2acb-4745-956a-21f990edaed7" containerName="payload-pre-processing"
Sep 04 22:17:16.748988 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:16.748880    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cd50664-2acb-4745-956a-21f990edaed7" containerName="payload-pre-processing"
Sep 04 22:17:16.748988 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:16.748951    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cd50664-2acb-4745-956a-21f990edaed7" containerName="payload-pre-processing"
Sep 04 22:17:16.748988 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:16.748960    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a2aae30-4b34-46b2-b0dd-595eb60f39fa" containerName="payload-processing"
Sep 04 22:17:16.751919 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:16.751899    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3abe72-nnsvm"
Sep 04 22:17:16.754212 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:16.754182    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dftg6\""
Sep 04 22:17:16.754353 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:16.754182    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 04 22:17:16.758701 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:16.758666    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3abe72-nnsvm"]
Sep 04 22:17:16.831269 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:16.831227    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeb6efec-02c5-4af9-b7f2-a5d955e32a9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-3abe72-nnsvm\" (UID: \"eeb6efec-02c5-4af9-b7f2-a5d955e32a9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3abe72-nnsvm"
Sep 04 22:17:16.932497 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:16.932450    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeb6efec-02c5-4af9-b7f2-a5d955e32a9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-3abe72-nnsvm\" (UID: \"eeb6efec-02c5-4af9-b7f2-a5d955e32a9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3abe72-nnsvm"
Sep 04 22:17:16.933111 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:16.933086    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeb6efec-02c5-4af9-b7f2-a5d955e32a9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-3abe72-nnsvm\" (UID: \"eeb6efec-02c5-4af9-b7f2-a5d955e32a9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3abe72-nnsvm"
Sep 04 22:17:17.064306 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:17.064196    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3abe72-nnsvm"
Sep 04 22:17:17.339923 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:17.339890    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3abe72-nnsvm"]
Sep 04 22:17:17.340164 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:17:17.340140    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeb6efec_02c5_4af9_b7f2_a5d955e32a9c.slice/crio-3b034672deb706dd5b22fd0a46b56cf05030dddde4f2e22b125c18f6920da974 WatchSource:0}: Error finding container 3b034672deb706dd5b22fd0a46b56cf05030dddde4f2e22b125c18f6920da974: Status 404 returned error can't find the container with id 3b034672deb706dd5b22fd0a46b56cf05030dddde4f2e22b125c18f6920da974
Sep 04 22:17:17.554669 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:17.554625    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:17.554878 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:17.554691    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:17.567737 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:17.567706    2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:17.883544 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:17.883440    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3abe72-nnsvm" event={"ID":"eeb6efec-02c5-4af9-b7f2-a5d955e32a9c","Type":"ContainerStarted","Data":"fc9aee5f10cb8ce91d34aebde2128113c28acff7a58b7ab54c92c6b250b1eca0"}
Sep 04 22:17:17.883544 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:17.883507    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3abe72-nnsvm" event={"ID":"eeb6efec-02c5-4af9-b7f2-a5d955e32a9c","Type":"ContainerStarted","Data":"3b034672deb706dd5b22fd0a46b56cf05030dddde4f2e22b125c18f6920da974"}
Sep 04 22:17:17.895249 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:17.895215    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:17:17.900172 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:17.900119    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3abe72-nnsvm" podStartSLOduration=1.900102712 podStartE2EDuration="1.900102712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:17:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:17:17.896829859 +0000 UTC m=+1412.213396597" watchObservedRunningTime="2026-09-04 22:17:17.900102712 +0000 UTC m=+1412.216669449"
Sep 04 22:17:18.891119 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:18.891081    2557 generic.go:356] "Generic (PLEG): container finished" podID="eeb6efec-02c5-4af9-b7f2-a5d955e32a9c" containerID="fc9aee5f10cb8ce91d34aebde2128113c28acff7a58b7ab54c92c6b250b1eca0" exitCode=0
Sep 04 22:17:18.891536 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:18.891144    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3abe72-nnsvm" event={"ID":"eeb6efec-02c5-4af9-b7f2-a5d955e32a9c","Type":"ContainerDied","Data":"fc9aee5f10cb8ce91d34aebde2128113c28acff7a58b7ab54c92c6b250b1eca0"}
Sep 04 22:17:20.041444 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:20.035232    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3abe72-nnsvm"
Sep 04 22:17:20.065548 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:20.065514    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeb6efec-02c5-4af9-b7f2-a5d955e32a9c-maas-api-service-ca\") pod \"eeb6efec-02c5-4af9-b7f2-a5d955e32a9c\" (UID: \"eeb6efec-02c5-4af9-b7f2-a5d955e32a9c\") "
Sep 04 22:17:20.065880 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:20.065851    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eeb6efec-02c5-4af9-b7f2-a5d955e32a9c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "eeb6efec-02c5-4af9-b7f2-a5d955e32a9c" (UID: "eeb6efec-02c5-4af9-b7f2-a5d955e32a9c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:17:20.167242 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:20.167141    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/eeb6efec-02c5-4af9-b7f2-a5d955e32a9c-maas-api-service-ca\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:17:20.904386 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:20.904349    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3abe72-nnsvm"
Sep 04 22:17:20.904598 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:20.904429    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3abe72-nnsvm" event={"ID":"eeb6efec-02c5-4af9-b7f2-a5d955e32a9c","Type":"ContainerDied","Data":"3b034672deb706dd5b22fd0a46b56cf05030dddde4f2e22b125c18f6920da974"}
Sep 04 22:17:20.904598 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:20.904479    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b034672deb706dd5b22fd0a46b56cf05030dddde4f2e22b125c18f6920da974"
Sep 04 22:17:26.041061 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:26.041022    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:17:27.211562 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.211524    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl"]
Sep 04 22:17:27.212025 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.211734    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl" podUID="4466206b-3c94-4dfe-b1d5-8c689298684b" containerName="payload-processing" containerID="cri-o://113728004bf9f5dbd9d7377d1ca848af9b18e35050c5e0511dafe48b00376ec0" gracePeriod=30
Sep 04 22:17:27.223684 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.223650    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78"]
Sep 04 22:17:27.224141 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.224108    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78" podUID="8ab87507-bdff-439a-8ece-8902dffe7dfd" containerName="payload-pre-processing" containerID="cri-o://35854cc0c119afd8f5e6dedf09db29278ce33f86fd971b32083cbbb5e025737b" gracePeriod=30
Sep 04 22:17:27.368982 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.368941    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3abe72-nnsvm"]
Sep 04 22:17:27.373076 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.373039    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-3abe72-nnsvm"]
Sep 04 22:17:27.502726 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.502695    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl"
Sep 04 22:17:27.505868 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.505841    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78"
Sep 04 22:17:27.638335 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.638292    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k67tk\" (UniqueName: \"kubernetes.io/projected/8ab87507-bdff-439a-8ece-8902dffe7dfd-kube-api-access-k67tk\") pod \"8ab87507-bdff-439a-8ece-8902dffe7dfd\" (UID: \"8ab87507-bdff-439a-8ece-8902dffe7dfd\") "
Sep 04 22:17:27.638521 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.638388    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nm6m8\" (UniqueName: \"kubernetes.io/projected/4466206b-3c94-4dfe-b1d5-8c689298684b-kube-api-access-nm6m8\") pod \"4466206b-3c94-4dfe-b1d5-8c689298684b\" (UID: \"4466206b-3c94-4dfe-b1d5-8c689298684b\") "
Sep 04 22:17:27.638586 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.638546    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ab87507-bdff-439a-8ece-8902dffe7dfd-plugins-config-volume\") pod \"8ab87507-bdff-439a-8ece-8902dffe7dfd\" (UID: \"8ab87507-bdff-439a-8ece-8902dffe7dfd\") "
Sep 04 22:17:27.638663 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.638645    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4466206b-3c94-4dfe-b1d5-8c689298684b-plugins-config-volume\") pod \"4466206b-3c94-4dfe-b1d5-8c689298684b\" (UID: \"4466206b-3c94-4dfe-b1d5-8c689298684b\") "
Sep 04 22:17:27.638929 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.638895    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ab87507-bdff-439a-8ece-8902dffe7dfd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8ab87507-bdff-439a-8ece-8902dffe7dfd" (UID: "8ab87507-bdff-439a-8ece-8902dffe7dfd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:17:27.638929 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.638904    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4466206b-3c94-4dfe-b1d5-8c689298684b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4466206b-3c94-4dfe-b1d5-8c689298684b" (UID: "4466206b-3c94-4dfe-b1d5-8c689298684b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:17:27.641104 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.641080    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ab87507-bdff-439a-8ece-8902dffe7dfd-kube-api-access-k67tk" (OuterVolumeSpecName: "kube-api-access-k67tk") pod "8ab87507-bdff-439a-8ece-8902dffe7dfd" (UID: "8ab87507-bdff-439a-8ece-8902dffe7dfd"). InnerVolumeSpecName "kube-api-access-k67tk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:17:27.641215 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.641115    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4466206b-3c94-4dfe-b1d5-8c689298684b-kube-api-access-nm6m8" (OuterVolumeSpecName: "kube-api-access-nm6m8") pod "4466206b-3c94-4dfe-b1d5-8c689298684b" (UID: "4466206b-3c94-4dfe-b1d5-8c689298684b"). InnerVolumeSpecName "kube-api-access-nm6m8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:17:27.740253 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.740156    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k67tk\" (UniqueName: \"kubernetes.io/projected/8ab87507-bdff-439a-8ece-8902dffe7dfd-kube-api-access-k67tk\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:17:27.740253 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.740194    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nm6m8\" (UniqueName: \"kubernetes.io/projected/4466206b-3c94-4dfe-b1d5-8c689298684b-kube-api-access-nm6m8\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:17:27.740253 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.740204    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ab87507-bdff-439a-8ece-8902dffe7dfd-plugins-config-volume\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:17:27.740253 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.740212    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4466206b-3c94-4dfe-b1d5-8c689298684b-plugins-config-volume\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:17:27.946106 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.946069    2557 generic.go:356] "Generic (PLEG): container finished" podID="4466206b-3c94-4dfe-b1d5-8c689298684b" containerID="113728004bf9f5dbd9d7377d1ca848af9b18e35050c5e0511dafe48b00376ec0" exitCode=0
Sep 04 22:17:27.946297 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.946113    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl"
Sep 04 22:17:27.946297 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.946155    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl" event={"ID":"4466206b-3c94-4dfe-b1d5-8c689298684b","Type":"ContainerDied","Data":"113728004bf9f5dbd9d7377d1ca848af9b18e35050c5e0511dafe48b00376ec0"}
Sep 04 22:17:27.946297 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.946217    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl" event={"ID":"4466206b-3c94-4dfe-b1d5-8c689298684b","Type":"ContainerDied","Data":"89fb8d45ae846a6edd9b70846881e8635049c033b064d5c845ced1d57cdb0ed2"}
Sep 04 22:17:27.946297 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.946252    2557 scope.go:117] "RemoveContainer" containerID="113728004bf9f5dbd9d7377d1ca848af9b18e35050c5e0511dafe48b00376ec0"
Sep 04 22:17:27.947620 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.947593    2557 generic.go:356] "Generic (PLEG): container finished" podID="8ab87507-bdff-439a-8ece-8902dffe7dfd" containerID="35854cc0c119afd8f5e6dedf09db29278ce33f86fd971b32083cbbb5e025737b" exitCode=0
Sep 04 22:17:27.947723 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.947630    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78"
Sep 04 22:17:27.947723 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.947652    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78" event={"ID":"8ab87507-bdff-439a-8ece-8902dffe7dfd","Type":"ContainerDied","Data":"35854cc0c119afd8f5e6dedf09db29278ce33f86fd971b32083cbbb5e025737b"}
Sep 04 22:17:27.947723 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.947707    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78" event={"ID":"8ab87507-bdff-439a-8ece-8902dffe7dfd","Type":"ContainerDied","Data":"a35f64706e585465ac63833f5a65bf415951dcfeb5fd0077c523e360405c55ce"}
Sep 04 22:17:27.956872 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.956852    2557 scope.go:117] "RemoveContainer" containerID="113728004bf9f5dbd9d7377d1ca848af9b18e35050c5e0511dafe48b00376ec0"
Sep 04 22:17:27.957215 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:17:27.957197    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"113728004bf9f5dbd9d7377d1ca848af9b18e35050c5e0511dafe48b00376ec0\": container with ID starting with 113728004bf9f5dbd9d7377d1ca848af9b18e35050c5e0511dafe48b00376ec0 not found: ID does not exist" containerID="113728004bf9f5dbd9d7377d1ca848af9b18e35050c5e0511dafe48b00376ec0"
Sep 04 22:17:27.957296 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.957223    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"113728004bf9f5dbd9d7377d1ca848af9b18e35050c5e0511dafe48b00376ec0"} err="failed to get container status \"113728004bf9f5dbd9d7377d1ca848af9b18e35050c5e0511dafe48b00376ec0\": rpc error: code = NotFound desc = could not find container \"113728004bf9f5dbd9d7377d1ca848af9b18e35050c5e0511dafe48b00376ec0\": container with ID starting with 113728004bf9f5dbd9d7377d1ca848af9b18e35050c5e0511dafe48b00376ec0 not found: ID does not exist"
Sep 04 22:17:27.957296 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.957240    2557 scope.go:117] "RemoveContainer" containerID="35854cc0c119afd8f5e6dedf09db29278ce33f86fd971b32083cbbb5e025737b"
Sep 04 22:17:27.966201 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.966180    2557 scope.go:117] "RemoveContainer" containerID="35854cc0c119afd8f5e6dedf09db29278ce33f86fd971b32083cbbb5e025737b"
Sep 04 22:17:27.966652 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:17:27.966632    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35854cc0c119afd8f5e6dedf09db29278ce33f86fd971b32083cbbb5e025737b\": container with ID starting with 35854cc0c119afd8f5e6dedf09db29278ce33f86fd971b32083cbbb5e025737b not found: ID does not exist" containerID="35854cc0c119afd8f5e6dedf09db29278ce33f86fd971b32083cbbb5e025737b"
Sep 04 22:17:27.966698 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.966662    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35854cc0c119afd8f5e6dedf09db29278ce33f86fd971b32083cbbb5e025737b"} err="failed to get container status \"35854cc0c119afd8f5e6dedf09db29278ce33f86fd971b32083cbbb5e025737b\": rpc error: code = NotFound desc = could not find container \"35854cc0c119afd8f5e6dedf09db29278ce33f86fd971b32083cbbb5e025737b\": container with ID starting with 35854cc0c119afd8f5e6dedf09db29278ce33f86fd971b32083cbbb5e025737b not found: ID does not exist"
Sep 04 22:17:27.972031 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.972006    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl"]
Sep 04 22:17:27.975893 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.975858    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-3abe72-759f75d775-thmcl"]
Sep 04 22:17:27.987504 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.987464    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78"]
Sep 04 22:17:27.991076 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:27.991014    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-3abe72-7d66d7b74-cfh78"]
Sep 04 22:17:28.352482 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:28.352446    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4466206b-3c94-4dfe-b1d5-8c689298684b" path="/var/lib/kubelet/pods/4466206b-3c94-4dfe-b1d5-8c689298684b/volumes"
Sep 04 22:17:28.352866 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:28.352840    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ab87507-bdff-439a-8ece-8902dffe7dfd" path="/var/lib/kubelet/pods/8ab87507-bdff-439a-8ece-8902dffe7dfd/volumes"
Sep 04 22:17:28.353137 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:28.353123    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eeb6efec-02c5-4af9-b7f2-a5d955e32a9c" path="/var/lib/kubelet/pods/eeb6efec-02c5-4af9-b7f2-a5d955e32a9c/volumes"
Sep 04 22:17:36.853164 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:36.853120    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:17:37.855338 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:37.855301    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq"]
Sep 04 22:17:37.855835 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:37.855816    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eeb6efec-02c5-4af9-b7f2-a5d955e32a9c" containerName="revoke-keys"
Sep 04 22:17:37.855880 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:37.855837    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="eeb6efec-02c5-4af9-b7f2-a5d955e32a9c" containerName="revoke-keys"
Sep 04 22:17:37.855880 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:37.855847    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4466206b-3c94-4dfe-b1d5-8c689298684b" containerName="payload-processing"
Sep 04 22:17:37.855880 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:37.855853    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4466206b-3c94-4dfe-b1d5-8c689298684b" containerName="payload-processing"
Sep 04 22:17:37.855880 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:37.855876    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ab87507-bdff-439a-8ece-8902dffe7dfd" containerName="payload-pre-processing"
Sep 04 22:17:37.856009 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:37.855883    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ab87507-bdff-439a-8ece-8902dffe7dfd" containerName="payload-pre-processing"
Sep 04 22:17:37.856009 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:37.855955    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ab87507-bdff-439a-8ece-8902dffe7dfd" containerName="payload-pre-processing"
Sep 04 22:17:37.856009 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:37.855965    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="eeb6efec-02c5-4af9-b7f2-a5d955e32a9c" containerName="revoke-keys"
Sep 04 22:17:37.856009 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:37.855973    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4466206b-3c94-4dfe-b1d5-8c689298684b" containerName="payload-processing"
Sep 04 22:17:37.860309 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:37.860289    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq"
Sep 04 22:17:37.862724 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:37.862700    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-b7805e\""
Sep 04 22:17:37.862839 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:37.862761    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-b7805e-dockercfg-bzgtq\""
Sep 04 22:17:37.870103 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:37.870077    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq"]
Sep 04 22:17:37.888896 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:37.888860    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw"]
Sep 04 22:17:37.892316 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:37.892297    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw"
Sep 04 22:17:37.904461 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:37.904431    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw"]
Sep 04 22:17:37.940215 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:37.940168    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9zcv\" (UniqueName: \"kubernetes.io/projected/b2b05836-c528-453c-95ea-2c4622c6020a-kube-api-access-b9zcv\") pod \"payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq\" (UID: \"b2b05836-c528-453c-95ea-2c4622c6020a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq"
Sep 04 22:17:37.940457 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:37.940245    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2b05836-c528-453c-95ea-2c4622c6020a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq\" (UID: \"b2b05836-c528-453c-95ea-2c4622c6020a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq"
Sep 04 22:17:38.041093 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:38.041053    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9zcv\" (UniqueName: \"kubernetes.io/projected/b2b05836-c528-453c-95ea-2c4622c6020a-kube-api-access-b9zcv\") pod \"payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq\" (UID: \"b2b05836-c528-453c-95ea-2c4622c6020a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq"
Sep 04 22:17:38.041093 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:38.041107    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw2mm\" (UniqueName: \"kubernetes.io/projected/d202a3f6-1e73-4b1b-8171-5344c3df1dea-kube-api-access-mw2mm\") pod \"payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw\" (UID: \"d202a3f6-1e73-4b1b-8171-5344c3df1dea\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw"
Sep 04 22:17:38.041338 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:38.041153    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d202a3f6-1e73-4b1b-8171-5344c3df1dea-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw\" (UID: \"d202a3f6-1e73-4b1b-8171-5344c3df1dea\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw"
Sep 04 22:17:38.041338 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:38.041186    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2b05836-c528-453c-95ea-2c4622c6020a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq\" (UID: \"b2b05836-c528-453c-95ea-2c4622c6020a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq"
Sep 04 22:17:38.041934 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:38.041911    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2b05836-c528-453c-95ea-2c4622c6020a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq\" (UID: \"b2b05836-c528-453c-95ea-2c4622c6020a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq"
Sep 04 22:17:38.049918 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:38.049891    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9zcv\" (UniqueName: \"kubernetes.io/projected/b2b05836-c528-453c-95ea-2c4622c6020a-kube-api-access-b9zcv\") pod \"payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq\" (UID: \"b2b05836-c528-453c-95ea-2c4622c6020a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq"
Sep 04 22:17:38.142875 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:38.142778    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw2mm\" (UniqueName: \"kubernetes.io/projected/d202a3f6-1e73-4b1b-8171-5344c3df1dea-kube-api-access-mw2mm\") pod \"payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw\" (UID: \"d202a3f6-1e73-4b1b-8171-5344c3df1dea\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw"
Sep 04 22:17:38.142875 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:38.142836    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d202a3f6-1e73-4b1b-8171-5344c3df1dea-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw\" (UID: \"d202a3f6-1e73-4b1b-8171-5344c3df1dea\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw"
Sep 04 22:17:38.143595 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:38.143574    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d202a3f6-1e73-4b1b-8171-5344c3df1dea-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw\" (UID: \"d202a3f6-1e73-4b1b-8171-5344c3df1dea\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw"
Sep 04 22:17:38.151777 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:38.151743    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw2mm\" (UniqueName: \"kubernetes.io/projected/d202a3f6-1e73-4b1b-8171-5344c3df1dea-kube-api-access-mw2mm\") pod \"payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw\" (UID: \"d202a3f6-1e73-4b1b-8171-5344c3df1dea\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw"
Sep 04 22:17:38.172483 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:38.172391    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq"
Sep 04 22:17:38.203751 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:38.203713    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw"
Sep 04 22:17:38.447835 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:38.447799    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq"]
Sep 04 22:17:38.448970 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:17:38.448946    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2b05836_c528_453c_95ea_2c4622c6020a.slice/crio-df9d001d8a886c2951fe992a0e973e9a853d0a91a82fa2557a04b6187e68f564 WatchSource:0}: Error finding container df9d001d8a886c2951fe992a0e973e9a853d0a91a82fa2557a04b6187e68f564: Status 404 returned error can't find the container with id df9d001d8a886c2951fe992a0e973e9a853d0a91a82fa2557a04b6187e68f564
Sep 04 22:17:38.490007 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:38.489979    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw"]
Sep 04 22:17:38.492064 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:17:38.492041    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd202a3f6_1e73_4b1b_8171_5344c3df1dea.slice/crio-9a0aaa7f98af74631767b21573d6d89bc97bcf84a545bf66cf16e8a516e4cb27 WatchSource:0}: Error finding container 9a0aaa7f98af74631767b21573d6d89bc97bcf84a545bf66cf16e8a516e4cb27: Status 404 returned error can't find the container with id 9a0aaa7f98af74631767b21573d6d89bc97bcf84a545bf66cf16e8a516e4cb27
Sep 04 22:17:39.018831 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:39.018788    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw" event={"ID":"d202a3f6-1e73-4b1b-8171-5344c3df1dea","Type":"ContainerStarted","Data":"ba483bf87a0737bcee9f0f4aace3f0f3041c310b4cd2ef840aeaa0ce46849b9a"}
Sep 04 22:17:39.019304 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:39.018843    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw" event={"ID":"d202a3f6-1e73-4b1b-8171-5344c3df1dea","Type":"ContainerStarted","Data":"9a0aaa7f98af74631767b21573d6d89bc97bcf84a545bf66cf16e8a516e4cb27"}
Sep 04 22:17:39.019304 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:39.018871    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw"
Sep 04 22:17:39.021108 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:39.021071    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq" event={"ID":"b2b05836-c528-453c-95ea-2c4622c6020a","Type":"ContainerStarted","Data":"df2f7cb90b8e6fe0e432a3da8cdedd014f7248a81bf014321e44a2245eb348cd"}
Sep 04 22:17:39.021256 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:39.021133    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq" event={"ID":"b2b05836-c528-453c-95ea-2c4622c6020a","Type":"ContainerStarted","Data":"df9d001d8a886c2951fe992a0e973e9a853d0a91a82fa2557a04b6187e68f564"}
Sep 04 22:17:39.021256 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:39.021173    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq"
Sep 04 22:17:39.042183 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:39.042129    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw" podStartSLOduration=2.042112593 podStartE2EDuration="2.042112593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:17:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:17:39.040311731 +0000 UTC m=+1433.356878470" watchObservedRunningTime="2026-09-04 22:17:39.042112593 +0000 UTC m=+1433.358679331"
Sep 04 22:17:39.059203 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:39.059147    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq" podStartSLOduration=2.059129481 podStartE2EDuration="2.059129481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-04 22:17:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:17:39.055987735 +0000 UTC m=+1433.372554473" watchObservedRunningTime="2026-09-04 22:17:39.059129481 +0000 UTC m=+1433.375696213"
Sep 04 22:17:48.950088 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:48.950047    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:17:50.034875 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:50.034840    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq"
Sep 04 22:17:50.035299 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:17:50.034927    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw"
Sep 04 22:18:01.666365 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:01.666321    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b7805e-tbzv2"]
Sep 04 22:18:01.671929 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:01.671905    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b7805e-tbzv2"
Sep 04 22:18:01.674538 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:01.674506    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 04 22:18:01.674716 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:01.674653    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dftg6\""
Sep 04 22:18:01.678935 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:01.678908    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b7805e-tbzv2"]
Sep 04 22:18:01.780260 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:01.780220    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4e5e253-04e4-4a7b-a464-0b16cd47f71d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-b7805e-tbzv2\" (UID: \"d4e5e253-04e4-4a7b-a464-0b16cd47f71d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b7805e-tbzv2"
Sep 04 22:18:01.881348 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:01.881304    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4e5e253-04e4-4a7b-a464-0b16cd47f71d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-b7805e-tbzv2\" (UID: \"d4e5e253-04e4-4a7b-a464-0b16cd47f71d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b7805e-tbzv2"
Sep 04 22:18:01.881963 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:01.881941    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4e5e253-04e4-4a7b-a464-0b16cd47f71d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-b7805e-tbzv2\" (UID: \"d4e5e253-04e4-4a7b-a464-0b16cd47f71d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b7805e-tbzv2"
Sep 04 22:18:01.985250 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:01.985150    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b7805e-tbzv2"
Sep 04 22:18:02.258328 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:02.258291    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b7805e-tbzv2"]
Sep 04 22:18:02.260103 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:18:02.260075    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4e5e253_04e4_4a7b_a464_0b16cd47f71d.slice/crio-f60760902108165b55d9550bf79babd9224cd85e8189e10403ec1aaa00a993fe WatchSource:0}: Error finding container f60760902108165b55d9550bf79babd9224cd85e8189e10403ec1aaa00a993fe: Status 404 returned error can't find the container with id f60760902108165b55d9550bf79babd9224cd85e8189e10403ec1aaa00a993fe
Sep 04 22:18:03.172426 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:03.172386    2557 generic.go:356] "Generic (PLEG): container finished" podID="d4e5e253-04e4-4a7b-a464-0b16cd47f71d" containerID="f2da45a21b937b36726f19bfb7dff8e6972de0468f582867bd20efec9a78360b" exitCode=0
Sep 04 22:18:03.172843 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:03.172475    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b7805e-tbzv2" event={"ID":"d4e5e253-04e4-4a7b-a464-0b16cd47f71d","Type":"ContainerDied","Data":"f2da45a21b937b36726f19bfb7dff8e6972de0468f582867bd20efec9a78360b"}
Sep 04 22:18:03.172843 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:03.172531    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b7805e-tbzv2" event={"ID":"d4e5e253-04e4-4a7b-a464-0b16cd47f71d","Type":"ContainerStarted","Data":"f60760902108165b55d9550bf79babd9224cd85e8189e10403ec1aaa00a993fe"}
Sep 04 22:18:04.316790 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:04.316759    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b7805e-tbzv2"
Sep 04 22:18:04.403155 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:04.403116    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4e5e253-04e4-4a7b-a464-0b16cd47f71d-maas-api-service-ca\") pod \"d4e5e253-04e4-4a7b-a464-0b16cd47f71d\" (UID: \"d4e5e253-04e4-4a7b-a464-0b16cd47f71d\") "
Sep 04 22:18:04.403559 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:04.403534    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4e5e253-04e4-4a7b-a464-0b16cd47f71d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d4e5e253-04e4-4a7b-a464-0b16cd47f71d" (UID: "d4e5e253-04e4-4a7b-a464-0b16cd47f71d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:18:04.504732 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:04.504697    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4e5e253-04e4-4a7b-a464-0b16cd47f71d-maas-api-service-ca\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:18:05.186480 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:05.186443    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b7805e-tbzv2"
Sep 04 22:18:05.186480 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:05.186464    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b7805e-tbzv2" event={"ID":"d4e5e253-04e4-4a7b-a464-0b16cd47f71d","Type":"ContainerDied","Data":"f60760902108165b55d9550bf79babd9224cd85e8189e10403ec1aaa00a993fe"}
Sep 04 22:18:05.186688 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:05.186511    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f60760902108165b55d9550bf79babd9224cd85e8189e10403ec1aaa00a993fe"
Sep 04 22:18:07.557869 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:07.557830    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw"]
Sep 04 22:18:07.558338 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:07.558078    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw" podUID="d202a3f6-1e73-4b1b-8171-5344c3df1dea" containerName="payload-processing" containerID="cri-o://ba483bf87a0737bcee9f0f4aace3f0f3041c310b4cd2ef840aeaa0ce46849b9a" gracePeriod=30
Sep 04 22:18:07.565101 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:07.565000    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq"]
Sep 04 22:18:07.565593 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:07.565273    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq" podUID="b2b05836-c528-453c-95ea-2c4622c6020a" containerName="payload-pre-processing" containerID="cri-o://df2f7cb90b8e6fe0e432a3da8cdedd014f7248a81bf014321e44a2245eb348cd" gracePeriod=30
Sep 04 22:18:07.701256 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:07.701189    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b7805e-tbzv2"]
Sep 04 22:18:07.703818 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:07.703788    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-b7805e-tbzv2"]
Sep 04 22:18:07.877588 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:07.877559    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq"
Sep 04 22:18:07.881003 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:07.880977    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw"
Sep 04 22:18:07.942461 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:07.942405    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mw2mm\" (UniqueName: \"kubernetes.io/projected/d202a3f6-1e73-4b1b-8171-5344c3df1dea-kube-api-access-mw2mm\") pod \"d202a3f6-1e73-4b1b-8171-5344c3df1dea\" (UID: \"d202a3f6-1e73-4b1b-8171-5344c3df1dea\") "
Sep 04 22:18:07.942461 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:07.942459    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2b05836-c528-453c-95ea-2c4622c6020a-plugins-config-volume\") pod \"b2b05836-c528-453c-95ea-2c4622c6020a\" (UID: \"b2b05836-c528-453c-95ea-2c4622c6020a\") "
Sep 04 22:18:07.942726 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:07.942493    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d202a3f6-1e73-4b1b-8171-5344c3df1dea-plugins-config-volume\") pod \"d202a3f6-1e73-4b1b-8171-5344c3df1dea\" (UID: \"d202a3f6-1e73-4b1b-8171-5344c3df1dea\") "
Sep 04 22:18:07.942726 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:07.942573    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9zcv\" (UniqueName: \"kubernetes.io/projected/b2b05836-c528-453c-95ea-2c4622c6020a-kube-api-access-b9zcv\") pod \"b2b05836-c528-453c-95ea-2c4622c6020a\" (UID: \"b2b05836-c528-453c-95ea-2c4622c6020a\") "
Sep 04 22:18:07.942958 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:07.942928    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2b05836-c528-453c-95ea-2c4622c6020a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b2b05836-c528-453c-95ea-2c4622c6020a" (UID: "b2b05836-c528-453c-95ea-2c4622c6020a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:18:07.943048 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:07.942953    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d202a3f6-1e73-4b1b-8171-5344c3df1dea-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d202a3f6-1e73-4b1b-8171-5344c3df1dea" (UID: "d202a3f6-1e73-4b1b-8171-5344c3df1dea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:18:07.945236 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:07.945192    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2b05836-c528-453c-95ea-2c4622c6020a-kube-api-access-b9zcv" (OuterVolumeSpecName: "kube-api-access-b9zcv") pod "b2b05836-c528-453c-95ea-2c4622c6020a" (UID: "b2b05836-c528-453c-95ea-2c4622c6020a"). InnerVolumeSpecName "kube-api-access-b9zcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:18:07.945236 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:07.945219    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d202a3f6-1e73-4b1b-8171-5344c3df1dea-kube-api-access-mw2mm" (OuterVolumeSpecName: "kube-api-access-mw2mm") pod "d202a3f6-1e73-4b1b-8171-5344c3df1dea" (UID: "d202a3f6-1e73-4b1b-8171-5344c3df1dea"). InnerVolumeSpecName "kube-api-access-mw2mm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:18:08.044522 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.044474    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9zcv\" (UniqueName: \"kubernetes.io/projected/b2b05836-c528-453c-95ea-2c4622c6020a-kube-api-access-b9zcv\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:18:08.044522 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.044511    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mw2mm\" (UniqueName: \"kubernetes.io/projected/d202a3f6-1e73-4b1b-8171-5344c3df1dea-kube-api-access-mw2mm\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:18:08.044522 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.044521    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2b05836-c528-453c-95ea-2c4622c6020a-plugins-config-volume\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:18:08.044522 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.044530    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d202a3f6-1e73-4b1b-8171-5344c3df1dea-plugins-config-volume\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:18:08.205171 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.205082    2557 generic.go:356] "Generic (PLEG): container finished" podID="d202a3f6-1e73-4b1b-8171-5344c3df1dea" containerID="ba483bf87a0737bcee9f0f4aace3f0f3041c310b4cd2ef840aeaa0ce46849b9a" exitCode=0
Sep 04 22:18:08.205171 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.205146    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw"
Sep 04 22:18:08.205356 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.205165    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw" event={"ID":"d202a3f6-1e73-4b1b-8171-5344c3df1dea","Type":"ContainerDied","Data":"ba483bf87a0737bcee9f0f4aace3f0f3041c310b4cd2ef840aeaa0ce46849b9a"}
Sep 04 22:18:08.205356 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.205253    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw" event={"ID":"d202a3f6-1e73-4b1b-8171-5344c3df1dea","Type":"ContainerDied","Data":"9a0aaa7f98af74631767b21573d6d89bc97bcf84a545bf66cf16e8a516e4cb27"}
Sep 04 22:18:08.205356 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.205278    2557 scope.go:117] "RemoveContainer" containerID="ba483bf87a0737bcee9f0f4aace3f0f3041c310b4cd2ef840aeaa0ce46849b9a"
Sep 04 22:18:08.206757 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.206730    2557 generic.go:356] "Generic (PLEG): container finished" podID="b2b05836-c528-453c-95ea-2c4622c6020a" containerID="df2f7cb90b8e6fe0e432a3da8cdedd014f7248a81bf014321e44a2245eb348cd" exitCode=0
Sep 04 22:18:08.206895 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.206805    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq"
Sep 04 22:18:08.206895 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.206834    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq" event={"ID":"b2b05836-c528-453c-95ea-2c4622c6020a","Type":"ContainerDied","Data":"df2f7cb90b8e6fe0e432a3da8cdedd014f7248a81bf014321e44a2245eb348cd"}
Sep 04 22:18:08.206982 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.206892    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq" event={"ID":"b2b05836-c528-453c-95ea-2c4622c6020a","Type":"ContainerDied","Data":"df9d001d8a886c2951fe992a0e973e9a853d0a91a82fa2557a04b6187e68f564"}
Sep 04 22:18:08.216958 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.216935    2557 scope.go:117] "RemoveContainer" containerID="ba483bf87a0737bcee9f0f4aace3f0f3041c310b4cd2ef840aeaa0ce46849b9a"
Sep 04 22:18:08.217337 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:18:08.217305    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba483bf87a0737bcee9f0f4aace3f0f3041c310b4cd2ef840aeaa0ce46849b9a\": container with ID starting with ba483bf87a0737bcee9f0f4aace3f0f3041c310b4cd2ef840aeaa0ce46849b9a not found: ID does not exist" containerID="ba483bf87a0737bcee9f0f4aace3f0f3041c310b4cd2ef840aeaa0ce46849b9a"
Sep 04 22:18:08.217457 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.217349    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba483bf87a0737bcee9f0f4aace3f0f3041c310b4cd2ef840aeaa0ce46849b9a"} err="failed to get container status \"ba483bf87a0737bcee9f0f4aace3f0f3041c310b4cd2ef840aeaa0ce46849b9a\": rpc error: code = NotFound desc = could not find container \"ba483bf87a0737bcee9f0f4aace3f0f3041c310b4cd2ef840aeaa0ce46849b9a\": container with ID starting with ba483bf87a0737bcee9f0f4aace3f0f3041c310b4cd2ef840aeaa0ce46849b9a not found: ID does not exist"
Sep 04 22:18:08.217457 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.217374    2557 scope.go:117] "RemoveContainer" containerID="df2f7cb90b8e6fe0e432a3da8cdedd014f7248a81bf014321e44a2245eb348cd"
Sep 04 22:18:08.226558 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.226534    2557 scope.go:117] "RemoveContainer" containerID="df2f7cb90b8e6fe0e432a3da8cdedd014f7248a81bf014321e44a2245eb348cd"
Sep 04 22:18:08.226934 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:18:08.226912    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df2f7cb90b8e6fe0e432a3da8cdedd014f7248a81bf014321e44a2245eb348cd\": container with ID starting with df2f7cb90b8e6fe0e432a3da8cdedd014f7248a81bf014321e44a2245eb348cd not found: ID does not exist" containerID="df2f7cb90b8e6fe0e432a3da8cdedd014f7248a81bf014321e44a2245eb348cd"
Sep 04 22:18:08.226984 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.226943    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df2f7cb90b8e6fe0e432a3da8cdedd014f7248a81bf014321e44a2245eb348cd"} err="failed to get container status \"df2f7cb90b8e6fe0e432a3da8cdedd014f7248a81bf014321e44a2245eb348cd\": rpc error: code = NotFound desc = could not find container \"df2f7cb90b8e6fe0e432a3da8cdedd014f7248a81bf014321e44a2245eb348cd\": container with ID starting with df2f7cb90b8e6fe0e432a3da8cdedd014f7248a81bf014321e44a2245eb348cd not found: ID does not exist"
Sep 04 22:18:08.231348 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.231314    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw"]
Sep 04 22:18:08.235435 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.235390    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-b7805e-7fff88ccd9-8f6nw"]
Sep 04 22:18:08.245075 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.245040    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq"]
Sep 04 22:18:08.251136 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.251102    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-b7805e-5c46945674-9dvzq"]
Sep 04 22:18:08.353763 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.353726    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2b05836-c528-453c-95ea-2c4622c6020a" path="/var/lib/kubelet/pods/b2b05836-c528-453c-95ea-2c4622c6020a/volumes"
Sep 04 22:18:08.354121 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.354107    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d202a3f6-1e73-4b1b-8171-5344c3df1dea" path="/var/lib/kubelet/pods/d202a3f6-1e73-4b1b-8171-5344c3df1dea/volumes"
Sep 04 22:18:08.354406 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:08.354395    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4e5e253-04e4-4a7b-a464-0b16cd47f71d" path="/var/lib/kubelet/pods/d4e5e253-04e4-4a7b-a464-0b16cd47f71d/volumes"
Sep 04 22:18:11.247260 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:11.247222    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:18:17.640575 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:17.640536    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cc92e2-bv22m"]
Sep 04 22:18:17.640974 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:17.640906    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d202a3f6-1e73-4b1b-8171-5344c3df1dea" containerName="payload-processing"
Sep 04 22:18:17.640974 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:17.640917    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d202a3f6-1e73-4b1b-8171-5344c3df1dea" containerName="payload-processing"
Sep 04 22:18:17.640974 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:17.640938    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4e5e253-04e4-4a7b-a464-0b16cd47f71d" containerName="revoke-keys"
Sep 04 22:18:17.640974 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:17.640943    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4e5e253-04e4-4a7b-a464-0b16cd47f71d" containerName="revoke-keys"
Sep 04 22:18:17.640974 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:17.640950    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2b05836-c528-453c-95ea-2c4622c6020a" containerName="payload-pre-processing"
Sep 04 22:18:17.640974 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:17.640955    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2b05836-c528-453c-95ea-2c4622c6020a" containerName="payload-pre-processing"
Sep 04 22:18:17.641159 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:17.641013    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2b05836-c528-453c-95ea-2c4622c6020a" containerName="payload-pre-processing"
Sep 04 22:18:17.641159 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:17.641024    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d202a3f6-1e73-4b1b-8171-5344c3df1dea" containerName="payload-processing"
Sep 04 22:18:17.641159 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:17.641031    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4e5e253-04e4-4a7b-a464-0b16cd47f71d" containerName="revoke-keys"
Sep 04 22:18:17.643963 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:17.643941    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cc92e2-bv22m"
Sep 04 22:18:17.646305 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:17.646280    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 04 22:18:17.646398 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:17.646338    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-dftg6\""
Sep 04 22:18:17.654918 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:17.654878    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cc92e2-bv22m"]
Sep 04 22:18:17.739285 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:17.739240    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc47bef7-4146-44ed-b36b-35ffb3142b68-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-cc92e2-bv22m\" (UID: \"dc47bef7-4146-44ed-b36b-35ffb3142b68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cc92e2-bv22m"
Sep 04 22:18:17.840540 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:17.840497    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc47bef7-4146-44ed-b36b-35ffb3142b68-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-cc92e2-bv22m\" (UID: \"dc47bef7-4146-44ed-b36b-35ffb3142b68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cc92e2-bv22m"
Sep 04 22:18:17.841209 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:17.841183    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc47bef7-4146-44ed-b36b-35ffb3142b68-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-cc92e2-bv22m\" (UID: \"dc47bef7-4146-44ed-b36b-35ffb3142b68\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cc92e2-bv22m"
Sep 04 22:18:17.962632 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:17.962537    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cc92e2-bv22m"
Sep 04 22:18:18.236780 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:18.236753    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cc92e2-bv22m"]
Sep 04 22:18:18.239190 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:18:18.239156    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc47bef7_4146_44ed_b36b_35ffb3142b68.slice/crio-d3aaa0946b08e49dc47bfe274defcf9963fbd9b83f0af7f1ce75d5cae85e716f WatchSource:0}: Error finding container d3aaa0946b08e49dc47bfe274defcf9963fbd9b83f0af7f1ce75d5cae85e716f: Status 404 returned error can't find the container with id d3aaa0946b08e49dc47bfe274defcf9963fbd9b83f0af7f1ce75d5cae85e716f
Sep 04 22:18:18.271404 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:18.271366    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cc92e2-bv22m" event={"ID":"dc47bef7-4146-44ed-b36b-35ffb3142b68","Type":"ContainerStarted","Data":"d3aaa0946b08e49dc47bfe274defcf9963fbd9b83f0af7f1ce75d5cae85e716f"}
Sep 04 22:18:19.279723 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:19.279682    2557 generic.go:356] "Generic (PLEG): container finished" podID="dc47bef7-4146-44ed-b36b-35ffb3142b68" containerID="aabb20b64110f64593119c8db5a3e0e4a1476a2bcd583aa8dc890637ca60bb80" exitCode=0
Sep 04 22:18:19.280123 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:19.279771    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cc92e2-bv22m" event={"ID":"dc47bef7-4146-44ed-b36b-35ffb3142b68","Type":"ContainerDied","Data":"aabb20b64110f64593119c8db5a3e0e4a1476a2bcd583aa8dc890637ca60bb80"}
Sep 04 22:18:20.423818 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:20.423790    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cc92e2-bv22m"
Sep 04 22:18:20.569832 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:20.569736    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc47bef7-4146-44ed-b36b-35ffb3142b68-maas-api-service-ca\") pod \"dc47bef7-4146-44ed-b36b-35ffb3142b68\" (UID: \"dc47bef7-4146-44ed-b36b-35ffb3142b68\") "
Sep 04 22:18:20.570126 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:20.570099    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc47bef7-4146-44ed-b36b-35ffb3142b68-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dc47bef7-4146-44ed-b36b-35ffb3142b68" (UID: "dc47bef7-4146-44ed-b36b-35ffb3142b68"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:18:20.671380 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:20.671335    2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc47bef7-4146-44ed-b36b-35ffb3142b68-maas-api-service-ca\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:18:21.292294 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:21.292263    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cc92e2-bv22m"
Sep 04 22:18:21.292497 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:21.292301    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cc92e2-bv22m" event={"ID":"dc47bef7-4146-44ed-b36b-35ffb3142b68","Type":"ContainerDied","Data":"d3aaa0946b08e49dc47bfe274defcf9963fbd9b83f0af7f1ce75d5cae85e716f"}
Sep 04 22:18:21.292497 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:21.292347    2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d3aaa0946b08e49dc47bfe274defcf9963fbd9b83f0af7f1ce75d5cae85e716f"
Sep 04 22:18:30.446304 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:30.446260    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:18:33.433094 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:33.433052    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cc92e2-bv22m"]
Sep 04 22:18:33.435421 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:33.435389    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-cc92e2-bv22m"]
Sep 04 22:18:34.353338 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:34.353286    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc47bef7-4146-44ed-b36b-35ffb3142b68" path="/var/lib/kubelet/pods/dc47bef7-4146-44ed-b36b-35ffb3142b68/volumes"
Sep 04 22:18:38.786561 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:38.786521    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"]
Sep 04 22:18:41.341781 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:41.341739    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:18:53.806387 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:53.806335    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm" podUID="b4c0465a-eeb5-435f-be67-ac5f9e5c54b0" containerName="main" containerID="cri-o://27f5e72017512fc11008da2d5758ddf55a9b799acb9288fe5625c7f7e9846e5e" gracePeriod=45
Sep 04 22:18:54.063467 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.063381    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:18:54.189884 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.189846    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-home\") pod \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") "
Sep 04 22:18:54.190109 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.189924    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-model-cache\") pod \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") "
Sep 04 22:18:54.190109 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.189966    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d28zt\" (UniqueName: \"kubernetes.io/projected/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-kube-api-access-d28zt\") pod \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") "
Sep 04 22:18:54.190109 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.190030    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-dshm\") pod \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") "
Sep 04 22:18:54.190109 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.190066    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-tmp-dir\") pod \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") "
Sep 04 22:18:54.190314 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.190119    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-tls-certs\") pod \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\" (UID: \"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0\") "
Sep 04 22:18:54.190401 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.190368    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-model-cache" (OuterVolumeSpecName: "model-cache") pod "b4c0465a-eeb5-435f-be67-ac5f9e5c54b0" (UID: "b4c0465a-eeb5-435f-be67-ac5f9e5c54b0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:18:54.190878 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.190550    2557 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-model-cache\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:18:54.190878 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.190823    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-home" (OuterVolumeSpecName: "home") pod "b4c0465a-eeb5-435f-be67-ac5f9e5c54b0" (UID: "b4c0465a-eeb5-435f-be67-ac5f9e5c54b0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:18:54.190878 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.190840    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b4c0465a-eeb5-435f-be67-ac5f9e5c54b0" (UID: "b4c0465a-eeb5-435f-be67-ac5f9e5c54b0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:18:54.193185 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.193149    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b4c0465a-eeb5-435f-be67-ac5f9e5c54b0" (UID: "b4c0465a-eeb5-435f-be67-ac5f9e5c54b0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 04 22:18:54.193306 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.193194    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-dshm" (OuterVolumeSpecName: "dshm") pod "b4c0465a-eeb5-435f-be67-ac5f9e5c54b0" (UID: "b4c0465a-eeb5-435f-be67-ac5f9e5c54b0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:18:54.193306 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.193219    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-kube-api-access-d28zt" (OuterVolumeSpecName: "kube-api-access-d28zt") pod "b4c0465a-eeb5-435f-be67-ac5f9e5c54b0" (UID: "b4c0465a-eeb5-435f-be67-ac5f9e5c54b0"). InnerVolumeSpecName "kube-api-access-d28zt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:18:54.291810 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.291762    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d28zt\" (UniqueName: \"kubernetes.io/projected/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-kube-api-access-d28zt\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:18:54.291810 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.291804    2557 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-dshm\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:18:54.291810 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.291817    2557 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-tmp-dir\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:18:54.292080 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.291828    2557 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-tls-certs\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:18:54.292080 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.291840    2557 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0-home\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:18:54.494435 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.494372    2557 generic.go:356] "Generic (PLEG): container finished" podID="b4c0465a-eeb5-435f-be67-ac5f9e5c54b0" containerID="27f5e72017512fc11008da2d5758ddf55a9b799acb9288fe5625c7f7e9846e5e" exitCode=0
Sep 04 22:18:54.494435 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.494434    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"
Sep 04 22:18:54.494684 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.494459    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm" event={"ID":"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0","Type":"ContainerDied","Data":"27f5e72017512fc11008da2d5758ddf55a9b799acb9288fe5625c7f7e9846e5e"}
Sep 04 22:18:54.494684 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.494505    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm" event={"ID":"b4c0465a-eeb5-435f-be67-ac5f9e5c54b0","Type":"ContainerDied","Data":"d3372c444a738b13d15ea0293d8c2ab28eb5941415676f299537c8d5af5ded28"}
Sep 04 22:18:54.494684 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.494528    2557 scope.go:117] "RemoveContainer" containerID="27f5e72017512fc11008da2d5758ddf55a9b799acb9288fe5625c7f7e9846e5e"
Sep 04 22:18:54.503769 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.503749    2557 scope.go:117] "RemoveContainer" containerID="27f5e72017512fc11008da2d5758ddf55a9b799acb9288fe5625c7f7e9846e5e"
Sep 04 22:18:54.504108 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:18:54.504070    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27f5e72017512fc11008da2d5758ddf55a9b799acb9288fe5625c7f7e9846e5e\": container with ID starting with 27f5e72017512fc11008da2d5758ddf55a9b799acb9288fe5625c7f7e9846e5e not found: ID does not exist" containerID="27f5e72017512fc11008da2d5758ddf55a9b799acb9288fe5625c7f7e9846e5e"
Sep 04 22:18:54.504158 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.504116    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27f5e72017512fc11008da2d5758ddf55a9b799acb9288fe5625c7f7e9846e5e"} err="failed to get container status \"27f5e72017512fc11008da2d5758ddf55a9b799acb9288fe5625c7f7e9846e5e\": rpc error: code = NotFound desc = could not find container \"27f5e72017512fc11008da2d5758ddf55a9b799acb9288fe5625c7f7e9846e5e\": container with ID starting with 27f5e72017512fc11008da2d5758ddf55a9b799acb9288fe5625c7f7e9846e5e not found: ID does not exist"
Sep 04 22:18:54.511434 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.511393    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"]
Sep 04 22:18:54.516387 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:54.516359    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-cc92e2/ipp-route-cc92e2-kserve-769bfbb7bc-mlbpm"]
Sep 04 22:18:56.353012 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:18:56.352973    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4c0465a-eeb5-435f-be67-ac5f9e5c54b0" path="/var/lib/kubelet/pods/b4c0465a-eeb5-435f-be67-ac5f9e5c54b0/volumes"
Sep 04 22:19:21.876907 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:21.876868    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"]
Sep 04 22:19:21.877404 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:21.877127    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j" podUID="2efcc572-0661-4dd9-98b1-b5816bd4b037" containerName="istio-proxy" containerID="cri-o://0f6b69adf3d1256cdc402b50f5cd9d76ca6c55bd7c66ddd0c57d4ae908dc45fb" gracePeriod=30
Sep 04 22:19:27.134652 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.134616    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:19:27.208510 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.208378    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-workload-certs\") pod \"2efcc572-0661-4dd9-98b1-b5816bd4b037\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") "
Sep 04 22:19:27.208510 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.208500    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-token\") pod \"2efcc572-0661-4dd9-98b1-b5816bd4b037\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") "
Sep 04 22:19:27.208745 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.208532    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2efcc572-0661-4dd9-98b1-b5816bd4b037-istiod-ca-cert\") pod \"2efcc572-0661-4dd9-98b1-b5816bd4b037\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") "
Sep 04 22:19:27.208745 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.208571    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-podinfo\") pod \"2efcc572-0661-4dd9-98b1-b5816bd4b037\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") "
Sep 04 22:19:27.208745 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.208595    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-credential-socket\") pod \"2efcc572-0661-4dd9-98b1-b5816bd4b037\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") "
Sep 04 22:19:27.208745 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.208618    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-workload-socket\") pod \"2efcc572-0661-4dd9-98b1-b5816bd4b037\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") "
Sep 04 22:19:27.208745 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.208648    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-data\") pod \"2efcc572-0661-4dd9-98b1-b5816bd4b037\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") "
Sep 04 22:19:27.208745 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.208670    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-envoy\") pod \"2efcc572-0661-4dd9-98b1-b5816bd4b037\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") "
Sep 04 22:19:27.208745 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.208719    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8gcv\" (UniqueName: \"kubernetes.io/projected/2efcc572-0661-4dd9-98b1-b5816bd4b037-kube-api-access-g8gcv\") pod \"2efcc572-0661-4dd9-98b1-b5816bd4b037\" (UID: \"2efcc572-0661-4dd9-98b1-b5816bd4b037\") "
Sep 04 22:19:27.209113 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.208740    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2efcc572-0661-4dd9-98b1-b5816bd4b037" (UID: "2efcc572-0661-4dd9-98b1-b5816bd4b037"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:19:27.209113 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.208887    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2efcc572-0661-4dd9-98b1-b5816bd4b037" (UID: "2efcc572-0661-4dd9-98b1-b5816bd4b037"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:19:27.209113 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.208966    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2efcc572-0661-4dd9-98b1-b5816bd4b037" (UID: "2efcc572-0661-4dd9-98b1-b5816bd4b037"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:19:27.209113 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.209035    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-data" (OuterVolumeSpecName: "istio-data") pod "2efcc572-0661-4dd9-98b1-b5816bd4b037" (UID: "2efcc572-0661-4dd9-98b1-b5816bd4b037"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:19:27.209113 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.209065    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2efcc572-0661-4dd9-98b1-b5816bd4b037-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2efcc572-0661-4dd9-98b1-b5816bd4b037" (UID: "2efcc572-0661-4dd9-98b1-b5816bd4b037"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:19:27.209113 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.209095    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-credential-socket\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:19:27.209113 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.209110    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-workload-socket\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:19:27.209435 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.209121    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-workload-certs\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:19:27.210906 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.210880    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-token" (OuterVolumeSpecName: "istio-token") pod "2efcc572-0661-4dd9-98b1-b5816bd4b037" (UID: "2efcc572-0661-4dd9-98b1-b5816bd4b037"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:19:27.210906 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.210892    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2efcc572-0661-4dd9-98b1-b5816bd4b037" (UID: "2efcc572-0661-4dd9-98b1-b5816bd4b037"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 04 22:19:27.210906 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.210880    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2efcc572-0661-4dd9-98b1-b5816bd4b037" (UID: "2efcc572-0661-4dd9-98b1-b5816bd4b037"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:19:27.211110 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.210985    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2efcc572-0661-4dd9-98b1-b5816bd4b037-kube-api-access-g8gcv" (OuterVolumeSpecName: "kube-api-access-g8gcv") pod "2efcc572-0661-4dd9-98b1-b5816bd4b037" (UID: "2efcc572-0661-4dd9-98b1-b5816bd4b037"). InnerVolumeSpecName "kube-api-access-g8gcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:19:27.310093 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.310050    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-token\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:19:27.310093 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.310087    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2efcc572-0661-4dd9-98b1-b5816bd4b037-istiod-ca-cert\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:19:27.310093 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.310096    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-podinfo\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:19:27.310093 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.310103    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-data\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:19:27.310366 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.310114    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2efcc572-0661-4dd9-98b1-b5816bd4b037-istio-envoy\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:19:27.310366 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.310121    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8gcv\" (UniqueName: \"kubernetes.io/projected/2efcc572-0661-4dd9-98b1-b5816bd4b037-kube-api-access-g8gcv\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:19:27.703910 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.703871    2557 generic.go:356] "Generic (PLEG): container finished" podID="2efcc572-0661-4dd9-98b1-b5816bd4b037" containerID="0f6b69adf3d1256cdc402b50f5cd9d76ca6c55bd7c66ddd0c57d4ae908dc45fb" exitCode=0
Sep 04 22:19:27.704090 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.703927    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"
Sep 04 22:19:27.704090 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.703948    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j" event={"ID":"2efcc572-0661-4dd9-98b1-b5816bd4b037","Type":"ContainerDied","Data":"0f6b69adf3d1256cdc402b50f5cd9d76ca6c55bd7c66ddd0c57d4ae908dc45fb"}
Sep 04 22:19:27.704090 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.704031    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j" event={"ID":"2efcc572-0661-4dd9-98b1-b5816bd4b037","Type":"ContainerDied","Data":"2278a90f7dda0d44fce7215338cc84a7f88487970d9d038c971604c6511d74be"}
Sep 04 22:19:27.704090 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.704056    2557 scope.go:117] "RemoveContainer" containerID="0f6b69adf3d1256cdc402b50f5cd9d76ca6c55bd7c66ddd0c57d4ae908dc45fb"
Sep 04 22:19:27.714896 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.714872    2557 scope.go:117] "RemoveContainer" containerID="8c02e29a1e4330ef9a790785b9104f53c5702b32cd4124a82bde8433194b010c"
Sep 04 22:19:27.723970 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.723948    2557 scope.go:117] "RemoveContainer" containerID="0f6b69adf3d1256cdc402b50f5cd9d76ca6c55bd7c66ddd0c57d4ae908dc45fb"
Sep 04 22:19:27.724281 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:19:27.724257    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f6b69adf3d1256cdc402b50f5cd9d76ca6c55bd7c66ddd0c57d4ae908dc45fb\": container with ID starting with 0f6b69adf3d1256cdc402b50f5cd9d76ca6c55bd7c66ddd0c57d4ae908dc45fb not found: ID does not exist" containerID="0f6b69adf3d1256cdc402b50f5cd9d76ca6c55bd7c66ddd0c57d4ae908dc45fb"
Sep 04 22:19:27.724350 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.724290    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f6b69adf3d1256cdc402b50f5cd9d76ca6c55bd7c66ddd0c57d4ae908dc45fb"} err="failed to get container status \"0f6b69adf3d1256cdc402b50f5cd9d76ca6c55bd7c66ddd0c57d4ae908dc45fb\": rpc error: code = NotFound desc = could not find container \"0f6b69adf3d1256cdc402b50f5cd9d76ca6c55bd7c66ddd0c57d4ae908dc45fb\": container with ID starting with 0f6b69adf3d1256cdc402b50f5cd9d76ca6c55bd7c66ddd0c57d4ae908dc45fb not found: ID does not exist"
Sep 04 22:19:27.724350 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.724310    2557 scope.go:117] "RemoveContainer" containerID="8c02e29a1e4330ef9a790785b9104f53c5702b32cd4124a82bde8433194b010c"
Sep 04 22:19:27.724626 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:19:27.724606    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c02e29a1e4330ef9a790785b9104f53c5702b32cd4124a82bde8433194b010c\": container with ID starting with 8c02e29a1e4330ef9a790785b9104f53c5702b32cd4124a82bde8433194b010c not found: ID does not exist" containerID="8c02e29a1e4330ef9a790785b9104f53c5702b32cd4124a82bde8433194b010c"
Sep 04 22:19:27.724675 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.724635    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c02e29a1e4330ef9a790785b9104f53c5702b32cd4124a82bde8433194b010c"} err="failed to get container status \"8c02e29a1e4330ef9a790785b9104f53c5702b32cd4124a82bde8433194b010c\": rpc error: code = NotFound desc = could not find container \"8c02e29a1e4330ef9a790785b9104f53c5702b32cd4124a82bde8433194b010c\": container with ID starting with 8c02e29a1e4330ef9a790785b9104f53c5702b32cd4124a82bde8433194b010c not found: ID does not exist"
Sep 04 22:19:27.727966 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.727941    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"]
Sep 04 22:19:27.732472 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:27.732445    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-cc92e2-openshift-default-584848fff8-xgg4j"]
Sep 04 22:19:28.353160 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:28.353122    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2efcc572-0661-4dd9-98b1-b5816bd4b037" path="/var/lib/kubelet/pods/2efcc572-0661-4dd9-98b1-b5816bd4b037/volumes"
Sep 04 22:19:35.246352 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:35.246314    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:19:39.218778 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:39.218741    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"]
Sep 04 22:19:39.219207 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:39.218986    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr" podUID="c93d0e5c-af64-4138-9542-13b6c1042254" containerName="istio-proxy" containerID="cri-o://e0ebb65adab96dbd2feae6531c8a4e5b025cddb3bc05f0aba8f7c0a56795d22e" gracePeriod=30
Sep 04 22:19:42.604820 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:42.604782    2557 patch_prober.go:28] interesting pod/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 04 22:19:42.605191 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:42.604849    2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr" podUID="c93d0e5c-af64-4138-9542-13b6c1042254" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 04 22:19:44.477441 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.477391    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:19:44.582818 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.582779    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-credential-socket\") pod \"c93d0e5c-af64-4138-9542-13b6c1042254\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") "
Sep 04 22:19:44.582818 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.582837    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-workload-certs\") pod \"c93d0e5c-af64-4138-9542-13b6c1042254\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") "
Sep 04 22:19:44.583078 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.582863    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-istio-data\") pod \"c93d0e5c-af64-4138-9542-13b6c1042254\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") "
Sep 04 22:19:44.583078 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.582881    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqwvw\" (UniqueName: \"kubernetes.io/projected/c93d0e5c-af64-4138-9542-13b6c1042254-kube-api-access-mqwvw\") pod \"c93d0e5c-af64-4138-9542-13b6c1042254\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") "
Sep 04 22:19:44.583078 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.582922    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-istio-envoy\") pod \"c93d0e5c-af64-4138-9542-13b6c1042254\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") "
Sep 04 22:19:44.583078 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.582954    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c93d0e5c-af64-4138-9542-13b6c1042254-istio-token\") pod \"c93d0e5c-af64-4138-9542-13b6c1042254\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") "
Sep 04 22:19:44.583078 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.583011    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c93d0e5c-af64-4138-9542-13b6c1042254-istio-podinfo\") pod \"c93d0e5c-af64-4138-9542-13b6c1042254\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") "
Sep 04 22:19:44.583078 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.583036    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-workload-socket\") pod \"c93d0e5c-af64-4138-9542-13b6c1042254\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") "
Sep 04 22:19:44.583303 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.583088    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c93d0e5c-af64-4138-9542-13b6c1042254-istiod-ca-cert\") pod \"c93d0e5c-af64-4138-9542-13b6c1042254\" (UID: \"c93d0e5c-af64-4138-9542-13b6c1042254\") "
Sep 04 22:19:44.583303 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.583128    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c93d0e5c-af64-4138-9542-13b6c1042254" (UID: "c93d0e5c-af64-4138-9542-13b6c1042254"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:19:44.583303 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.583167    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-istio-data" (OuterVolumeSpecName: "istio-data") pod "c93d0e5c-af64-4138-9542-13b6c1042254" (UID: "c93d0e5c-af64-4138-9542-13b6c1042254"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:19:44.583303 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.583264    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c93d0e5c-af64-4138-9542-13b6c1042254" (UID: "c93d0e5c-af64-4138-9542-13b6c1042254"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:19:44.583540 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.583365    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c93d0e5c-af64-4138-9542-13b6c1042254" (UID: "c93d0e5c-af64-4138-9542-13b6c1042254"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:19:44.583540 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.583505    2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-credential-socket\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:19:44.583540 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.583525    2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-workload-certs\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:19:44.583540 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.583539    2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-istio-data\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:19:44.583883 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.583555    2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-workload-socket\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:19:44.583883 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.583596    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c93d0e5c-af64-4138-9542-13b6c1042254-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c93d0e5c-af64-4138-9542-13b6c1042254" (UID: "c93d0e5c-af64-4138-9542-13b6c1042254"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:19:44.585364 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.585336    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c93d0e5c-af64-4138-9542-13b6c1042254" (UID: "c93d0e5c-af64-4138-9542-13b6c1042254"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 04 22:19:44.585487 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.585388    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c93d0e5c-af64-4138-9542-13b6c1042254-kube-api-access-mqwvw" (OuterVolumeSpecName: "kube-api-access-mqwvw") pod "c93d0e5c-af64-4138-9542-13b6c1042254" (UID: "c93d0e5c-af64-4138-9542-13b6c1042254"). InnerVolumeSpecName "kube-api-access-mqwvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:19:44.585799 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.585772    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c93d0e5c-af64-4138-9542-13b6c1042254-istio-token" (OuterVolumeSpecName: "istio-token") pod "c93d0e5c-af64-4138-9542-13b6c1042254" (UID: "c93d0e5c-af64-4138-9542-13b6c1042254"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:19:44.585886 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.585796    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c93d0e5c-af64-4138-9542-13b6c1042254-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c93d0e5c-af64-4138-9542-13b6c1042254" (UID: "c93d0e5c-af64-4138-9542-13b6c1042254"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 04 22:19:44.684696 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.684656    2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c93d0e5c-af64-4138-9542-13b6c1042254-istio-token\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:19:44.684696 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.684691    2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c93d0e5c-af64-4138-9542-13b6c1042254-istio-podinfo\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:19:44.684696 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.684701    2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c93d0e5c-af64-4138-9542-13b6c1042254-istiod-ca-cert\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:19:44.684939 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.684710    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqwvw\" (UniqueName: \"kubernetes.io/projected/c93d0e5c-af64-4138-9542-13b6c1042254-kube-api-access-mqwvw\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:19:44.684939 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.684722    2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c93d0e5c-af64-4138-9542-13b6c1042254-istio-envoy\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:19:44.811334 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.811296    2557 generic.go:356] "Generic (PLEG): container finished" podID="c93d0e5c-af64-4138-9542-13b6c1042254" containerID="e0ebb65adab96dbd2feae6531c8a4e5b025cddb3bc05f0aba8f7c0a56795d22e" exitCode=0
Sep 04 22:19:44.811552 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.811341    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"
Sep 04 22:19:44.811552 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.811378    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr" event={"ID":"c93d0e5c-af64-4138-9542-13b6c1042254","Type":"ContainerDied","Data":"e0ebb65adab96dbd2feae6531c8a4e5b025cddb3bc05f0aba8f7c0a56795d22e"}
Sep 04 22:19:44.811552 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.811466    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr" event={"ID":"c93d0e5c-af64-4138-9542-13b6c1042254","Type":"ContainerDied","Data":"ddc1bfccb797d9a8f03fdac05272dca8443d4ce76169010ed2f5a17d1dcb4afb"}
Sep 04 22:19:44.811552 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.811503    2557 scope.go:117] "RemoveContainer" containerID="e0ebb65adab96dbd2feae6531c8a4e5b025cddb3bc05f0aba8f7c0a56795d22e"
Sep 04 22:19:44.822029 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.822007    2557 scope.go:117] "RemoveContainer" containerID="a985b151e803fbe1bce0897c228e41ec24a7cf4f4150170d3485e95aaf383a6e"
Sep 04 22:19:44.831562 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.831535    2557 scope.go:117] "RemoveContainer" containerID="e0ebb65adab96dbd2feae6531c8a4e5b025cddb3bc05f0aba8f7c0a56795d22e"
Sep 04 22:19:44.831927 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:19:44.831890    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0ebb65adab96dbd2feae6531c8a4e5b025cddb3bc05f0aba8f7c0a56795d22e\": container with ID starting with e0ebb65adab96dbd2feae6531c8a4e5b025cddb3bc05f0aba8f7c0a56795d22e not found: ID does not exist" containerID="e0ebb65adab96dbd2feae6531c8a4e5b025cddb3bc05f0aba8f7c0a56795d22e"
Sep 04 22:19:44.832030 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.831921    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0ebb65adab96dbd2feae6531c8a4e5b025cddb3bc05f0aba8f7c0a56795d22e"} err="failed to get container status \"e0ebb65adab96dbd2feae6531c8a4e5b025cddb3bc05f0aba8f7c0a56795d22e\": rpc error: code = NotFound desc = could not find container \"e0ebb65adab96dbd2feae6531c8a4e5b025cddb3bc05f0aba8f7c0a56795d22e\": container with ID starting with e0ebb65adab96dbd2feae6531c8a4e5b025cddb3bc05f0aba8f7c0a56795d22e not found: ID does not exist"
Sep 04 22:19:44.832030 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.831938    2557 scope.go:117] "RemoveContainer" containerID="a985b151e803fbe1bce0897c228e41ec24a7cf4f4150170d3485e95aaf383a6e"
Sep 04 22:19:44.832267 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:19:44.832239    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a985b151e803fbe1bce0897c228e41ec24a7cf4f4150170d3485e95aaf383a6e\": container with ID starting with a985b151e803fbe1bce0897c228e41ec24a7cf4f4150170d3485e95aaf383a6e not found: ID does not exist" containerID="a985b151e803fbe1bce0897c228e41ec24a7cf4f4150170d3485e95aaf383a6e"
Sep 04 22:19:44.832308 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.832278    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a985b151e803fbe1bce0897c228e41ec24a7cf4f4150170d3485e95aaf383a6e"} err="failed to get container status \"a985b151e803fbe1bce0897c228e41ec24a7cf4f4150170d3485e95aaf383a6e\": rpc error: code = NotFound desc = could not find container \"a985b151e803fbe1bce0897c228e41ec24a7cf4f4150170d3485e95aaf383a6e\": container with ID starting with a985b151e803fbe1bce0897c228e41ec24a7cf4f4150170d3485e95aaf383a6e not found: ID does not exist"
Sep 04 22:19:44.833538 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.833493    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"]
Sep 04 22:19:44.839863 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:44.839833    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-f2fbdf-openshift-default-5fb7c78cf6-xzbqr"]
Sep 04 22:19:46.354118 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:46.354075    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c93d0e5c-af64-4138-9542-13b6c1042254" path="/var/lib/kubelet/pods/c93d0e5c-af64-4138-9542-13b6c1042254/volumes"
Sep 04 22:19:46.554705 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:46.554673    2557 scope.go:117] "RemoveContainer" containerID="2e829cc83be95667bfb16dbf6c9bd76ea758d2323e2cc43d3be2af293fcca8d2"
Sep 04 22:19:52.446763 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:19:52.446702    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:20:07.913057 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:07.913005    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7a26c9-29809335-tgggm"]
Sep 04 22:20:07.915454 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:07.915425    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-7a26c9-29809335-tgggm"]
Sep 04 22:20:07.935767 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:07.935725    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4"]
Sep 04 22:20:07.936068 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:07.936033    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4" podUID="4c1b06e8-98cb-4c9b-badf-6dc8e846e184" containerName="payload-processing" containerID="cri-o://51f7e815e79adfc17065ae77f436254a2f971e37637524db7b7aae425b8532f6" gracePeriod=30
Sep 04 22:20:07.944708 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:07.944601    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx"]
Sep 04 22:20:07.944935 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:07.944864    2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx" podUID="1d3f722b-6821-4b1e-9404-9a07af50a378" containerName="payload-pre-processing" containerID="cri-o://25c9501fb4b90790235344bac6be3951dece0e2514936e1a655a5289a6cba6c8" gracePeriod=30
Sep 04 22:20:08.216591 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.216564    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4"
Sep 04 22:20:08.219765 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.219742    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx"
Sep 04 22:20:08.294341 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.294298    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c1b06e8-98cb-4c9b-badf-6dc8e846e184-plugins-config-volume\") pod \"4c1b06e8-98cb-4c9b-badf-6dc8e846e184\" (UID: \"4c1b06e8-98cb-4c9b-badf-6dc8e846e184\") "
Sep 04 22:20:08.294581 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.294360    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d3f722b-6821-4b1e-9404-9a07af50a378-plugins-config-volume\") pod \"1d3f722b-6821-4b1e-9404-9a07af50a378\" (UID: \"1d3f722b-6821-4b1e-9404-9a07af50a378\") "
Sep 04 22:20:08.294581 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.294386    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ll67d\" (UniqueName: \"kubernetes.io/projected/1d3f722b-6821-4b1e-9404-9a07af50a378-kube-api-access-ll67d\") pod \"1d3f722b-6821-4b1e-9404-9a07af50a378\" (UID: \"1d3f722b-6821-4b1e-9404-9a07af50a378\") "
Sep 04 22:20:08.294581 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.294490    2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tkz8q\" (UniqueName: \"kubernetes.io/projected/4c1b06e8-98cb-4c9b-badf-6dc8e846e184-kube-api-access-tkz8q\") pod \"4c1b06e8-98cb-4c9b-badf-6dc8e846e184\" (UID: \"4c1b06e8-98cb-4c9b-badf-6dc8e846e184\") "
Sep 04 22:20:08.294786 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.294750    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c1b06e8-98cb-4c9b-badf-6dc8e846e184-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4c1b06e8-98cb-4c9b-badf-6dc8e846e184" (UID: "4c1b06e8-98cb-4c9b-badf-6dc8e846e184"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:20:08.294848 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.294796    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d3f722b-6821-4b1e-9404-9a07af50a378-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1d3f722b-6821-4b1e-9404-9a07af50a378" (UID: "1d3f722b-6821-4b1e-9404-9a07af50a378"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 04 22:20:08.296655 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.296627    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d3f722b-6821-4b1e-9404-9a07af50a378-kube-api-access-ll67d" (OuterVolumeSpecName: "kube-api-access-ll67d") pod "1d3f722b-6821-4b1e-9404-9a07af50a378" (UID: "1d3f722b-6821-4b1e-9404-9a07af50a378"). InnerVolumeSpecName "kube-api-access-ll67d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:20:08.296735 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.296659    2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c1b06e8-98cb-4c9b-badf-6dc8e846e184-kube-api-access-tkz8q" (OuterVolumeSpecName: "kube-api-access-tkz8q") pod "4c1b06e8-98cb-4c9b-badf-6dc8e846e184" (UID: "4c1b06e8-98cb-4c9b-badf-6dc8e846e184"). InnerVolumeSpecName "kube-api-access-tkz8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 04 22:20:08.353212 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.353173    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7db8920-fded-486e-90ff-d1f1bc6078e8" path="/var/lib/kubelet/pods/b7db8920-fded-486e-90ff-d1f1bc6078e8/volumes"
Sep 04 22:20:08.395719 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.395676    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkz8q\" (UniqueName: \"kubernetes.io/projected/4c1b06e8-98cb-4c9b-badf-6dc8e846e184-kube-api-access-tkz8q\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:20:08.395719 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.395715    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4c1b06e8-98cb-4c9b-badf-6dc8e846e184-plugins-config-volume\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:20:08.395719 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.395725    2557 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1d3f722b-6821-4b1e-9404-9a07af50a378-plugins-config-volume\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:20:08.396004 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.395735    2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ll67d\" (UniqueName: \"kubernetes.io/projected/1d3f722b-6821-4b1e-9404-9a07af50a378-kube-api-access-ll67d\") on node \"ip-10-0-134-141.ec2.internal\" DevicePath \"\""
Sep 04 22:20:08.952604 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.952564    2557 generic.go:356] "Generic (PLEG): container finished" podID="1d3f722b-6821-4b1e-9404-9a07af50a378" containerID="25c9501fb4b90790235344bac6be3951dece0e2514936e1a655a5289a6cba6c8" exitCode=0
Sep 04 22:20:08.953062 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.952613    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx"
Sep 04 22:20:08.953062 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.952612    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx" event={"ID":"1d3f722b-6821-4b1e-9404-9a07af50a378","Type":"ContainerDied","Data":"25c9501fb4b90790235344bac6be3951dece0e2514936e1a655a5289a6cba6c8"}
Sep 04 22:20:08.953062 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.952682    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx" event={"ID":"1d3f722b-6821-4b1e-9404-9a07af50a378","Type":"ContainerDied","Data":"bceb1e0239f2f7aa6ef22cb1a8347f8485d9e99282ce0b2cde76724c1f4f48f4"}
Sep 04 22:20:08.953062 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.952714    2557 scope.go:117] "RemoveContainer" containerID="25c9501fb4b90790235344bac6be3951dece0e2514936e1a655a5289a6cba6c8"
Sep 04 22:20:08.954160 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.954117    2557 generic.go:356] "Generic (PLEG): container finished" podID="4c1b06e8-98cb-4c9b-badf-6dc8e846e184" containerID="51f7e815e79adfc17065ae77f436254a2f971e37637524db7b7aae425b8532f6" exitCode=0
Sep 04 22:20:08.954160 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.954153    2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4"
Sep 04 22:20:08.954260 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.954160    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4" event={"ID":"4c1b06e8-98cb-4c9b-badf-6dc8e846e184","Type":"ContainerDied","Data":"51f7e815e79adfc17065ae77f436254a2f971e37637524db7b7aae425b8532f6"}
Sep 04 22:20:08.954260 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.954203    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4" event={"ID":"4c1b06e8-98cb-4c9b-badf-6dc8e846e184","Type":"ContainerDied","Data":"2fd85c6f61c8ef1f97f2112378b175497637c45c845b9c33e8499817e848d775"}
Sep 04 22:20:08.962771 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.962747    2557 scope.go:117] "RemoveContainer" containerID="25c9501fb4b90790235344bac6be3951dece0e2514936e1a655a5289a6cba6c8"
Sep 04 22:20:08.963111 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:20:08.963089    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25c9501fb4b90790235344bac6be3951dece0e2514936e1a655a5289a6cba6c8\": container with ID starting with 25c9501fb4b90790235344bac6be3951dece0e2514936e1a655a5289a6cba6c8 not found: ID does not exist" containerID="25c9501fb4b90790235344bac6be3951dece0e2514936e1a655a5289a6cba6c8"
Sep 04 22:20:08.963160 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.963120    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25c9501fb4b90790235344bac6be3951dece0e2514936e1a655a5289a6cba6c8"} err="failed to get container status \"25c9501fb4b90790235344bac6be3951dece0e2514936e1a655a5289a6cba6c8\": rpc error: code = NotFound desc = could not find container \"25c9501fb4b90790235344bac6be3951dece0e2514936e1a655a5289a6cba6c8\": container with ID starting with 25c9501fb4b90790235344bac6be3951dece0e2514936e1a655a5289a6cba6c8 not found: ID does not exist"
Sep 04 22:20:08.963160 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.963137    2557 scope.go:117] "RemoveContainer" containerID="51f7e815e79adfc17065ae77f436254a2f971e37637524db7b7aae425b8532f6"
Sep 04 22:20:08.971586 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.971560    2557 scope.go:117] "RemoveContainer" containerID="51f7e815e79adfc17065ae77f436254a2f971e37637524db7b7aae425b8532f6"
Sep 04 22:20:08.971957 ip-10-0-134-141 kubenswrapper[2557]: E0904 22:20:08.971931    2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51f7e815e79adfc17065ae77f436254a2f971e37637524db7b7aae425b8532f6\": container with ID starting with 51f7e815e79adfc17065ae77f436254a2f971e37637524db7b7aae425b8532f6 not found: ID does not exist" containerID="51f7e815e79adfc17065ae77f436254a2f971e37637524db7b7aae425b8532f6"
Sep 04 22:20:08.972033 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.971975    2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51f7e815e79adfc17065ae77f436254a2f971e37637524db7b7aae425b8532f6"} err="failed to get container status \"51f7e815e79adfc17065ae77f436254a2f971e37637524db7b7aae425b8532f6\": rpc error: code = NotFound desc = could not find container \"51f7e815e79adfc17065ae77f436254a2f971e37637524db7b7aae425b8532f6\": container with ID starting with 51f7e815e79adfc17065ae77f436254a2f971e37637524db7b7aae425b8532f6 not found: ID does not exist"
Sep 04 22:20:08.975384 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.975349    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4"]
Sep 04 22:20:08.981269 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.981243    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-7a26c9-bbff86557-s68g4"]
Sep 04 22:20:08.990805 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.990775    2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx"]
Sep 04 22:20:08.994528 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:08.994501    2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7a26c9-7788887478-vgzxx"]
Sep 04 22:20:10.042176 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:10.042133    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:20:10.353540 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:10.353452    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d3f722b-6821-4b1e-9404-9a07af50a378" path="/var/lib/kubelet/pods/1d3f722b-6821-4b1e-9404-9a07af50a378/volumes"
Sep 04 22:20:10.353875 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:10.353861    2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c1b06e8-98cb-4c9b-badf-6dc8e846e184" path="/var/lib/kubelet/pods/4c1b06e8-98cb-4c9b-badf-6dc8e846e184/volumes"
Sep 04 22:20:18.050371 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:20:18.050327    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:21:40.444479 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:21:40.444437    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:21:46.610726 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:21:46.610690    2557 scope.go:117] "RemoveContainer" containerID="8088bbd78e97ea5e1e51350fef3be9ea32d76c5a1158bed5cc05458f646bd360"
Sep 04 22:22:03.951384 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:22:03.951344    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:22:10.542756 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:22:10.542662    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:22:15.549039 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:22:15.548986    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:22:20.345791 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:22:20.345748    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:22:27.847803 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:22:27.847759    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:22:37.753360 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:22:37.753316    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:22:42.940883 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:22:42.940839    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:22:46.632024 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:22:46.631989    2557 scope.go:117] "RemoveContainer" containerID="71441e6b929ecc27224a27bc5fab79c2e30a672ae30de4024a90ac2fd30ee77b"
Sep 04 22:22:46.640620 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:22:46.640599    2557 scope.go:117] "RemoveContainer" containerID="3e3759a95b6d596cf18737c4373f5adddd7591208188cb8db2377bbcbad3efcc"
Sep 04 22:23:46.664808 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:23:46.664769    2557 scope.go:117] "RemoveContainer" containerID="fc9aee5f10cb8ce91d34aebde2128113c28acff7a58b7ab54c92c6b250b1eca0"
Sep 04 22:23:46.674499 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:23:46.674475    2557 scope.go:117] "RemoveContainer" containerID="4e07aa2d99f4da5611d416a3435d9a918267755c653a3ccf72932154e822a01d"
Sep 04 22:24:07.846070 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:24:07.846029    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:24:12.345291 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:24:12.345248    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:24:18.844389 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:24:18.844346    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:24:24.640390 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:24:24.640353    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:24:34.143758 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:24:34.143712    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:24:46.703529 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:24:46.703492    2557 scope.go:117] "RemoveContainer" containerID="f2da45a21b937b36726f19bfb7dff8e6972de0468f582867bd20efec9a78360b"
Sep 04 22:24:46.712197 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:24:46.712172    2557 scope.go:117] "RemoveContainer" containerID="aabb20b64110f64593119c8db5a3e0e4a1476a2bcd583aa8dc890637ca60bb80"
Sep 04 22:24:49.646927 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:24:49.646878    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:25:22.136833 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:25:22.136789    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:25:28.939305 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:25:28.939251    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:25:34.743179 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:25:34.743141    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:25:39.849545 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:25:39.849502    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-57drl"]
Sep 04 22:25:51.919178 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:25:51.919146    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-764b98687b-dtxjd_9c8ffe98-d551-44a2-ab6a-53ee22459e1d/copy-manifests/0.log"
Sep 04 22:25:51.925950 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:25:51.925923    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-764b98687b-dtxjd_9c8ffe98-d551-44a2-ab6a-53ee22459e1d/manager/0.log"
Sep 04 22:25:52.747825 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:25:52.747793    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29809335-cxzkw_a91b6bd3-6244-4b0c-ba43-ec22e4aaf1b5/cleanup/0.log"
Sep 04 22:25:54.210608 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:25:54.210570    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-57drl_655b62f3-2bbc-4aeb-a1fc-90274bebb94e/limitador/0.log"
Sep 04 22:25:54.708113 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:25:54.708086    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr_a1f7a723-2ac2-494d-99c1-8343f76130bb/istio-proxy/0.log"
Sep 04 22:25:54.825554 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:25:54.825518    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rmlv4_d94c6c55-a84c-4c9c-8f75-f772730fa0dd/discovery/0.log"
Sep 04 22:25:55.187896 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:25:55.187860    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-8tbt9_921df383-5403-4688-a3f6-ffe059353a5a/istio-proxy/1.log"
Sep 04 22:25:56.134241 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:25:56.134210    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-c77vw_d65975c5-c461-4d5e-b216-1f3efd2bc7c5/main/0.log"
Sep 04 22:25:56.361950 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:25:56.361905    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-4q4t7_a89fc6fc-d17b-4a96-97fd-a6d75fe6090d/main/0.log"
Sep 04 22:25:56.474180 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:25:56.474091    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-jgkkf_c3b9225b-32cb-4fc2-bbb5-bc84e0f68d07/main/0.log"
Sep 04 22:26:04.792282 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:04.792250    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-p7b77_8fcc2326-c662-4e67-bdaf-60a54cce4235/global-pull-secret-syncer/0.log"
Sep 04 22:26:04.905347 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:04.905306    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xcgjm_db97b042-b391-4f31-8f0f-3deceacb5157/konnectivity-agent/0.log"
Sep 04 22:26:04.964351 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:04.964296    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-141.ec2.internal_79b44ba13a4b8fc8cb712cd9da0778c3/haproxy/0.log"
Sep 04 22:26:09.811282 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:09.811249    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-57drl_655b62f3-2bbc-4aeb-a1fc-90274bebb94e/limitador/0.log"
Sep 04 22:26:11.413958 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:11.413926    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f720a6df-15a0-41b0-8367-58de99e72753/alertmanager/0.log"
Sep 04 22:26:11.440608 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:11.440583    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f720a6df-15a0-41b0-8367-58de99e72753/config-reloader/0.log"
Sep 04 22:26:11.460508 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:11.460480    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f720a6df-15a0-41b0-8367-58de99e72753/kube-rbac-proxy-web/0.log"
Sep 04 22:26:11.484240 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:11.484213    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f720a6df-15a0-41b0-8367-58de99e72753/kube-rbac-proxy/0.log"
Sep 04 22:26:11.508763 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:11.508734    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f720a6df-15a0-41b0-8367-58de99e72753/kube-rbac-proxy-metric/0.log"
Sep 04 22:26:11.539847 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:11.539812    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f720a6df-15a0-41b0-8367-58de99e72753/prom-label-proxy/0.log"
Sep 04 22:26:11.565571 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:11.565542    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_f720a6df-15a0-41b0-8367-58de99e72753/init-config-reloader/0.log"
Sep 04 22:26:11.646598 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:11.646563    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-qwtkd_dd9483eb-f008-4171-9233-2724cb1ee215/kube-state-metrics/0.log"
Sep 04 22:26:11.665107 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:11.665032    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-qwtkd_dd9483eb-f008-4171-9233-2724cb1ee215/kube-rbac-proxy-main/0.log"
Sep 04 22:26:11.686880 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:11.686852    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-qwtkd_dd9483eb-f008-4171-9233-2724cb1ee215/kube-rbac-proxy-self/0.log"
Sep 04 22:26:11.854513 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:11.854484    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m25wx_6420905f-c55a-4e5a-aa93-b36775b74d93/node-exporter/0.log"
Sep 04 22:26:11.887993 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:11.887961    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m25wx_6420905f-c55a-4e5a-aa93-b36775b74d93/kube-rbac-proxy/0.log"
Sep 04 22:26:11.909005 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:11.908978    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m25wx_6420905f-c55a-4e5a-aa93-b36775b74d93/init-textfile/0.log"
Sep 04 22:26:12.005045 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:12.005014    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-fsbsn_1f435c58-0d37-40d1-8bf3-4cad62f0944d/kube-rbac-proxy-main/0.log"
Sep 04 22:26:12.023495 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:12.023470    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-fsbsn_1f435c58-0d37-40d1-8bf3-4cad62f0944d/kube-rbac-proxy-self/0.log"
Sep 04 22:26:12.048091 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:12.048063    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-657fd6c54-fsbsn_1f435c58-0d37-40d1-8bf3-4cad62f0944d/openshift-state-metrics/0.log"
Sep 04 22:26:14.995847 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.995804    2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx"]
Sep 04 22:26:14.996240 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996138    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4c0465a-eeb5-435f-be67-ac5f9e5c54b0" containerName="main"
Sep 04 22:26:14.996240 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996150    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4c0465a-eeb5-435f-be67-ac5f9e5c54b0" containerName="main"
Sep 04 22:26:14.996240 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996159    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc47bef7-4146-44ed-b36b-35ffb3142b68" containerName="revoke-keys"
Sep 04 22:26:14.996240 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996165    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc47bef7-4146-44ed-b36b-35ffb3142b68" containerName="revoke-keys"
Sep 04 22:26:14.996240 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996173    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d3f722b-6821-4b1e-9404-9a07af50a378" containerName="payload-pre-processing"
Sep 04 22:26:14.996240 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996179    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d3f722b-6821-4b1e-9404-9a07af50a378" containerName="payload-pre-processing"
Sep 04 22:26:14.996240 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996187    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2efcc572-0661-4dd9-98b1-b5816bd4b037" containerName="istio-proxy"
Sep 04 22:26:14.996240 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996193    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2efcc572-0661-4dd9-98b1-b5816bd4b037" containerName="istio-proxy"
Sep 04 22:26:14.996240 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996199    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c93d0e5c-af64-4138-9542-13b6c1042254" containerName="istio-proxy"
Sep 04 22:26:14.996240 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996205    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c93d0e5c-af64-4138-9542-13b6c1042254" containerName="istio-proxy"
Sep 04 22:26:14.996240 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996213    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c1b06e8-98cb-4c9b-badf-6dc8e846e184" containerName="payload-processing"
Sep 04 22:26:14.996240 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996218    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c1b06e8-98cb-4c9b-badf-6dc8e846e184" containerName="payload-processing"
Sep 04 22:26:14.996240 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996223    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2efcc572-0661-4dd9-98b1-b5816bd4b037" containerName="istio-proxy"
Sep 04 22:26:14.996240 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996229    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2efcc572-0661-4dd9-98b1-b5816bd4b037" containerName="istio-proxy"
Sep 04 22:26:14.996240 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996236    2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c93d0e5c-af64-4138-9542-13b6c1042254" containerName="istio-proxy"
Sep 04 22:26:14.996240 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996241    2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="c93d0e5c-af64-4138-9542-13b6c1042254" containerName="istio-proxy"
Sep 04 22:26:14.996765 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996301    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc47bef7-4146-44ed-b36b-35ffb3142b68" containerName="revoke-keys"
Sep 04 22:26:14.996765 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996308    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c1b06e8-98cb-4c9b-badf-6dc8e846e184" containerName="payload-processing"
Sep 04 22:26:14.996765 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996315    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d3f722b-6821-4b1e-9404-9a07af50a378" containerName="payload-pre-processing"
Sep 04 22:26:14.996765 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996322    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c93d0e5c-af64-4138-9542-13b6c1042254" containerName="istio-proxy"
Sep 04 22:26:14.996765 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996327    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2efcc572-0661-4dd9-98b1-b5816bd4b037" containerName="istio-proxy"
Sep 04 22:26:14.996765 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996331    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4c0465a-eeb5-435f-be67-ac5f9e5c54b0" containerName="main"
Sep 04 22:26:14.996765 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996336    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2efcc572-0661-4dd9-98b1-b5816bd4b037" containerName="istio-proxy"
Sep 04 22:26:14.996765 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.996341    2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="c93d0e5c-af64-4138-9542-13b6c1042254" containerName="istio-proxy"
Sep 04 22:26:14.999684 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:14.999662    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx"
Sep 04 22:26:15.002827 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.002790    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gqs8f\"/\"openshift-service-ca.crt\""
Sep 04 22:26:15.003641 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.003614    2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gqs8f\"/\"default-dockercfg-725pc\""
Sep 04 22:26:15.003764 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.003595    2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gqs8f\"/\"kube-root-ca.crt\""
Sep 04 22:26:15.009456 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.009425    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx"]
Sep 04 22:26:15.142307 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.142263    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a5c8190d-b659-4f5e-bf13-69fb195539d3-proc\") pod \"perf-node-gather-daemonset-jvzvx\" (UID: \"a5c8190d-b659-4f5e-bf13-69fb195539d3\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx"
Sep 04 22:26:15.142307 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.142314    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dl6m4\" (UniqueName: \"kubernetes.io/projected/a5c8190d-b659-4f5e-bf13-69fb195539d3-kube-api-access-dl6m4\") pod \"perf-node-gather-daemonset-jvzvx\" (UID: \"a5c8190d-b659-4f5e-bf13-69fb195539d3\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx"
Sep 04 22:26:15.142571 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.142339    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a5c8190d-b659-4f5e-bf13-69fb195539d3-sys\") pod \"perf-node-gather-daemonset-jvzvx\" (UID: \"a5c8190d-b659-4f5e-bf13-69fb195539d3\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx"
Sep 04 22:26:15.142571 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.142385    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a5c8190d-b659-4f5e-bf13-69fb195539d3-podres\") pod \"perf-node-gather-daemonset-jvzvx\" (UID: \"a5c8190d-b659-4f5e-bf13-69fb195539d3\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx"
Sep 04 22:26:15.142571 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.142513    2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a5c8190d-b659-4f5e-bf13-69fb195539d3-lib-modules\") pod \"perf-node-gather-daemonset-jvzvx\" (UID: \"a5c8190d-b659-4f5e-bf13-69fb195539d3\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx"
Sep 04 22:26:15.244019 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.243976    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a5c8190d-b659-4f5e-bf13-69fb195539d3-proc\") pod \"perf-node-gather-daemonset-jvzvx\" (UID: \"a5c8190d-b659-4f5e-bf13-69fb195539d3\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx"
Sep 04 22:26:15.244236 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.244039    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dl6m4\" (UniqueName: \"kubernetes.io/projected/a5c8190d-b659-4f5e-bf13-69fb195539d3-kube-api-access-dl6m4\") pod \"perf-node-gather-daemonset-jvzvx\" (UID: \"a5c8190d-b659-4f5e-bf13-69fb195539d3\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx"
Sep 04 22:26:15.244236 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.244074    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a5c8190d-b659-4f5e-bf13-69fb195539d3-sys\") pod \"perf-node-gather-daemonset-jvzvx\" (UID: \"a5c8190d-b659-4f5e-bf13-69fb195539d3\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx"
Sep 04 22:26:15.244236 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.244099    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a5c8190d-b659-4f5e-bf13-69fb195539d3-podres\") pod \"perf-node-gather-daemonset-jvzvx\" (UID: \"a5c8190d-b659-4f5e-bf13-69fb195539d3\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx"
Sep 04 22:26:15.244236 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.244105    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a5c8190d-b659-4f5e-bf13-69fb195539d3-proc\") pod \"perf-node-gather-daemonset-jvzvx\" (UID: \"a5c8190d-b659-4f5e-bf13-69fb195539d3\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx"
Sep 04 22:26:15.244236 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.244174    2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a5c8190d-b659-4f5e-bf13-69fb195539d3-lib-modules\") pod \"perf-node-gather-daemonset-jvzvx\" (UID: \"a5c8190d-b659-4f5e-bf13-69fb195539d3\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx"
Sep 04 22:26:15.244236 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.244190    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a5c8190d-b659-4f5e-bf13-69fb195539d3-sys\") pod \"perf-node-gather-daemonset-jvzvx\" (UID: \"a5c8190d-b659-4f5e-bf13-69fb195539d3\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx"
Sep 04 22:26:15.244580 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.244239    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a5c8190d-b659-4f5e-bf13-69fb195539d3-podres\") pod \"perf-node-gather-daemonset-jvzvx\" (UID: \"a5c8190d-b659-4f5e-bf13-69fb195539d3\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx"
Sep 04 22:26:15.244580 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.244288    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a5c8190d-b659-4f5e-bf13-69fb195539d3-lib-modules\") pod \"perf-node-gather-daemonset-jvzvx\" (UID: \"a5c8190d-b659-4f5e-bf13-69fb195539d3\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx"
Sep 04 22:26:15.251856 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.251831    2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dl6m4\" (UniqueName: \"kubernetes.io/projected/a5c8190d-b659-4f5e-bf13-69fb195539d3-kube-api-access-dl6m4\") pod \"perf-node-gather-daemonset-jvzvx\" (UID: \"a5c8190d-b659-4f5e-bf13-69fb195539d3\") " pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx"
Sep 04 22:26:15.311983 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.311939    2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx"
Sep 04 22:26:15.589077 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.589044    2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx"]
Sep 04 22:26:15.590788 ip-10-0-134-141 kubenswrapper[2557]: W0904 22:26:15.590758    2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda5c8190d_b659_4f5e_bf13_69fb195539d3.slice/crio-57f1627c6bd25662ce475b33576207fa7449531e7247d484aee6eb4238194f29 WatchSource:0}: Error finding container 57f1627c6bd25662ce475b33576207fa7449531e7247d484aee6eb4238194f29: Status 404 returned error can't find the container with id 57f1627c6bd25662ce475b33576207fa7449531e7247d484aee6eb4238194f29
Sep 04 22:26:15.956350 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.956254    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx" event={"ID":"a5c8190d-b659-4f5e-bf13-69fb195539d3","Type":"ContainerStarted","Data":"57ed56d84e114c16b2c4d5bd90efc89faeee597b3b69540ab312c7e613219a5b"}
Sep 04 22:26:15.956350 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.956309    2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx" event={"ID":"a5c8190d-b659-4f5e-bf13-69fb195539d3","Type":"ContainerStarted","Data":"57f1627c6bd25662ce475b33576207fa7449531e7247d484aee6eb4238194f29"}
Sep 04 22:26:15.956350 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.956330    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx"
Sep 04 22:26:15.973059 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:15.972996    2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx" podStartSLOduration=1.972980739 podStartE2EDuration="1.972980739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-04 22:26:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-04 22:26:15.97083116 +0000 UTC m=+1950.287397898" watchObservedRunningTime="2026-09-04 22:26:15.972980739 +0000 UTC m=+1950.289547504"
Sep 04 22:26:16.805190 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:16.805154    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bc7b4_3c09d3a4-17fe-4fda-b7e7-723ad50bbe14/dns/0.log"
Sep 04 22:26:16.823340 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:16.823308    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bc7b4_3c09d3a4-17fe-4fda-b7e7-723ad50bbe14/kube-rbac-proxy/0.log"
Sep 04 22:26:16.930755 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:16.930726    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-tr8wn_b36d1fd6-09dc-43b7-b179-051cd03a518f/dns-node-resolver/0.log"
Sep 04 22:26:17.448396 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:17.448365    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-86f655765c-vdjrp_cfb97231-da9f-4aec-b5c5-e77b3a0c0c8d/registry/0.log"
Sep 04 22:26:17.468801 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:17.468772    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-c6lfc_29eb21bd-c9b9-4206-8029-86c0d353d68d/node-ca/0.log"
Sep 04 22:26:18.415477 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:18.415434    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr_a1f7a723-2ac2-494d-99c1-8343f76130bb/istio-proxy/0.log"
Sep 04 22:26:18.607685 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:18.607650    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rmlv4_d94c6c55-a84c-4c9c-8f75-f772730fa0dd/discovery/0.log"
Sep 04 22:26:18.673129 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:18.673047    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-8tbt9_921df383-5403-4688-a3f6-ffe059353a5a/istio-proxy/0.log"
Sep 04 22:26:18.680209 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:18.680169    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-8tbt9_921df383-5403-4688-a3f6-ffe059353a5a/istio-proxy/1.log"
Sep 04 22:26:19.588384 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:19.588324    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-66fwd_59f90e70-25a7-4c3d-9536-03a9bb65e69b/serve-healthcheck-canary/0.log"
Sep 04 22:26:20.166131 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:20.166063    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jsnbb_07654df4-9b51-4323-ad80-879b19f73dfa/kube-rbac-proxy/0.log"
Sep 04 22:26:20.184255 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:20.184225    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jsnbb_07654df4-9b51-4323-ad80-879b19f73dfa/exporter/0.log"
Sep 04 22:26:20.203021 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:20.202991    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jsnbb_07654df4-9b51-4323-ad80-879b19f73dfa/extractor/0.log"
Sep 04 22:26:21.972593 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:21.972561    2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gqs8f/perf-node-gather-daemonset-jvzvx"
Sep 04 22:26:22.818756 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:22.818719    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-764b98687b-dtxjd_9c8ffe98-d551-44a2-ab6a-53ee22459e1d/manager/0.log"
Sep 04 22:26:22.837638 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:22.837609    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-764b98687b-dtxjd_9c8ffe98-d551-44a2-ab6a-53ee22459e1d/copy-manifests/0.log"
Sep 04 22:26:24.312188 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:24.312160    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-587bddd5d4-qppmg_eafa9ce5-8bfc-49e9-8579-dabf50b16b1a/manager/0.log"
Sep 04 22:26:25.544303 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:25.544268    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr_a1f7a723-2ac2-494d-99c1-8343f76130bb/istio-proxy/0.log"
Sep 04 22:26:25.734169 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:25.734075    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rmlv4_d94c6c55-a84c-4c9c-8f75-f772730fa0dd/discovery/0.log"
Sep 04 22:26:25.783409 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:25.783372    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-8tbt9_921df383-5403-4688-a3f6-ffe059353a5a/istio-proxy/0.log"
Sep 04 22:26:25.790946 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:25.790916    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-8tbt9_921df383-5403-4688-a3f6-ffe059353a5a/istio-proxy/1.log"
Sep 04 22:26:27.369013 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:27.368978    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-vwklr_a1f7a723-2ac2-494d-99c1-8343f76130bb/istio-proxy/0.log"
Sep 04 22:26:27.564042 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:27.564003    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-rmlv4_d94c6c55-a84c-4c9c-8f75-f772730fa0dd/discovery/0.log"
Sep 04 22:26:27.616871 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:27.616843    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-8tbt9_921df383-5403-4688-a3f6-ffe059353a5a/istio-proxy/0.log"
Sep 04 22:26:27.623750 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:27.623665    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-8tbt9_921df383-5403-4688-a3f6-ffe059353a5a/istio-proxy/1.log"
Sep 04 22:26:30.666351 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:30.666317    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-9nl4m_fb5aaba9-9cb0-4fa7-a342-344f1e40cad8/kube-multus/0.log"
Sep 04 22:26:30.811466 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:30.811402    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzjmv_bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90/kube-multus-additional-cni-plugins/0.log"
Sep 04 22:26:30.832193 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:30.832165    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzjmv_bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90/egress-router-binary-copy/0.log"
Sep 04 22:26:30.853675 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:30.853650    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzjmv_bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90/cni-plugins/0.log"
Sep 04 22:26:30.873268 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:30.873237    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzjmv_bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90/bond-cni-plugin/0.log"
Sep 04 22:26:30.895196 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:30.895170    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzjmv_bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90/routeoverride-cni/0.log"
Sep 04 22:26:30.923355 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:30.923276    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzjmv_bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90/whereabouts-cni-bincopy/0.log"
Sep 04 22:26:30.947191 ip-10-0-134-141 kubenswrapper[2557]: I0904 22:26:30.947159    2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzjmv_bcdb4989-2fe2-44ce-b20a-8d3e1dc05b90/whereabouts-cni/0.log"