Jul 28 22:09:51.881839 ip-10-0-141-102 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 22:09:52.206590 ip-10-0-141-102 kubenswrapper[2551]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 22:09:52.206590 ip-10-0-141-102 kubenswrapper[2551]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 22:09:52.206590 ip-10-0-141-102 kubenswrapper[2551]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 22:09:52.206590 ip-10-0-141-102 kubenswrapper[2551]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 22:09:52.206590 ip-10-0-141-102 kubenswrapper[2551]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 22:09:52.208078 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.207992    2551 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 22:09:52.212243 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212227    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 22:09:52.212243 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212242    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 22:09:52.212307 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212247    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 22:09:52.212307 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212250    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 22:09:52.212307 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212254    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 22:09:52.212307 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212257    2551 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 22:09:52.212307 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212259    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 22:09:52.212307 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212262    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 22:09:52.212307 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212266    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 22:09:52.212307 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212269    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 22:09:52.212307 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212272    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 22:09:52.212307 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212274    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 22:09:52.212307 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212277    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 22:09:52.212307 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212280    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 22:09:52.212307 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212282    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 22:09:52.212307 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212285    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 22:09:52.212307 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212287    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 22:09:52.212307 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212290    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 22:09:52.212307 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212293    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 22:09:52.212307 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212295    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 22:09:52.212307 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212298    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 22:09:52.212871 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212301    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 22:09:52.212871 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212303    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 22:09:52.212871 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212306    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 22:09:52.212871 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212310    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 22:09:52.212871 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212315    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 22:09:52.212871 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212318    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 22:09:52.212871 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212320    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 22:09:52.212871 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212323    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 22:09:52.212871 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212326    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 22:09:52.212871 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212328    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 22:09:52.212871 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212331    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 22:09:52.212871 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212333    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 22:09:52.212871 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212336    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 22:09:52.212871 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212338    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 22:09:52.212871 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212341    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 22:09:52.212871 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212344    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 22:09:52.212871 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212348    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 22:09:52.212871 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212352    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 22:09:52.212871 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212355    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 22:09:52.213583 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212358    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 22:09:52.213583 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212362    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 22:09:52.213583 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212365    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 22:09:52.213583 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212368    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 22:09:52.213583 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212371    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 22:09:52.213583 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212374    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 22:09:52.213583 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212377    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 22:09:52.213583 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212379    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 22:09:52.213583 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212382    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 22:09:52.213583 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212385    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 22:09:52.213583 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212387    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 22:09:52.213583 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212390    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 22:09:52.213583 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212392    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 22:09:52.213583 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212394    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 22:09:52.213583 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212399    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 22:09:52.213583 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212402    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 22:09:52.213583 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212404    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 22:09:52.213583 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212407    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 22:09:52.213583 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212410    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 22:09:52.214070 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212413    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 22:09:52.214070 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212416    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 22:09:52.214070 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212419    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 22:09:52.214070 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212421    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 22:09:52.214070 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212424    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 22:09:52.214070 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212426    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 22:09:52.214070 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212429    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 22:09:52.214070 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212431    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 22:09:52.214070 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212434    2551 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 22:09:52.214070 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212436    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 22:09:52.214070 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212440    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 22:09:52.214070 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212442    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 22:09:52.214070 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212445    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 22:09:52.214070 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212447    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 22:09:52.214070 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212450    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 22:09:52.214070 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212452    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 22:09:52.214070 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212454    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 22:09:52.214070 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212457    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 22:09:52.214070 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212459    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 22:09:52.214070 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212462    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 22:09:52.214600 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212465    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 22:09:52.214600 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212467    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 22:09:52.214600 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212470    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 22:09:52.214600 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212680    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 22:09:52.214600 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212690    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 22:09:52.214600 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212696    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 22:09:52.214600 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212701    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 22:09:52.214600 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212708    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 22:09:52.214600 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212713    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 22:09:52.214600 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212717    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 22:09:52.214600 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212722    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 22:09:52.214600 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212727    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 22:09:52.214600 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212732    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 22:09:52.214600 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212737    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 22:09:52.214600 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212752    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 22:09:52.214600 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212762    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 22:09:52.214600 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212766    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 22:09:52.214600 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212770    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 22:09:52.214600 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212774    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 22:09:52.214600 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.212779    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 22:09:52.215072 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214157    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 22:09:52.215072 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214166    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 22:09:52.215072 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214170    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 22:09:52.215072 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214173    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 22:09:52.215072 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214176    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 22:09:52.215072 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214179    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 22:09:52.215072 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214182    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 22:09:52.215072 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214187    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 22:09:52.215072 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214191    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 22:09:52.215072 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214194    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 22:09:52.215072 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214197    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 22:09:52.215072 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214200    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 22:09:52.215072 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214203    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 22:09:52.215072 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214206    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 22:09:52.215072 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214209    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 22:09:52.215072 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214211    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 22:09:52.215072 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214214    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 22:09:52.215072 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214217    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 22:09:52.215072 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214219    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 22:09:52.215559 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214222    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 22:09:52.215559 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214224    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 22:09:52.215559 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214227    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 22:09:52.215559 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214229    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 22:09:52.215559 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214232    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 22:09:52.215559 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214235    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 22:09:52.215559 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214237    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 22:09:52.215559 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214239    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 22:09:52.215559 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214243    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 22:09:52.215559 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214245    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 22:09:52.215559 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214248    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 22:09:52.215559 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214251    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 22:09:52.215559 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214254    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 22:09:52.215559 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214257    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 22:09:52.215559 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214261    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 22:09:52.215559 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214265    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 22:09:52.215559 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214268    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 22:09:52.215559 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214271    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 22:09:52.215559 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214274    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 22:09:52.216029 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214276    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 22:09:52.216029 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214280    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 22:09:52.216029 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214282    2551 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 22:09:52.216029 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214285    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 22:09:52.216029 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214288    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 22:09:52.216029 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214290    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 22:09:52.216029 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214293    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 22:09:52.216029 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214295    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 22:09:52.216029 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214298    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 22:09:52.216029 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214301    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 22:09:52.216029 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214303    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 22:09:52.216029 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214306    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 22:09:52.216029 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214309    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 22:09:52.216029 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214312    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 22:09:52.216029 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214314    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 22:09:52.216029 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214317    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 22:09:52.216029 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214319    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 22:09:52.216029 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214322    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 22:09:52.216029 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214324    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 22:09:52.216029 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214328    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 22:09:52.216513 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214331    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 22:09:52.216513 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214333    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 22:09:52.216513 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214336    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 22:09:52.216513 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214338    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 22:09:52.216513 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214341    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 22:09:52.216513 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214344    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 22:09:52.216513 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214347    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 22:09:52.216513 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214349    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 22:09:52.216513 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214352    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 22:09:52.216513 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214355    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 22:09:52.216513 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214357    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 22:09:52.216513 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214360    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 22:09:52.216513 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214363    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 22:09:52.216513 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214365    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 22:09:52.216513 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214368    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 22:09:52.216513 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214370    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 22:09:52.216513 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214373    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 22:09:52.216513 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214375    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 22:09:52.216513 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214378    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 22:09:52.216513 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214381    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 22:09:52.217034 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214383    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 22:09:52.217034 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214385    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 22:09:52.217034 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214388    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 22:09:52.217034 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214392    2551 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 22:09:52.217034 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214395    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 22:09:52.217034 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214397    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 22:09:52.217034 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214400    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 22:09:52.217034 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214402    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 22:09:52.217034 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214404    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 22:09:52.217034 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214408    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 22:09:52.217034 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214412    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 22:09:52.217034 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214415    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 22:09:52.217034 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214418    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 22:09:52.217034 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214420    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 22:09:52.217034 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214423    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 22:09:52.217034 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214426    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 22:09:52.217034 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214428    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 22:09:52.217034 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214431    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 22:09:52.217034 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214434    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 22:09:52.217503 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214437    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 22:09:52.217503 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214439    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 22:09:52.217503 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214821    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 22:09:52.217503 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214827    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 22:09:52.217503 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214830    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 22:09:52.217503 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214832    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 22:09:52.217503 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214835    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 22:09:52.217503 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214838    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 22:09:52.217503 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214841    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 22:09:52.217503 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214843    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 22:09:52.217503 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214846    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 22:09:52.217503 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214848    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 22:09:52.217503 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214851    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 22:09:52.217503 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214854    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 22:09:52.217503 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214857    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 22:09:52.217503 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214859    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 22:09:52.217503 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214862    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 22:09:52.217503 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214864    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 22:09:52.217503 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214867    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 22:09:52.217503 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214870    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 22:09:52.218004 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214873    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 22:09:52.218004 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214875    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 22:09:52.218004 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214878    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 22:09:52.218004 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214880    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 22:09:52.218004 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214883    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 22:09:52.218004 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214886    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 22:09:52.218004 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214888    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 22:09:52.218004 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214891    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 22:09:52.218004 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214893    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 22:09:52.218004 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214896    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 22:09:52.218004 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214899    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 22:09:52.218004 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214904    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 22:09:52.218004 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214907    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 22:09:52.218004 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214910    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 22:09:52.218004 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214914    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 22:09:52.218004 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214916    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 22:09:52.218004 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214919    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 22:09:52.218004 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214922    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 22:09:52.218004 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214924    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 22:09:52.218004 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214927    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 22:09:52.218489 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214930    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 22:09:52.218489 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214932    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 22:09:52.218489 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214935    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 22:09:52.218489 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214938    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 22:09:52.218489 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214940    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 22:09:52.218489 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214943    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 22:09:52.218489 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214945    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 22:09:52.218489 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214948    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 22:09:52.218489 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214951    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 22:09:52.218489 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214953    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 22:09:52.218489 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214955    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 22:09:52.218489 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214958    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 22:09:52.218489 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214961    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 22:09:52.218489 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214963    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 22:09:52.218489 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214966    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 22:09:52.218489 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214968    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 22:09:52.218489 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214971    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 22:09:52.218489 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214973    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 22:09:52.218489 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214976    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 22:09:52.218489 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214978    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 22:09:52.218986 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214980    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 22:09:52.218986 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214983    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 22:09:52.218986 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214988    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 22:09:52.218986 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214991    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 22:09:52.218986 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214993    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 22:09:52.218986 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214996    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 22:09:52.218986 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.214998    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 22:09:52.218986 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215002    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 22:09:52.218986 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215004    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 22:09:52.218986 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215007    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 22:09:52.218986 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215009    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 22:09:52.218986 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215012    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 22:09:52.218986 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215014    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 22:09:52.218986 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215017    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 22:09:52.218986 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215020    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 22:09:52.218986 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215022    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 22:09:52.218986 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215027    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 22:09:52.218986 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215030    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 22:09:52.218986 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215033    2551 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 22:09:52.219480 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215035    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 22:09:52.219480 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215038    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 22:09:52.219480 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215040    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 22:09:52.219480 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215044    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 22:09:52.219480 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215047    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 22:09:52.219480 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215050    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 22:09:52.219480 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215053    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 22:09:52.219480 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215056    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 22:09:52.219480 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215059    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 22:09:52.219480 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215061    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 22:09:52.219480 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215064    2551 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 22:09:52.219480 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215066    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 22:09:52.219480 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215069    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 22:09:52.219480 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215071    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 22:09:52.219480 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215074    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 22:09:52.219480 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215078    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 22:09:52.219480 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215081    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 22:09:52.219480 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215083    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 22:09:52.219480 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215086    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 22:09:52.219480 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215089    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 22:09:52.219988 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215091    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 22:09:52.219988 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215095    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 22:09:52.219988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215159    2551 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 22:09:52.219988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215173    2551 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 22:09:52.219988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215180    2551 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 22:09:52.219988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215188    2551 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 22:09:52.219988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215194    2551 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 22:09:52.219988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215200    2551 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 22:09:52.219988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215205    2551 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 22:09:52.219988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215208    2551 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 22:09:52.219988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215212    2551 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 22:09:52.219988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215214    2551 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 22:09:52.219988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215218    2551 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 22:09:52.219988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215221    2551 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 22:09:52.219988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215224    2551 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 22:09:52.219988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215227    2551 flags.go:64] FLAG: --cgroup-root=""
Jul 28 22:09:52.219988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215230    2551 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 22:09:52.219988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215233    2551 flags.go:64] FLAG: --client-ca-file=""
Jul 28 22:09:52.219988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215237    2551 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 22:09:52.219988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215240    2551 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 22:09:52.219988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215254    2551 flags.go:64] FLAG: --cluster-domain=""
Jul 28 22:09:52.219988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215256    2551 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 22:09:52.219988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215259    2551 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215263    2551 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215266    2551 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215274    2551 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215277    2551 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215281    2551 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215284    2551 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215288    2551 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215291    2551 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215294    2551 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215297    2551 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215300    2551 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215305    2551 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215308    2551 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215311    2551 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215314    2551 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215317    2551 flags.go:64] FLAG: --enable-server="true"
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215321    2551 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215324    2551 flags.go:64] FLAG: --event-burst="100"
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215327    2551 flags.go:64] FLAG: --event-qps="50"
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215330    2551 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215333    2551 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215336    2551 flags.go:64] FLAG: --eviction-hard=""
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215339    2551 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 22:09:52.220549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215342    2551 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 22:09:52.221130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215346    2551 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 22:09:52.221130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215349    2551 flags.go:64] FLAG: --eviction-soft=""
Jul 28 22:09:52.221130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215351    2551 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 22:09:52.221130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215354    2551 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 22:09:52.221130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215357    2551 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 22:09:52.221130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215361    2551 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 22:09:52.221130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215364    2551 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 22:09:52.221130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215367    2551 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 22:09:52.221130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215370    2551 flags.go:64] FLAG: --feature-gates=""
Jul 28 22:09:52.221130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215374    2551 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 22:09:52.221130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215377    2551 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 22:09:52.221130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215379    2551 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 22:09:52.221130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215384    2551 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 22:09:52.221130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215387    2551 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 22:09:52.221130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215391    2551 flags.go:64] FLAG: --help="false"
Jul 28 22:09:52.221130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215394    2551 flags.go:64] FLAG: --hostname-override="ip-10-0-141-102.ec2.internal"
Jul 28 22:09:52.221130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215397    2551 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 22:09:52.221130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215400    2551 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 22:09:52.221130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215403    2551 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 22:09:52.221130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215406    2551 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 22:09:52.221130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215410    2551 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 22:09:52.221130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215413    2551 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 22:09:52.221130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215416    2551 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215419    2551 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215421    2551 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215424    2551 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215427    2551 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215430    2551 flags.go:64] FLAG: --kube-reserved=""
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215433    2551 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215436    2551 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215439    2551 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215442    2551 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215444    2551 flags.go:64] FLAG: --lock-file=""
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215447    2551 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215450    2551 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215453    2551 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215458    2551 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215461    2551 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215464    2551 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215467    2551 flags.go:64] FLAG: --logging-format="text"
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215469    2551 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215473    2551 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215475    2551 flags.go:64] FLAG: --manifest-url=""
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215478    2551 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215483    2551 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215486    2551 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215491    2551 flags.go:64] FLAG: --max-pods="110"
Jul 28 22:09:52.221780 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215494    2551 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215497    2551 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215500    2551 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215503    2551 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215506    2551 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215509    2551 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215512    2551 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215519    2551 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215535    2551 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215540    2551 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215545    2551 flags.go:64] FLAG: --pod-cidr=""
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215548    2551 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215555    2551 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215558    2551 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215561    2551 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215565    2551 flags.go:64] FLAG: --port="10250"
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215568    2551 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215571    2551 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e26369f4208c0660"
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215575    2551 flags.go:64] FLAG: --qos-reserved=""
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215578    2551 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215581    2551 flags.go:64] FLAG: --register-node="true"
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215583    2551 flags.go:64] FLAG: --register-with-taints=""
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215587    2551 flags.go:64] FLAG: --registry-burst="10"
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215590    2551 flags.go:64] FLAG: --registry-qps="5"
Jul 28 22:09:52.222389 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215593    2551 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215596    2551 flags.go:64] FLAG: --reserved-memory=""
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215599    2551 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215602    2551 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215605    2551 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215608    2551 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215611    2551 flags.go:64] FLAG: --runonce="false"
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215615    2551 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215618    2551 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215621    2551 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215624    2551 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215626    2551 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215629    2551 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215632    2551 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215635    2551 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215638    2551 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215641    2551 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215644    2551 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215647    2551 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215650    2551 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215653    2551 flags.go:64] FLAG: --system-cgroups=""
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215656    2551 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215661    2551 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215664    2551 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215666    2551 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 22:09:52.223030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215670    2551 flags.go:64] FLAG: --tls-min-version=""
Jul 28 22:09:52.223639 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215673    2551 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 22:09:52.223639 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215675    2551 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 22:09:52.223639 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215678    2551 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 22:09:52.223639 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215681    2551 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 22:09:52.223639 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215684    2551 flags.go:64] FLAG: --v="2"
Jul 28 22:09:52.223639 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215688    2551 flags.go:64] FLAG: --version="false"
Jul 28 22:09:52.223639 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215691    2551 flags.go:64] FLAG: --vmodule=""
Jul 28 22:09:52.223639 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215695    2551 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 22:09:52.223639 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.215698    2551 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 22:09:52.223639 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215816    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 22:09:52.223639 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215820    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 22:09:52.223639 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215823    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 22:09:52.223639 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215826    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 22:09:52.223639 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215831    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 22:09:52.223639 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215834    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 22:09:52.223639 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215837    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 22:09:52.223639 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215840    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 22:09:52.223639 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215843    2551 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 22:09:52.223639 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215846    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 22:09:52.223639 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215848    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 22:09:52.223639 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215851    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 22:09:52.223639 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215853    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 22:09:52.224168 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215857    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 22:09:52.224168 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215859    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 22:09:52.224168 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215862    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 22:09:52.224168 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215864    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 22:09:52.224168 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215867    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 22:09:52.224168 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215870    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 22:09:52.224168 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215872    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 22:09:52.224168 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215874    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 22:09:52.224168 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215877    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 22:09:52.224168 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215879    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 22:09:52.224168 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215882    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 22:09:52.224168 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215884    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 22:09:52.224168 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215888    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 22:09:52.224168 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215892    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 22:09:52.224168 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215894    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 22:09:52.224168 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215898    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 22:09:52.224168 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215901    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 22:09:52.224168 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215904    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 22:09:52.224168 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215907    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 22:09:52.224658 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215910    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 22:09:52.224658 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215912    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 22:09:52.224658 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215915    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 22:09:52.224658 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215917    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 22:09:52.224658 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215921    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 22:09:52.224658 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215923    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 22:09:52.224658 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215926    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 22:09:52.224658 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215929    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 22:09:52.224658 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215931    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 22:09:52.224658 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215934    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 22:09:52.224658 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215936    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 22:09:52.224658 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215939    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 22:09:52.224658 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215941    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 22:09:52.224658 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215944    2551 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 22:09:52.224658 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215947    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 22:09:52.224658 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215949    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 22:09:52.224658 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215952    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 22:09:52.224658 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215954    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 22:09:52.224658 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215957    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 22:09:52.224658 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215959    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 22:09:52.225144 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215962    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 22:09:52.225144 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215965    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 22:09:52.225144 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215967    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 22:09:52.225144 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215970    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 22:09:52.225144 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215972    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 22:09:52.225144 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215975    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 22:09:52.225144 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215977    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 22:09:52.225144 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215980    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 22:09:52.225144 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215982    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 22:09:52.225144 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215985    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 22:09:52.225144 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215988    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 22:09:52.225144 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215990    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 22:09:52.225144 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215993    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 22:09:52.225144 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215996    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 22:09:52.225144 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.215998    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 22:09:52.225144 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216001    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 22:09:52.225144 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216004    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 22:09:52.225144 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216007    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 22:09:52.225144 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216011    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 22:09:52.225144 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216014    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 22:09:52.225636 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216017    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 22:09:52.225636 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216019    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 22:09:52.225636 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216022    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 22:09:52.225636 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216025    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 22:09:52.225636 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216028    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 22:09:52.225636 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216030    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 22:09:52.225636 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216033    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 22:09:52.225636 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216036    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 22:09:52.225636 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216039    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 22:09:52.225636 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216041    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 22:09:52.225636 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216044    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 22:09:52.225636 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216046    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 22:09:52.225636 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216049    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 22:09:52.225636 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216051    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 22:09:52.225636 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216054    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 22:09:52.225636 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216056    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 22:09:52.225636 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216059    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 22:09:52.225636 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216061    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 22:09:52.225636 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216064    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 22:09:52.226105 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216067    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 22:09:52.226105 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216069    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 22:09:52.226105 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216072    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 22:09:52.226105 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216074    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 22:09:52.226105 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216077    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 22:09:52.226105 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216081    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 22:09:52.226105 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216084    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 22:09:52.226105 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.216087    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 22:09:52.226105 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.216092    2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 22:09:52.226105 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.222870    2551 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 22:09:52.226105 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.222886    2551 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 22:09:52.226105 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.223446    2551 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 22:09:52.226105 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.223454    2551 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 22:09:52.226105 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223517    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 22:09:52.226105 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223534    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 22:09:52.226517 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223540    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 22:09:52.226517 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223546    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 22:09:52.226517 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223549    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 22:09:52.226517 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223552    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 22:09:52.226517 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223554    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 22:09:52.226517 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223558    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 22:09:52.226517 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223561    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 22:09:52.226517 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223563    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 22:09:52.226517 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223566    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 22:09:52.226517 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223569    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 22:09:52.226517 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223571    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 22:09:52.226517 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223574    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 22:09:52.226517 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223577    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 22:09:52.226517 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223579    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 22:09:52.226517 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223582    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 22:09:52.226517 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223585    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 22:09:52.226517 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223588    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 22:09:52.226517 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223591    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 22:09:52.226517 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223594    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 22:09:52.226517 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223597    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 22:09:52.227026 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223600    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 22:09:52.227026 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223603    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 22:09:52.227026 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223605    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 22:09:52.227026 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223608    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 22:09:52.227026 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223611    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 22:09:52.227026 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223613    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 22:09:52.227026 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223617    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 22:09:52.227026 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223621    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 22:09:52.227026 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223625    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 22:09:52.227026 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223629    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 22:09:52.227026 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223632    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 22:09:52.227026 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223635    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 22:09:52.227026 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223638    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 22:09:52.227026 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223641    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 22:09:52.227026 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223644    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 22:09:52.227026 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223647    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 22:09:52.227026 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223650    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 22:09:52.227026 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223652    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 22:09:52.227026 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223655    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 22:09:52.227563 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223658    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 22:09:52.227563 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223661    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 22:09:52.227563 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223663    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 22:09:52.227563 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223666    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 22:09:52.227563 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223668    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 22:09:52.227563 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223671    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 22:09:52.227563 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223674    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 22:09:52.227563 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223676    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 22:09:52.227563 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223679    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 22:09:52.227563 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223681    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 22:09:52.227563 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223684    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 22:09:52.227563 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223687    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 22:09:52.227563 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223689    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 22:09:52.227563 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223692    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 22:09:52.227563 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223694    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 22:09:52.227563 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223697    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 22:09:52.227563 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223699    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 22:09:52.227563 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223702    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 22:09:52.227563 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223704    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 22:09:52.228040 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223708    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 22:09:52.228040 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223711    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 22:09:52.228040 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223715    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 22:09:52.228040 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223718    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 22:09:52.228040 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223721    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 22:09:52.228040 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223724    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 22:09:52.228040 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223726    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 22:09:52.228040 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223729    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 22:09:52.228040 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223731    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 22:09:52.228040 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223734    2551 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 22:09:52.228040 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223737    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 22:09:52.228040 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223739    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 22:09:52.228040 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223742    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 22:09:52.228040 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223744    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 22:09:52.228040 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223747    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 22:09:52.228040 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223749    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 22:09:52.228040 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223752    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 22:09:52.228040 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223754    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 22:09:52.228040 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223757    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 22:09:52.228040 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223760    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 22:09:52.228586 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223763    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 22:09:52.228586 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223765    2551 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 22:09:52.228586 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223768    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 22:09:52.228586 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223770    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 22:09:52.228586 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223773    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 22:09:52.228586 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223775    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 22:09:52.228586 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223778    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 22:09:52.228586 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223780    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 22:09:52.228586 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223782    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 22:09:52.228586 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223785    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 22:09:52.228586 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223787    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 22:09:52.228586 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223790    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 22:09:52.228586 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223793    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 22:09:52.228586 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223798    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 22:09:52.228586 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223801    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 22:09:52.228586 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223804    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 22:09:52.228586 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223807    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 22:09:52.228586 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223811    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 22:09:52.228586 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223814    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 22:09:52.229037 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.223819    2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 22:09:52.229037 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223967    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 22:09:52.229037 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223972    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 22:09:52.229037 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223975    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 22:09:52.229037 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223978    2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 22:09:52.229037 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223981    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 22:09:52.229037 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223984    2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 22:09:52.229037 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223986    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 22:09:52.229037 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223989    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 22:09:52.229037 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223992    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 22:09:52.229037 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223994    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 22:09:52.229037 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223997    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 22:09:52.229037 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.223999    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 22:09:52.229037 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224002    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 22:09:52.229037 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224004    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 22:09:52.229400 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224007    2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 22:09:52.229400 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224009    2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 22:09:52.229400 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224012    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 22:09:52.229400 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224015    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 22:09:52.229400 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224017    2551 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 22:09:52.229400 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224020    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 22:09:52.229400 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224022    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 22:09:52.229400 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224025    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 22:09:52.229400 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224027    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 22:09:52.229400 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224030    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 22:09:52.229400 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224034    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 22:09:52.229400 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224037    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 22:09:52.229400 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224041    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 22:09:52.229400 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224044    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 22:09:52.229400 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224048    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 22:09:52.229400 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224051    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 22:09:52.229400 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224053    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 22:09:52.229400 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224056    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 22:09:52.229400 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224058    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 22:09:52.229881 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224061    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 22:09:52.229881 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224063    2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 22:09:52.229881 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224066    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 22:09:52.229881 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224068    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 22:09:52.229881 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224071    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 22:09:52.229881 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224074    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 22:09:52.229881 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224076    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 22:09:52.229881 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224079    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 22:09:52.229881 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224081    2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 22:09:52.229881 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224084    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 22:09:52.229881 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224086    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 22:09:52.229881 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224089    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 22:09:52.229881 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224091    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 22:09:52.229881 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224094    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 22:09:52.229881 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224096    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 22:09:52.229881 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224099    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 22:09:52.229881 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224101    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 22:09:52.229881 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224104    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 22:09:52.229881 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224106    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 22:09:52.229881 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224109    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 22:09:52.230363 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224111    2551 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 22:09:52.230363 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224114    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 22:09:52.230363 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224116    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 22:09:52.230363 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224118    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 22:09:52.230363 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224121    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 22:09:52.230363 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224124    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 22:09:52.230363 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224126    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 22:09:52.230363 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224129    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 22:09:52.230363 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224132    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 22:09:52.230363 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224134    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 22:09:52.230363 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224137    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 22:09:52.230363 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224139    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 22:09:52.230363 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224142    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 22:09:52.230363 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224144    2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 22:09:52.230363 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224147    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 22:09:52.230363 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224149    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 22:09:52.230363 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224153    2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 22:09:52.230363 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224156    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 22:09:52.230363 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224159    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 22:09:52.230363 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224162    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 22:09:52.230861 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224164    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 22:09:52.230861 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224168    2551 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 22:09:52.230861 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224171    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 22:09:52.230861 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224174    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 22:09:52.230861 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224176    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 22:09:52.230861 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224179    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 22:09:52.230861 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224182    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 22:09:52.230861 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224185    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 22:09:52.230861 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224187    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 22:09:52.230861 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224190    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 22:09:52.230861 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224192    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 22:09:52.230861 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224194    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 22:09:52.230861 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224197    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 22:09:52.230861 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224199    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 22:09:52.230861 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224202    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 22:09:52.230861 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224204    2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 22:09:52.230861 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224207    2551 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 22:09:52.230861 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224210    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 22:09:52.230861 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224213    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 22:09:52.230861 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224215    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 22:09:52.231340 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224219    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 22:09:52.231340 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224223    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 22:09:52.231340 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224225    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 22:09:52.231340 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224228    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 22:09:52.231340 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224230    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 22:09:52.231340 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:52.224233    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 22:09:52.231340 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.224237    2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 22:09:52.231340 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.224347    2551 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 22:09:52.231340 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.226317    2551 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 22:09:52.231340 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.227507    2551 server.go:1019] "Starting client certificate rotation"
Jul 28 22:09:52.231340 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.227613    2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 22:09:52.231340 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.228231    2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 22:09:52.231340 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.229815    2551 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 22:09:52.234211 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.234191    2551 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 22:09:52.242624 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.242549    2551 log.go:25] "Validated CRI v1 runtime API"
Jul 28 22:09:52.244624 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.244609    2551 log.go:25] "Validated CRI v1 image API"
Jul 28 22:09:52.245991 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.245973    2551 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 22:09:52.248931 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.248911    2551 fs.go:135] Filesystem UUIDs: map[22c70259-550f-48f5-b3e3-049c98ea7fbd:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f6f1a9ba-124f-41b0-aace-6167cd934113:/dev/nvme0n1p3]
Jul 28 22:09:52.248993 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.248929    2551 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 22:09:52.254987 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.254880    2551 manager.go:216] Machine: {Timestamp:2026-07-28 22:09:52.253701994 +0000 UTC m=+0.271667679 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100713 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24cff0b6c72b80d34cc7f98a34b02e SystemUUID:ec24cff0-b6c7-2b80-d34c-c7f98a34b02e BootID:0c5fdc56-1560-41ac-b296-cafada69b2e4 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:32:51:6c:0b:bd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:32:51:6c:0b:bd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1a:fe:dd:53:6c:a6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 22:09:52.254987 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.254981    2551 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 22:09:52.255104 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.255088    2551 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 22:09:52.255415 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.255398    2551 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 22:09:52.257549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.257511    2551 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 22:09:52.257686 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.257552    2551 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-102.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 22:09:52.258219 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.258209    2551 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 22:09:52.258249 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.258221    2551 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 22:09:52.258249 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.258229    2551 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 22:09:52.258768 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.258758    2551 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 22:09:52.259615 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.259605    2551 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 22:09:52.262247 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.262236    2551 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 22:09:52.262366 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.262357    2551 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 22:09:52.262399 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.262387    2551 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 22:09:52.262426 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.262401    2551 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 22:09:52.262454 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.262425    2551 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 22:09:52.262454 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.262438    2551 kubelet.go:398] "Adding apiserver pod source"
Jul 28 22:09:52.262454 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.262451    2551 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 22:09:52.262842 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.262828    2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 22:09:52.262886 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.262847    2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 22:09:52.264747 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.264721    2551 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 22:09:52.266639 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.266626    2551 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 22:09:52.266698 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.266656    2551 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 22:09:52.267935 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.267920    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 22:09:52.267972 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.267945    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 22:09:52.267972 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.267956    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 22:09:52.267972 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.267966    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 22:09:52.268103 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.267978    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 22:09:52.268103 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.267987    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 22:09:52.268103 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.267998    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 22:09:52.268103 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.268014    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 22:09:52.268103 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.268023    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 22:09:52.268103 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.268029    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 22:09:52.268103 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.268039    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 22:09:52.268103 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.268047    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 22:09:52.268667 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.268656    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 22:09:52.268667 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.268667    2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 22:09:52.271835 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.271818    2551 server.go:1268] "Started kubelet"
Jul 28 22:09:52.272133 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.272022    2551 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 22:09:52.272282 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.272121    2551 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 22:09:52.275372 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.275351    2551 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 22:09:52.275915 ip-10-0-141-102 systemd[1]: Started Kubernetes Kubelet.
Jul 28 22:09:52.276338 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:52.276293    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-102.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 22:09:52.276472 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.276342    2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-102.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 22:09:52.276777 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:52.276637    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 22:09:52.277359 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.277336    2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wm2nm"
Jul 28 22:09:52.277598 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:52.277562    2551 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 22:09:52.277692 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.277595    2551 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 22:09:52.279680 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.279662    2551 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 22:09:52.280547 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:52.279723    2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-102.ec2.internal.18c6939a142d17d5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-102.ec2.internal,UID:ip-10-0-141-102.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-102.ec2.internal,},FirstTimestamp:2026-07-28 22:09:52.271726549 +0000 UTC m=+0.289692234,LastTimestamp:2026-07-28 22:09:52.271726549 +0000 UTC m=+0.289692234,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-102.ec2.internal,}"
Jul 28 22:09:52.284943 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.284817    2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wm2nm"
Jul 28 22:09:52.290410 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.290395    2551 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 22:09:52.290466 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.290421    2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 22:09:52.290513 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:52.290503    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-102.ec2.internal\" not found"
Jul 28 22:09:52.291066 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.290963    2551 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 22:09:52.291066 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.291000    2551 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 22:09:52.291066 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.291013    2551 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 22:09:52.291268 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.291109    2551 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 22:09:52.291268 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.291118    2551 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 22:09:52.291361 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.291294    2551 factory.go:55] Registering systemd factory
Jul 28 22:09:52.291361 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.291308    2551 factory.go:223] Registration of the systemd container factory successfully
Jul 28 22:09:52.291575 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.291557    2551 factory.go:153] Registering CRI-O factory
Jul 28 22:09:52.291645 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.291577    2551 factory.go:223] Registration of the crio container factory successfully
Jul 28 22:09:52.291645 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.291635    2551 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 22:09:52.291730 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.291660    2551 factory.go:103] Registering Raw factory
Jul 28 22:09:52.291730 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.291677    2551 manager.go:1196] Started watching for new ooms in manager
Jul 28 22:09:52.292126 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:52.292100    2551 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-102.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 22:09:52.292205 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:52.292189    2551 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 22:09:52.292323 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.292310    2551 manager.go:349] Starting recovery of all containers
Jul 28 22:09:52.302038 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.302019    2551 manager.go:354] Recovery completed
Jul 28 22:09:52.306178 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.306163    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 22:09:52.308658 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.308641    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-102.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 22:09:52.308735 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.308675    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-102.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 22:09:52.308735 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.308692    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-102.ec2.internal" event="NodeHasSufficientPID"
Jul 28 22:09:52.309202 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.309187    2551 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 22:09:52.309202 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.309200    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 22:09:52.309285 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.309199    2551 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 22:09:52.309285 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.309235    2551 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 22:09:52.309896 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.309874    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-102.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 22:09:52.309943 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.309919    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-102.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 22:09:52.309943 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.309931    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-102.ec2.internal" event="NodeHasSufficientPID"
Jul 28 22:09:52.311787 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.311771    2551 policy_none.go:49] "None policy: Start"
Jul 28 22:09:52.311787 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.311787    2551 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 22:09:52.311929 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.311797    2551 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 22:09:52.313750 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.313737    2551 policy_none.go:47] "Start"
Jul 28 22:09:52.357710 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.357554    2551 manager.go:337] "Starting Device Plugin manager"
Jul 28 22:09:52.357796 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:52.357742    2551 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 22:09:52.357796 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.357753    2551 server.go:87] "Starting device plugin registration server"
Jul 28 22:09:52.357992 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.357981    2551 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 22:09:52.358030 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.357997    2551 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 22:09:52.358132 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.358115    2551 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 22:09:52.358230 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.358195    2551 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 22:09:52.358230 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.358204    2551 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 22:09:52.358733 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:52.358716    2551 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 22:09:52.358812 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:52.358760    2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-102.ec2.internal\" not found"
Jul 28 22:09:52.454821 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.454773    2551 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 22:09:52.456822 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.456024    2551 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 22:09:52.456822 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.456071    2551 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 22:09:52.456822 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.456100    2551 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 22:09:52.456822 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:52.456143    2551 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 22:09:52.458414 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.458393    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 22:09:52.458915 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.458900    2551 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 22:09:52.459751 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.459736    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-102.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 22:09:52.459819 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.459764    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-102.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 22:09:52.459819 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.459774    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-102.ec2.internal" event="NodeHasSufficientPID"
Jul 28 22:09:52.459819 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.459796    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-102.ec2.internal"
Jul 28 22:09:52.465559 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.465539    2551 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-102.ec2.internal"
Jul 28 22:09:52.465604 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:52.465562    2551 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-102.ec2.internal\": node \"ip-10-0-141-102.ec2.internal\" not found"
Jul 28 22:09:52.480255 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:52.480237    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-102.ec2.internal\" not found"
Jul 28 22:09:52.557035 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.556990    2551 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-102.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-102.ec2.internal"]
Jul 28 22:09:52.557149 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.557092    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 22:09:52.558878 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.558860    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-102.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 22:09:52.558951 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.558894    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-102.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 22:09:52.558951 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.558906    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-102.ec2.internal" event="NodeHasSufficientPID"
Jul 28 22:09:52.561190 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.561170    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 22:09:52.561290 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.561275    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-102.ec2.internal"
Jul 28 22:09:52.561354 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.561306    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 22:09:52.562059 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.562042    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-102.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 22:09:52.562137 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.562067    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-102.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 22:09:52.562137 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.562077    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-102.ec2.internal" event="NodeHasSufficientPID"
Jul 28 22:09:52.562137 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.562099    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-102.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 22:09:52.562137 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.562119    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-102.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 22:09:52.562137 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.562134    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-102.ec2.internal" event="NodeHasSufficientPID"
Jul 28 22:09:52.564480 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.564461    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-102.ec2.internal"
Jul 28 22:09:52.564538 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.564500    2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 22:09:52.565290 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.565260    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-102.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 22:09:52.565360 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.565293    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-102.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 22:09:52.565360 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.565309    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-102.ec2.internal" event="NodeHasSufficientPID"
Jul 28 22:09:52.581164 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:52.581135    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-102.ec2.internal\" not found"
Jul 28 22:09:52.581249 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:52.581229    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-102.ec2.internal\" not found" node="ip-10-0-141-102.ec2.internal"
Jul 28 22:09:52.585683 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:52.585667    2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-102.ec2.internal\" not found" node="ip-10-0-141-102.ec2.internal"
Jul 28 22:09:52.592873 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.592852    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8165253f3124970b10c0b16ca151ca09-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-102.ec2.internal\" (UID: \"8165253f3124970b10c0b16ca151ca09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-102.ec2.internal"
Jul 28 22:09:52.592964 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.592887    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8165253f3124970b10c0b16ca151ca09-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-102.ec2.internal\" (UID: \"8165253f3124970b10c0b16ca151ca09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-102.ec2.internal"
Jul 28 22:09:52.592964 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.592915    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/76decd8d884af4bc21a6a287671d52e8-config\") pod \"kube-apiserver-proxy-ip-10-0-141-102.ec2.internal\" (UID: \"76decd8d884af4bc21a6a287671d52e8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-102.ec2.internal"
Jul 28 22:09:52.681286 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:52.681241    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-102.ec2.internal\" not found"
Jul 28 22:09:52.693294 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.693264    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8165253f3124970b10c0b16ca151ca09-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-102.ec2.internal\" (UID: \"8165253f3124970b10c0b16ca151ca09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-102.ec2.internal"
Jul 28 22:09:52.693405 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.693299    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8165253f3124970b10c0b16ca151ca09-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-102.ec2.internal\" (UID: \"8165253f3124970b10c0b16ca151ca09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-102.ec2.internal"
Jul 28 22:09:52.693405 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.693320    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/76decd8d884af4bc21a6a287671d52e8-config\") pod \"kube-apiserver-proxy-ip-10-0-141-102.ec2.internal\" (UID: \"76decd8d884af4bc21a6a287671d52e8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-102.ec2.internal"
Jul 28 22:09:52.693405 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.693363    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8165253f3124970b10c0b16ca151ca09-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-102.ec2.internal\" (UID: \"8165253f3124970b10c0b16ca151ca09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-102.ec2.internal"
Jul 28 22:09:52.693405 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.693370    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8165253f3124970b10c0b16ca151ca09-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-102.ec2.internal\" (UID: \"8165253f3124970b10c0b16ca151ca09\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-102.ec2.internal"
Jul 28 22:09:52.693405 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.693366    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/76decd8d884af4bc21a6a287671d52e8-config\") pod \"kube-apiserver-proxy-ip-10-0-141-102.ec2.internal\" (UID: \"76decd8d884af4bc21a6a287671d52e8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-102.ec2.internal"
Jul 28 22:09:52.782204 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:52.782121    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-102.ec2.internal\" not found"
Jul 28 22:09:52.886770 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.886726    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-102.ec2.internal"
Jul 28 22:09:52.886860 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:52.886725    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-102.ec2.internal\" not found"
Jul 28 22:09:52.889106 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:52.889088    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-102.ec2.internal"
Jul 28 22:09:52.991221 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:52.991183    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-102.ec2.internal\" not found"
Jul 28 22:09:53.091568 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:53.091479    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-102.ec2.internal\" not found"
Jul 28 22:09:53.192291 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:53.192247    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-102.ec2.internal\" not found"
Jul 28 22:09:53.227139 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:53.227097    2551 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 22:09:53.227655 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:53.227314    2551 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 22:09:53.289930 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:53.289887    2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 22:04:52 +0000 UTC" deadline="2028-03-14 10:51:17.602889348 +0000 UTC"
Jul 28 22:09:53.289930 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:53.289917    2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14268h41m24.312975768s"
Jul 28 22:09:53.292371 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:53.292348    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-102.ec2.internal\" not found"
Jul 28 22:09:53.393294 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:53.393271    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-102.ec2.internal\" not found"
Jul 28 22:09:53.493517 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:53.493485    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-102.ec2.internal\" not found"
Jul 28 22:09:53.506925 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:53.506889    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76decd8d884af4bc21a6a287671d52e8.slice/crio-76a8de95a9be3bf0bd472fe93a0ed93054a0067bcdb6b6e55376d2d4b5f2ddd9 WatchSource:0}: Error finding container 76a8de95a9be3bf0bd472fe93a0ed93054a0067bcdb6b6e55376d2d4b5f2ddd9: Status 404 returned error can't find the container with id 76a8de95a9be3bf0bd472fe93a0ed93054a0067bcdb6b6e55376d2d4b5f2ddd9
Jul 28 22:09:53.512993 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:53.512970    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 22:09:53.518795 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:53.518775    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8165253f3124970b10c0b16ca151ca09.slice/crio-c8d664c4bbcf6f374ff079710bb3cbe43ff46b26b46193ae355e1a8c8c5134ab WatchSource:0}: Error finding container c8d664c4bbcf6f374ff079710bb3cbe43ff46b26b46193ae355e1a8c8c5134ab: Status 404 returned error can't find the container with id c8d664c4bbcf6f374ff079710bb3cbe43ff46b26b46193ae355e1a8c8c5134ab
Jul 28 22:09:53.594489 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:53.594443    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-102.ec2.internal\" not found"
Jul 28 22:09:53.696694 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:53.696613    2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-102.ec2.internal\" not found"
Jul 28 22:09:53.699205 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:53.699185    2551 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 22:09:53.761640 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:53.761611    2551 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 22:09:53.792586 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:53.792547    2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-102.ec2.internal"
Jul 28 22:09:53.803782 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:53.803759    2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 22:09:53.804657 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:53.804643    2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-102.ec2.internal"
Jul 28 22:09:53.812809 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:53.812789    2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 22:09:53.892137 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:53.892105    2551 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 22:09:54.265177 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.265143    2551 apiserver.go:52] "Watching apiserver"
Jul 28 22:09:54.271901 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.271870    2551 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 22:09:54.274036 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.274005    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-mbcll","openshift-ovn-kubernetes/ovnkube-node-222qs","kube-system/kube-apiserver-proxy-ip-10-0-141-102.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt","openshift-cluster-node-tuning-operator/tuned-mw9qg","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-102.ec2.internal","openshift-multus/network-metrics-daemon-k6pcj","openshift-network-diagnostics/network-check-target-zb7gv","openshift-network-operator/iptables-alerter-g6dq5","kube-system/global-pull-secret-syncer-7cn74","kube-system/konnectivity-agent-sm9g2","openshift-dns/node-resolver-trrlj","openshift-image-registry/node-ca-25fc9","openshift-multus/multus-additional-cni-plugins-dfgvv"]
Jul 28 22:09:54.276984 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.276963    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:09:54.277080 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:54.277055    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6pcj" podUID="c54b2692-1620-47a1-b04e-7b7f6b5df90a"
Jul 28 22:09:54.279686 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.279662    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-25fc9"
Jul 28 22:09:54.289257 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.289230    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt"
Jul 28 22:09:54.289370 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.289264    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.294025 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.291968    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 22:09:54.294025 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.292102    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 22:09:54.294025 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.292109    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pzrlj\""
Jul 28 22:09:54.294025 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.292137    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wqphj\""
Jul 28 22:09:54.294025 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.292223    2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 22:09:54.294025 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.292306    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 22:09:54.294025 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.292543    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 22:09:54.294025 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.292714    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 22:09:54.294025 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.293067    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 22:09:54.294529 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.293445    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 22:09:54.294529 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.293473    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 22:09:54.294529 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.294389    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:09:54.294529 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.293494    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 22:09:54.294529 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:54.294497    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zb7gv" podUID="9a7c5df8-e808-4a53-9697-7dc091ca7660"
Jul 28 22:09:54.294801 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.293841    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.294801 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.293632    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 22:09:54.295480 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.293527    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 22:09:54.297034 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.296619    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g6dq5"
Jul 28 22:09:54.297959 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.297927    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 22:09:54.298205 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.298188    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-62km7\""
Jul 28 22:09:54.298530 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.298513    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 22:09:54.298923 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.298903    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ggdns\""
Jul 28 22:09:54.299215 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.299193    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 22:09:54.299436 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.299418    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 22:09:54.299641 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.299624    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 22:09:54.299719 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.299658    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-v48n8\""
Jul 28 22:09:54.299964 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.299947    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1399c32f-c1f8-4d3e-8350-b7900e68a30c-cni-binary-copy\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.300040 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.299976    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-etc-systemd\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.300040 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300014    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-sys\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.300150 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300040    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1399c32f-c1f8-4d3e-8350-b7900e68a30c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.300150 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300071    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-etc-kubernetes\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.300150 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300090    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-etc-tuned\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.300150 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300109    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs\") pod \"network-metrics-daemon-k6pcj\" (UID: \"c54b2692-1620-47a1-b04e-7b7f6b5df90a\") " pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:09:54.300150 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300124    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v9njt\" (UID: \"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt"
Jul 28 22:09:54.300150 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300141    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87-etc-selinux\") pod \"aws-ebs-csi-driver-node-v9njt\" (UID: \"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt"
Jul 28 22:09:54.300503 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300155    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nngdk\" (UniqueName: \"kubernetes.io/projected/4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87-kube-api-access-nngdk\") pod \"aws-ebs-csi-driver-node-v9njt\" (UID: \"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt"
Jul 28 22:09:54.300503 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300185    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e7cd9a57-9827-497b-a1fa-5c168e415621-iptables-alerter-script\") pod \"iptables-alerter-g6dq5\" (UID: \"e7cd9a57-9827-497b-a1fa-5c168e415621\") " pod="openshift-network-operator/iptables-alerter-g6dq5"
Jul 28 22:09:54.300503 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300200    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9c7c65c8-313a-4e85-9c0b-4c190cf1cbea-serviceca\") pod \"node-ca-25fc9\" (UID: \"9c7c65c8-313a-4e85-9c0b-4c190cf1cbea\") " pod="openshift-image-registry/node-ca-25fc9"
Jul 28 22:09:54.300503 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300221    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1399c32f-c1f8-4d3e-8350-b7900e68a30c-os-release\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.300503 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300244    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-etc-modprobe-d\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.300503 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300271    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87-registration-dir\") pod \"aws-ebs-csi-driver-node-v9njt\" (UID: \"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt"
Jul 28 22:09:54.300503 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300294    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-etc-sysctl-d\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.300503 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300316    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-run\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.300503 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300357    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6qr9\" (UniqueName: \"kubernetes.io/projected/c54b2692-1620-47a1-b04e-7b7f6b5df90a-kube-api-access-v6qr9\") pod \"network-metrics-daemon-k6pcj\" (UID: \"c54b2692-1620-47a1-b04e-7b7f6b5df90a\") " pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:09:54.300503 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300384    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87-sys-fs\") pod \"aws-ebs-csi-driver-node-v9njt\" (UID: \"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt"
Jul 28 22:09:54.300503 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300408    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1399c32f-c1f8-4d3e-8350-b7900e68a30c-system-cni-dir\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.300503 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300442    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1399c32f-c1f8-4d3e-8350-b7900e68a30c-cnibin\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.300503 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300468    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqm6c\" (UniqueName: \"kubernetes.io/projected/1399c32f-c1f8-4d3e-8350-b7900e68a30c-kube-api-access-zqm6c\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.300503 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300492    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-tmp\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.301276 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300538    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87-socket-dir\") pod \"aws-ebs-csi-driver-node-v9njt\" (UID: \"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt"
Jul 28 22:09:54.301276 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300571    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87-device-dir\") pod \"aws-ebs-csi-driver-node-v9njt\" (UID: \"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt"
Jul 28 22:09:54.301276 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300608    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e7cd9a57-9827-497b-a1fa-5c168e415621-host-slash\") pod \"iptables-alerter-g6dq5\" (UID: \"e7cd9a57-9827-497b-a1fa-5c168e415621\") " pod="openshift-network-operator/iptables-alerter-g6dq5"
Jul 28 22:09:54.301276 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300784    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9c7c65c8-313a-4e85-9c0b-4c190cf1cbea-host\") pod \"node-ca-25fc9\" (UID: \"9c7c65c8-313a-4e85-9c0b-4c190cf1cbea\") " pod="openshift-image-registry/node-ca-25fc9"
Jul 28 22:09:54.301276 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300825    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1399c32f-c1f8-4d3e-8350-b7900e68a30c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.301276 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300852    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1399c32f-c1f8-4d3e-8350-b7900e68a30c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.301276 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300913    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-lib-modules\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.301276 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.300968    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79htf\" (UniqueName: \"kubernetes.io/projected/9c7c65c8-313a-4e85-9c0b-4c190cf1cbea-kube-api-access-79htf\") pod \"node-ca-25fc9\" (UID: \"9c7c65c8-313a-4e85-9c0b-4c190cf1cbea\") " pod="openshift-image-registry/node-ca-25fc9"
Jul 28 22:09:54.301276 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.301000    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-etc-sysconfig\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.301276 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.301023    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-etc-sysctl-conf\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.301276 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.301047    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-var-lib-kubelet\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.301276 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.301083    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-host\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.301276 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.301109    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.301276 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.301107    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twwl4\" (UniqueName: \"kubernetes.io/projected/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-kube-api-access-twwl4\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.301897 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.301282    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hx54\" (UniqueName: \"kubernetes.io/projected/e7cd9a57-9827-497b-a1fa-5c168e415621-kube-api-access-4hx54\") pod \"iptables-alerter-g6dq5\" (UID: \"e7cd9a57-9827-497b-a1fa-5c168e415621\") " pod="openshift-network-operator/iptables-alerter-g6dq5"
Jul 28 22:09:54.301897 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.301307    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v4p4\" (UniqueName: \"kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4\") pod \"network-check-target-zb7gv\" (UID: \"9a7c5df8-e808-4a53-9697-7dc091ca7660\") " pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:09:54.302013 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.301995    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.305743 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.304384    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:09:54.305743 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:54.304456    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7cn74" podUID="be77a0c4-e19b-49d2-b547-c36c3d79dd33"
Jul 28 22:09:54.305743 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.304919    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 22:09:54.305743 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.305135    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-l9c5p\""
Jul 28 22:09:54.305743 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.305315    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 22:09:54.305743 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.305523    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 22:09:54.305743 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.305591    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 22:09:54.306182 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.305980    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 22:09:54.306182 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.306161    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 22:09:54.306878 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.306366    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dph6m\""
Jul 28 22:09:54.306878 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.306406    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 22:09:54.307524 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.307056    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sm9g2"
Jul 28 22:09:54.308999 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.308980    2551 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 22:09:54.309307 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.309278    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 22:09:54.309588 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.309569    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-trrlj"
Jul 28 22:09:54.310175 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.310093    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 22:09:54.310175 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.310163    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-n4jnw\""
Jul 28 22:09:54.311525 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.311482    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 22:09:54.311852 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.311831    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gq5wl\""
Jul 28 22:09:54.311937 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.311917    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 22:09:54.344305 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.344282    2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-59g69"
Jul 28 22:09:54.352268 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.352247    2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-59g69"
Jul 28 22:09:54.393743 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.393708    2551 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 22:09:54.401983 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.401960    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-etc-kubernetes\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.402120 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.401989    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-etc-tuned\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.402120 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402021    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-system-cni-dir\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.402244 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402118    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-etc-kubernetes\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.402244 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402163    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-host-run-netns\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.402244 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402197    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nngdk\" (UniqueName: \"kubernetes.io/projected/4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87-kube-api-access-nngdk\") pod \"aws-ebs-csi-driver-node-v9njt\" (UID: \"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt"
Jul 28 22:09:54.402244 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402225    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e7cd9a57-9827-497b-a1fa-5c168e415621-iptables-alerter-script\") pod \"iptables-alerter-g6dq5\" (UID: \"e7cd9a57-9827-497b-a1fa-5c168e415621\") " pod="openshift-network-operator/iptables-alerter-g6dq5"
Jul 28 22:09:54.402442 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402251    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1399c32f-c1f8-4d3e-8350-b7900e68a30c-os-release\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.402442 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402275    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-os-release\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.402442 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402300    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-host-run-k8s-cni-cncf-io\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.402442 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402326    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-host-var-lib-cni-bin\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.402442 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402352    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-host-var-lib-cni-multus\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.402442 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402366    2551 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 22:09:54.402741 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402529    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1399c32f-c1f8-4d3e-8350-b7900e68a30c-os-release\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.402741 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402376    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-etc-kubernetes\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.402741 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402644    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87-registration-dir\") pod \"aws-ebs-csi-driver-node-v9njt\" (UID: \"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt"
Jul 28 22:09:54.402741 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402688    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-run\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.402741 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402721    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-node-log\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.402988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402766    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/481b2f34-6f30-4e75-986c-c7eba23f6214-ovn-node-metrics-cert\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.402988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402795    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx6hr\" (UniqueName: \"kubernetes.io/projected/481b2f34-6f30-4e75-986c-c7eba23f6214-kube-api-access-gx6hr\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.402988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402810    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87-registration-dir\") pod \"aws-ebs-csi-driver-node-v9njt\" (UID: \"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt"
Jul 28 22:09:54.402988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402819    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-hostroot\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.402988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402868    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87-sys-fs\") pod \"aws-ebs-csi-driver-node-v9njt\" (UID: \"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt"
Jul 28 22:09:54.402988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402896    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1399c32f-c1f8-4d3e-8350-b7900e68a30c-cnibin\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.402988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402928    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqm6c\" (UniqueName: \"kubernetes.io/projected/1399c32f-c1f8-4d3e-8350-b7900e68a30c-kube-api-access-zqm6c\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.402988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402957    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-tmp\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.402988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402960    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-run\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.402988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.402979    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e7cd9a57-9827-497b-a1fa-5c168e415621-iptables-alerter-script\") pod \"iptables-alerter-g6dq5\" (UID: \"e7cd9a57-9827-497b-a1fa-5c168e415621\") " pod="openshift-network-operator/iptables-alerter-g6dq5"
Jul 28 22:09:54.403482 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.403007    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-multus-socket-dir-parent\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.403482 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.403055    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/934b288b-0f1f-4bd5-a697-d41516122892-konnectivity-ca\") pod \"konnectivity-agent-sm9g2\" (UID: \"934b288b-0f1f-4bd5-a697-d41516122892\") " pod="kube-system/konnectivity-agent-sm9g2"
Jul 28 22:09:54.403482 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.403273    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-var-lib-kubelet\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.403482 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.403283    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1399c32f-c1f8-4d3e-8350-b7900e68a30c-cnibin\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.403482 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.403324    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/46fd1bad-a9c6-4872-aa09-48a8d38fc005-cni-binary-copy\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.403482 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.403334    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87-sys-fs\") pod \"aws-ebs-csi-driver-node-v9njt\" (UID: \"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt"
Jul 28 22:09:54.403482 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.403362    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87-socket-dir\") pod \"aws-ebs-csi-driver-node-v9njt\" (UID: \"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt"
Jul 28 22:09:54.403482 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.403379    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-var-lib-kubelet\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.403482 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.403389    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e7cd9a57-9827-497b-a1fa-5c168e415621-host-slash\") pod \"iptables-alerter-g6dq5\" (UID: \"e7cd9a57-9827-497b-a1fa-5c168e415621\") " pod="openshift-network-operator/iptables-alerter-g6dq5"
Jul 28 22:09:54.403482 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.403429    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1399c32f-c1f8-4d3e-8350-b7900e68a30c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.403482 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.403470    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e7cd9a57-9827-497b-a1fa-5c168e415621-host-slash\") pod \"iptables-alerter-g6dq5\" (UID: \"e7cd9a57-9827-497b-a1fa-5c168e415621\") " pod="openshift-network-operator/iptables-alerter-g6dq5"
Jul 28 22:09:54.403962 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.403490    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-lib-modules\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.403962 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.403514    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87-socket-dir\") pod \"aws-ebs-csi-driver-node-v9njt\" (UID: \"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt"
Jul 28 22:09:54.403962 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.403546    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-run-ovn\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.403962 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.403591    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.403962 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.403624    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-lib-modules\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.403962 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.403630    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79htf\" (UniqueName: \"kubernetes.io/projected/9c7c65c8-313a-4e85-9c0b-4c190cf1cbea-kube-api-access-79htf\") pod \"node-ca-25fc9\" (UID: \"9c7c65c8-313a-4e85-9c0b-4c190cf1cbea\") " pod="openshift-image-registry/node-ca-25fc9"
Jul 28 22:09:54.403962 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.403637    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1399c32f-c1f8-4d3e-8350-b7900e68a30c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.403962 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.403665    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-etc-sysconfig\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.403962 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.403792    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hx54\" (UniqueName: \"kubernetes.io/projected/e7cd9a57-9827-497b-a1fa-5c168e415621-kube-api-access-4hx54\") pod \"iptables-alerter-g6dq5\" (UID: \"e7cd9a57-9827-497b-a1fa-5c168e415621\") " pod="openshift-network-operator/iptables-alerter-g6dq5"
Jul 28 22:09:54.403962 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.403824    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-etc-sysconfig\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.403962 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.403877    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v4p4\" (UniqueName: \"kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4\") pod \"network-check-target-zb7gv\" (UID: \"9a7c5df8-e808-4a53-9697-7dc091ca7660\") " pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:09:54.403962 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.403952    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-multus-cni-dir\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.404538 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404070    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-host-run-multus-certs\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.404538 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404122    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03112039-98fd-478f-b873-2504ce8aae02-tmp-dir\") pod \"node-resolver-trrlj\" (UID: \"03112039-98fd-478f-b873-2504ce8aae02\") " pod="openshift-dns/node-resolver-trrlj"
Jul 28 22:09:54.404538 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404165    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1399c32f-c1f8-4d3e-8350-b7900e68a30c-cni-binary-copy\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.404538 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404193    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-etc-systemd\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.404538 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404219    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/46fd1bad-a9c6-4872-aa09-48a8d38fc005-multus-daemon-config\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.404538 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404243    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-host-slash\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.404538 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404280    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-host-run-ovn-kubernetes\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.404538 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404317    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-etc-systemd\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.404538 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404372    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-host-cni-netd\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.404538 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404404    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/481b2f34-6f30-4e75-986c-c7eba23f6214-env-overrides\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.404538 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404436    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1399c32f-c1f8-4d3e-8350-b7900e68a30c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.404538 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404476    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-cnibin\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.404538 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404502    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-multus-conf-dir\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.405232 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404551    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-host-kubelet\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.405232 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404587    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-var-lib-openvswitch\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.405232 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404620    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/be77a0c4-e19b-49d2-b547-c36c3d79dd33-kubelet-config\") pod \"global-pull-secret-syncer-7cn74\" (UID: \"be77a0c4-e19b-49d2-b547-c36c3d79dd33\") " pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:09:54.405232 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404675    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs\") pod \"network-metrics-daemon-k6pcj\" (UID: \"c54b2692-1620-47a1-b04e-7b7f6b5df90a\") " pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:09:54.405232 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404703    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v9njt\" (UID: \"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt"
Jul 28 22:09:54.405232 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404740    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87-etc-selinux\") pod \"aws-ebs-csi-driver-node-v9njt\" (UID: \"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt"
Jul 28 22:09:54.405232 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404743    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1399c32f-c1f8-4d3e-8350-b7900e68a30c-cni-binary-copy\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.405232 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404781    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9c7c65c8-313a-4e85-9c0b-4c190cf1cbea-serviceca\") pod \"node-ca-25fc9\" (UID: \"9c7c65c8-313a-4e85-9c0b-4c190cf1cbea\") " pod="openshift-image-registry/node-ca-25fc9"
Jul 28 22:09:54.405232 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404875    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87-etc-selinux\") pod \"aws-ebs-csi-driver-node-v9njt\" (UID: \"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt"
Jul 28 22:09:54.405232 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404909    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-etc-modprobe-d\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.405232 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404944    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-etc-sysctl-d\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.405232 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.404974    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx2tj\" (UniqueName: \"kubernetes.io/projected/46fd1bad-a9c6-4872-aa09-48a8d38fc005-kube-api-access-bx2tj\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.405232 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405025    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1399c32f-c1f8-4d3e-8350-b7900e68a30c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.405232 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405057    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-etc-modprobe-d\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.405232 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:54.405067    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:09:54.405232 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405088    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-run-openvswitch\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.405232 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405116    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-log-socket\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.406093 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405135    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v9njt\" (UID: \"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt"
Jul 28 22:09:54.406093 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405142    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-etc-sysctl-d\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.406093 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405156    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/481b2f34-6f30-4e75-986c-c7eba23f6214-ovnkube-config\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.406093 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405184    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/be77a0c4-e19b-49d2-b547-c36c3d79dd33-dbus\") pod \"global-pull-secret-syncer-7cn74\" (UID: \"be77a0c4-e19b-49d2-b547-c36c3d79dd33\") " pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:09:54.406093 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405211    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9c7c65c8-313a-4e85-9c0b-4c190cf1cbea-serviceca\") pod \"node-ca-25fc9\" (UID: \"9c7c65c8-313a-4e85-9c0b-4c190cf1cbea\") " pod="openshift-image-registry/node-ca-25fc9"
Jul 28 22:09:54.406093 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:54.405222    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs podName:c54b2692-1620-47a1-b04e-7b7f6b5df90a nodeName:}" failed. No retries permitted until 2026-07-28 22:09:54.905200271 +0000 UTC m=+2.923165950 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs") pod "network-metrics-daemon-k6pcj" (UID: "c54b2692-1620-47a1-b04e-7b7f6b5df90a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:09:54.406093 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405243    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6qr9\" (UniqueName: \"kubernetes.io/projected/c54b2692-1620-47a1-b04e-7b7f6b5df90a-kube-api-access-v6qr9\") pod \"network-metrics-daemon-k6pcj\" (UID: \"c54b2692-1620-47a1-b04e-7b7f6b5df90a\") " pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:09:54.406093 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405270    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1399c32f-c1f8-4d3e-8350-b7900e68a30c-system-cni-dir\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.406093 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405297    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-run-systemd\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.406093 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405340    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/481b2f34-6f30-4e75-986c-c7eba23f6214-ovnkube-script-lib\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.406093 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405351    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1399c32f-c1f8-4d3e-8350-b7900e68a30c-system-cni-dir\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.406093 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405435    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret\") pod \"global-pull-secret-syncer-7cn74\" (UID: \"be77a0c4-e19b-49d2-b547-c36c3d79dd33\") " pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:09:54.406093 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405486    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/934b288b-0f1f-4bd5-a697-d41516122892-agent-certs\") pod \"konnectivity-agent-sm9g2\" (UID: \"934b288b-0f1f-4bd5-a697-d41516122892\") " pod="kube-system/konnectivity-agent-sm9g2"
Jul 28 22:09:54.406093 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405518    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twwl4\" (UniqueName: \"kubernetes.io/projected/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-kube-api-access-twwl4\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.406093 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405573    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87-device-dir\") pod \"aws-ebs-csi-driver-node-v9njt\" (UID: \"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt"
Jul 28 22:09:54.406093 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405601    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9c7c65c8-313a-4e85-9c0b-4c190cf1cbea-host\") pod \"node-ca-25fc9\" (UID: \"9c7c65c8-313a-4e85-9c0b-4c190cf1cbea\") " pod="openshift-image-registry/node-ca-25fc9"
Jul 28 22:09:54.406869 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405629    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1399c32f-c1f8-4d3e-8350-b7900e68a30c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.406869 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405656    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-etc-openvswitch\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.406869 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405682    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-etc-sysctl-conf\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.406869 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405684    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9c7c65c8-313a-4e85-9c0b-4c190cf1cbea-host\") pod \"node-ca-25fc9\" (UID: \"9c7c65c8-313a-4e85-9c0b-4c190cf1cbea\") " pod="openshift-image-registry/node-ca-25fc9"
Jul 28 22:09:54.406869 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405684    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87-device-dir\") pod \"aws-ebs-csi-driver-node-v9njt\" (UID: \"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt"
Jul 28 22:09:54.406869 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405705    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-host\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.406869 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405740    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-host-var-lib-kubelet\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.406869 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405765    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-systemd-units\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.406869 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405776    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-host\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.406869 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405785    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-host-run-netns\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.406869 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405818    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-host-cni-bin\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.406869 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405831    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-etc-sysctl-conf\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.406869 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405847    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-sys\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.406869 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405872    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/03112039-98fd-478f-b873-2504ce8aae02-hosts-file\") pod \"node-resolver-trrlj\" (UID: \"03112039-98fd-478f-b873-2504ce8aae02\") " pod="openshift-dns/node-resolver-trrlj"
Jul 28 22:09:54.406869 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405900    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql2z9\" (UniqueName: \"kubernetes.io/projected/03112039-98fd-478f-b873-2504ce8aae02-kube-api-access-ql2z9\") pod \"node-resolver-trrlj\" (UID: \"03112039-98fd-478f-b873-2504ce8aae02\") " pod="openshift-dns/node-resolver-trrlj"
Jul 28 22:09:54.406869 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.405938    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-sys\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.406869 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.406149    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1399c32f-c1f8-4d3e-8350-b7900e68a30c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.407530 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.406970    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-etc-tuned\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.407530 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.407124    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-tmp\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.410118 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:54.410088    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 22:09:54.410118 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:54.410116    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 22:09:54.410276 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:54.410130    2551 projected.go:196] Error preparing data for projected volume kube-api-access-7v4p4 for pod openshift-network-diagnostics/network-check-target-zb7gv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:09:54.410276 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:54.410210    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4 podName:9a7c5df8-e808-4a53-9697-7dc091ca7660 nodeName:}" failed. No retries permitted until 2026-07-28 22:09:54.910191448 +0000 UTC m=+2.928157141 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7v4p4" (UniqueName: "kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4") pod "network-check-target-zb7gv" (UID: "9a7c5df8-e808-4a53-9697-7dc091ca7660") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:09:54.413214 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.413187    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nngdk\" (UniqueName: \"kubernetes.io/projected/4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87-kube-api-access-nngdk\") pod \"aws-ebs-csi-driver-node-v9njt\" (UID: \"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt"
Jul 28 22:09:54.413335 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.413196    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79htf\" (UniqueName: \"kubernetes.io/projected/9c7c65c8-313a-4e85-9c0b-4c190cf1cbea-kube-api-access-79htf\") pod \"node-ca-25fc9\" (UID: \"9c7c65c8-313a-4e85-9c0b-4c190cf1cbea\") " pod="openshift-image-registry/node-ca-25fc9"
Jul 28 22:09:54.413649 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.413591    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hx54\" (UniqueName: \"kubernetes.io/projected/e7cd9a57-9827-497b-a1fa-5c168e415621-kube-api-access-4hx54\") pod \"iptables-alerter-g6dq5\" (UID: \"e7cd9a57-9827-497b-a1fa-5c168e415621\") " pod="openshift-network-operator/iptables-alerter-g6dq5"
Jul 28 22:09:54.413783 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.413674    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqm6c\" (UniqueName: \"kubernetes.io/projected/1399c32f-c1f8-4d3e-8350-b7900e68a30c-kube-api-access-zqm6c\") pod \"multus-additional-cni-plugins-dfgvv\" (UID: \"1399c32f-c1f8-4d3e-8350-b7900e68a30c\") " pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.415911 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.415890    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6qr9\" (UniqueName: \"kubernetes.io/projected/c54b2692-1620-47a1-b04e-7b7f6b5df90a-kube-api-access-v6qr9\") pod \"network-metrics-daemon-k6pcj\" (UID: \"c54b2692-1620-47a1-b04e-7b7f6b5df90a\") " pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:09:54.420550 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.420530    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twwl4\" (UniqueName: \"kubernetes.io/projected/1077d04a-ecfd-4764-bc8c-2eaa924f7df3-kube-api-access-twwl4\") pod \"tuned-mw9qg\" (UID: \"1077d04a-ecfd-4764-bc8c-2eaa924f7df3\") " pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.461800 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.461737    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-102.ec2.internal" event={"ID":"8165253f3124970b10c0b16ca151ca09","Type":"ContainerStarted","Data":"c8d664c4bbcf6f374ff079710bb3cbe43ff46b26b46193ae355e1a8c8c5134ab"}
Jul 28 22:09:54.462941 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.462912    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-102.ec2.internal" event={"ID":"76decd8d884af4bc21a6a287671d52e8","Type":"ContainerStarted","Data":"76a8de95a9be3bf0bd472fe93a0ed93054a0067bcdb6b6e55376d2d4b5f2ddd9"}
Jul 28 22:09:54.506620 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.506576    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-run-ovn\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.506620 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.506627    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.506877 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.506670    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-multus-cni-dir\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.506877 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.506690    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-host-run-multus-certs\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.506877 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.506708    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03112039-98fd-478f-b873-2504ce8aae02-tmp-dir\") pod \"node-resolver-trrlj\" (UID: \"03112039-98fd-478f-b873-2504ce8aae02\") " pod="openshift-dns/node-resolver-trrlj"
Jul 28 22:09:54.506877 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.506710    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-run-ovn\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.506877 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.506735    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/46fd1bad-a9c6-4872-aa09-48a8d38fc005-multus-daemon-config\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.506877 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.506762    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-host-slash\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.506877 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.506778    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.506877 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.506786    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-host-run-ovn-kubernetes\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.506877 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.506822    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-host-cni-netd\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.506877 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.506833    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-host-run-ovn-kubernetes\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.506877 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.506849    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/481b2f34-6f30-4e75-986c-c7eba23f6214-env-overrides\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.506877 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.506876    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-cnibin\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.506877 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.506882    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-host-slash\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.507376 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.506915    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-multus-cni-dir\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.507376 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.506934    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-cnibin\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.507376 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.506919    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-host-cni-netd\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.507376 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.506971    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-multus-conf-dir\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.507376 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.506974    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-host-run-multus-certs\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.507376 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.506997    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-host-kubelet\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.507376 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507026    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-multus-conf-dir\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.507376 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507047    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-var-lib-openvswitch\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.507376 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507069    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-host-kubelet\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.507376 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507073    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/be77a0c4-e19b-49d2-b547-c36c3d79dd33-kubelet-config\") pod \"global-pull-secret-syncer-7cn74\" (UID: \"be77a0c4-e19b-49d2-b547-c36c3d79dd33\") " pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:09:54.507376 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507112    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-var-lib-openvswitch\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.507376 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507118    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx2tj\" (UniqueName: \"kubernetes.io/projected/46fd1bad-a9c6-4872-aa09-48a8d38fc005-kube-api-access-bx2tj\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.507376 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507149    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-run-openvswitch\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.507376 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507160    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/be77a0c4-e19b-49d2-b547-c36c3d79dd33-kubelet-config\") pod \"global-pull-secret-syncer-7cn74\" (UID: \"be77a0c4-e19b-49d2-b547-c36c3d79dd33\") " pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:09:54.507376 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507174    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-log-socket\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.507376 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507201    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/481b2f34-6f30-4e75-986c-c7eba23f6214-ovnkube-config\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.507376 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507215    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-run-openvswitch\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.507376 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507111    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03112039-98fd-478f-b873-2504ce8aae02-tmp-dir\") pod \"node-resolver-trrlj\" (UID: \"03112039-98fd-478f-b873-2504ce8aae02\") " pod="openshift-dns/node-resolver-trrlj"
Jul 28 22:09:54.508313 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507234    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/be77a0c4-e19b-49d2-b547-c36c3d79dd33-dbus\") pod \"global-pull-secret-syncer-7cn74\" (UID: \"be77a0c4-e19b-49d2-b547-c36c3d79dd33\") " pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:09:54.508313 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507241    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-log-socket\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.508313 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507274    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-run-systemd\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.508313 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507303    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/481b2f34-6f30-4e75-986c-c7eba23f6214-ovnkube-script-lib\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.508313 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507292    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/be77a0c4-e19b-49d2-b547-c36c3d79dd33-dbus\") pod \"global-pull-secret-syncer-7cn74\" (UID: \"be77a0c4-e19b-49d2-b547-c36c3d79dd33\") " pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:09:54.508313 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507333    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret\") pod \"global-pull-secret-syncer-7cn74\" (UID: \"be77a0c4-e19b-49d2-b547-c36c3d79dd33\") " pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:09:54.508313 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507362    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/934b288b-0f1f-4bd5-a697-d41516122892-agent-certs\") pod \"konnectivity-agent-sm9g2\" (UID: \"934b288b-0f1f-4bd5-a697-d41516122892\") " pod="kube-system/konnectivity-agent-sm9g2"
Jul 28 22:09:54.508313 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507364    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-run-systemd\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.508313 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507396    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-etc-openvswitch\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.508313 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507424    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-host-var-lib-kubelet\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.508313 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507438    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/46fd1bad-a9c6-4872-aa09-48a8d38fc005-multus-daemon-config\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.508313 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507452    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-systemd-units\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.508313 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:54.507468    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 22:09:54.508313 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507476    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-host-run-netns\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.508313 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507492    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-host-cni-bin\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.508313 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507507    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/03112039-98fd-478f-b873-2504ce8aae02-hosts-file\") pod \"node-resolver-trrlj\" (UID: \"03112039-98fd-478f-b873-2504ce8aae02\") " pod="openshift-dns/node-resolver-trrlj"
Jul 28 22:09:54.508313 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507508    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-host-var-lib-kubelet\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.508313 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507546    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/03112039-98fd-478f-b873-2504ce8aae02-hosts-file\") pod \"node-resolver-trrlj\" (UID: \"03112039-98fd-478f-b873-2504ce8aae02\") " pod="openshift-dns/node-resolver-trrlj"
Jul 28 22:09:54.509121 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507557    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-etc-openvswitch\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.509121 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:54.507568    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret podName:be77a0c4-e19b-49d2-b547-c36c3d79dd33 nodeName:}" failed. No retries permitted until 2026-07-28 22:09:55.007528631 +0000 UTC m=+3.025494319 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret") pod "global-pull-secret-syncer-7cn74" (UID: "be77a0c4-e19b-49d2-b547-c36c3d79dd33") : object "kube-system"/"original-pull-secret" not registered
Jul 28 22:09:54.509121 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507608    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-systemd-units\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.509121 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507616    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql2z9\" (UniqueName: \"kubernetes.io/projected/03112039-98fd-478f-b873-2504ce8aae02-kube-api-access-ql2z9\") pod \"node-resolver-trrlj\" (UID: \"03112039-98fd-478f-b873-2504ce8aae02\") " pod="openshift-dns/node-resolver-trrlj"
Jul 28 22:09:54.509121 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507649    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-host-run-netns\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.509121 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507698    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-system-cni-dir\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.509121 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507701    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-host-cni-bin\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.509121 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507650    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-system-cni-dir\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.509121 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507741    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-host-run-netns\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.509121 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507769    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-os-release\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.509121 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507794    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-host-run-k8s-cni-cncf-io\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.509121 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507785    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-host-run-netns\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.509121 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507823    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-host-var-lib-cni-bin\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.509121 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507829    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-os-release\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.509121 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507850    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-host-var-lib-cni-multus\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.509121 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507860    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-host-run-k8s-cni-cncf-io\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.509121 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507873    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-host-var-lib-cni-bin\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.509949 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507901    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-host-var-lib-cni-multus\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.509949 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507913    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-etc-kubernetes\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.509949 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507877    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-etc-kubernetes\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.509949 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507951    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-node-log\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.509949 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.507986    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/481b2f34-6f30-4e75-986c-c7eba23f6214-ovn-node-metrics-cert\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.509949 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.508014    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx6hr\" (UniqueName: \"kubernetes.io/projected/481b2f34-6f30-4e75-986c-c7eba23f6214-kube-api-access-gx6hr\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.509949 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.508019    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/481b2f34-6f30-4e75-986c-c7eba23f6214-node-log\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.509949 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.508038    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-hostroot\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.509949 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.508068    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-multus-socket-dir-parent\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.509949 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.508081    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-hostroot\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.509949 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.508095    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/934b288b-0f1f-4bd5-a697-d41516122892-konnectivity-ca\") pod \"konnectivity-agent-sm9g2\" (UID: \"934b288b-0f1f-4bd5-a697-d41516122892\") " pod="kube-system/konnectivity-agent-sm9g2"
Jul 28 22:09:54.509949 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.508121    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/46fd1bad-a9c6-4872-aa09-48a8d38fc005-cni-binary-copy\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.509949 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.508143    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/46fd1bad-a9c6-4872-aa09-48a8d38fc005-multus-socket-dir-parent\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.509949 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.508036    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/481b2f34-6f30-4e75-986c-c7eba23f6214-env-overrides\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.509949 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.508629    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/481b2f34-6f30-4e75-986c-c7eba23f6214-ovnkube-config\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.509949 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.508655    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/934b288b-0f1f-4bd5-a697-d41516122892-konnectivity-ca\") pod \"konnectivity-agent-sm9g2\" (UID: \"934b288b-0f1f-4bd5-a697-d41516122892\") " pod="kube-system/konnectivity-agent-sm9g2"
Jul 28 22:09:54.509949 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.508774    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/46fd1bad-a9c6-4872-aa09-48a8d38fc005-cni-binary-copy\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.509949 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.509021    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/481b2f34-6f30-4e75-986c-c7eba23f6214-ovnkube-script-lib\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.511017 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.510938    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/934b288b-0f1f-4bd5-a697-d41516122892-agent-certs\") pod \"konnectivity-agent-sm9g2\" (UID: \"934b288b-0f1f-4bd5-a697-d41516122892\") " pod="kube-system/konnectivity-agent-sm9g2"
Jul 28 22:09:54.511142 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.511062    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/481b2f34-6f30-4e75-986c-c7eba23f6214-ovn-node-metrics-cert\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.524199 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.523447    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx6hr\" (UniqueName: \"kubernetes.io/projected/481b2f34-6f30-4e75-986c-c7eba23f6214-kube-api-access-gx6hr\") pod \"ovnkube-node-222qs\" (UID: \"481b2f34-6f30-4e75-986c-c7eba23f6214\") " pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.524199 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.523583    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx2tj\" (UniqueName: \"kubernetes.io/projected/46fd1bad-a9c6-4872-aa09-48a8d38fc005-kube-api-access-bx2tj\") pod \"multus-mbcll\" (UID: \"46fd1bad-a9c6-4872-aa09-48a8d38fc005\") " pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.524199 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.523594    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql2z9\" (UniqueName: \"kubernetes.io/projected/03112039-98fd-478f-b873-2504ce8aae02-kube-api-access-ql2z9\") pod \"node-resolver-trrlj\" (UID: \"03112039-98fd-478f-b873-2504ce8aae02\") " pod="openshift-dns/node-resolver-trrlj"
Jul 28 22:09:54.539804 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.539772    2551 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 22:09:54.609915 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.609874    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-25fc9"
Jul 28 22:09:54.620403 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.620375    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt"
Jul 28 22:09:54.630455 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.630431    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dfgvv"
Jul 28 22:09:54.636267 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.636242    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mw9qg"
Jul 28 22:09:54.646105 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.646080    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-g6dq5"
Jul 28 22:09:54.662091 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.662070    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mbcll"
Jul 28 22:09:54.670256 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.670229    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:09:54.677237 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.677217    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sm9g2"
Jul 28 22:09:54.683944 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.683923    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-trrlj"
Jul 28 22:09:54.912080 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.911988    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v4p4\" (UniqueName: \"kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4\") pod \"network-check-target-zb7gv\" (UID: \"9a7c5df8-e808-4a53-9697-7dc091ca7660\") " pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:09:54.912080 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:54.912043    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs\") pod \"network-metrics-daemon-k6pcj\" (UID: \"c54b2692-1620-47a1-b04e-7b7f6b5df90a\") " pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:09:54.912308 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:54.912174    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 22:09:54.912308 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:54.912201    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 22:09:54.912308 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:54.912201    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:09:54.912308 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:54.912213    2551 projected.go:196] Error preparing data for projected volume kube-api-access-7v4p4 for pod openshift-network-diagnostics/network-check-target-zb7gv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:09:54.912308 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:54.912278    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs podName:c54b2692-1620-47a1-b04e-7b7f6b5df90a nodeName:}" failed. No retries permitted until 2026-07-28 22:09:55.912256627 +0000 UTC m=+3.930222323 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs") pod "network-metrics-daemon-k6pcj" (UID: "c54b2692-1620-47a1-b04e-7b7f6b5df90a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:09:54.912308 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:54.912302    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4 podName:9a7c5df8-e808-4a53-9697-7dc091ca7660 nodeName:}" failed. No retries permitted until 2026-07-28 22:09:55.912290934 +0000 UTC m=+3.930256612 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7v4p4" (UniqueName: "kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4") pod "network-check-target-zb7gv" (UID: "9a7c5df8-e808-4a53-9697-7dc091ca7660") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:09:55.012471 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:55.012427    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret\") pod \"global-pull-secret-syncer-7cn74\" (UID: \"be77a0c4-e19b-49d2-b547-c36c3d79dd33\") " pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:09:55.012655 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:55.012613    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 22:09:55.012700 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:55.012691    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret podName:be77a0c4-e19b-49d2-b547-c36c3d79dd33 nodeName:}" failed. No retries permitted until 2026-07-28 22:09:56.012674581 +0000 UTC m=+4.030640259 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret") pod "global-pull-secret-syncer-7cn74" (UID: "be77a0c4-e19b-49d2-b547-c36c3d79dd33") : object "kube-system"/"original-pull-secret" not registered
Jul 28 22:09:55.355905 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:55.355787    2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 22:04:54 +0000 UTC" deadline="2028-03-07 18:49:11.392636538 +0000 UTC"
Jul 28 22:09:55.355905 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:55.355857    2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14108h39m16.036784606s"
Jul 28 22:09:55.456886 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:55.456861    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:09:55.456998 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:55.456868    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:09:55.456998 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:55.456972    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zb7gv" podUID="9a7c5df8-e808-4a53-9697-7dc091ca7660"
Jul 28 22:09:55.457114 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:55.457084    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7cn74" podUID="be77a0c4-e19b-49d2-b547-c36c3d79dd33"
Jul 28 22:09:55.470306 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:55.470254    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1077d04a_ecfd_4764_bc8c_2eaa924f7df3.slice/crio-682774124e5a97db169226ae317e419c9888623d75bc560acac0a6685cf4491e WatchSource:0}: Error finding container 682774124e5a97db169226ae317e419c9888623d75bc560acac0a6685cf4491e: Status 404 returned error can't find the container with id 682774124e5a97db169226ae317e419c9888623d75bc560acac0a6685cf4491e
Jul 28 22:09:55.492233 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:55.492191    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c7c65c8_313a_4e85_9c0b_4c190cf1cbea.slice/crio-5d877c42ab59365f1064b08de3e9c32daa366968d9ee0b96fee06796c5d20c60 WatchSource:0}: Error finding container 5d877c42ab59365f1064b08de3e9c32daa366968d9ee0b96fee06796c5d20c60: Status 404 returned error can't find the container with id 5d877c42ab59365f1064b08de3e9c32daa366968d9ee0b96fee06796c5d20c60
Jul 28 22:09:55.495049 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:55.495026    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03112039_98fd_478f_b873_2504ce8aae02.slice/crio-7a36f43a333b47a3798bab3f51980f1a0c39ac2dfa1a32365c741367fb5bfb9c WatchSource:0}: Error finding container 7a36f43a333b47a3798bab3f51980f1a0c39ac2dfa1a32365c741367fb5bfb9c: Status 404 returned error can't find the container with id 7a36f43a333b47a3798bab3f51980f1a0c39ac2dfa1a32365c741367fb5bfb9c
Jul 28 22:09:55.498980 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:55.498955    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1399c32f_c1f8_4d3e_8350_b7900e68a30c.slice/crio-b7c0020e0cff595ec373640ff77bca793de845df2599bed6a97042d5f7b91689 WatchSource:0}: Error finding container b7c0020e0cff595ec373640ff77bca793de845df2599bed6a97042d5f7b91689: Status 404 returned error can't find the container with id b7c0020e0cff595ec373640ff77bca793de845df2599bed6a97042d5f7b91689
Jul 28 22:09:55.500405 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:55.500378    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod481b2f34_6f30_4e75_986c_c7eba23f6214.slice/crio-de46bb4209e472bd94c79d7fcadf9953b693a11970f992f9e234483bfdb2d046 WatchSource:0}: Error finding container de46bb4209e472bd94c79d7fcadf9953b693a11970f992f9e234483bfdb2d046: Status 404 returned error can't find the container with id de46bb4209e472bd94c79d7fcadf9953b693a11970f992f9e234483bfdb2d046
Jul 28 22:09:55.502190 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:55.502070    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4df53b69_f4b7_4cc9_beb6_ef1e20e2ff87.slice/crio-4168505a361be36c79de01a94f102302679a8c2a001c124cdc7ab58f2f1b19a7 WatchSource:0}: Error finding container 4168505a361be36c79de01a94f102302679a8c2a001c124cdc7ab58f2f1b19a7: Status 404 returned error can't find the container with id 4168505a361be36c79de01a94f102302679a8c2a001c124cdc7ab58f2f1b19a7
Jul 28 22:09:55.503381 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:55.503354    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7cd9a57_9827_497b_a1fa_5c168e415621.slice/crio-1cf6f43fc065bd27a0cd1b237109f8b0aa20e1d9c0d72c7a2500ede37764d240 WatchSource:0}: Error finding container 1cf6f43fc065bd27a0cd1b237109f8b0aa20e1d9c0d72c7a2500ede37764d240: Status 404 returned error can't find the container with id 1cf6f43fc065bd27a0cd1b237109f8b0aa20e1d9c0d72c7a2500ede37764d240
Jul 28 22:09:55.505136 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:55.505065    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46fd1bad_a9c6_4872_aa09_48a8d38fc005.slice/crio-706218b4279f5a4116d5e3fff97ece6d8f529b5e62bfa730d74725bc0531b648 WatchSource:0}: Error finding container 706218b4279f5a4116d5e3fff97ece6d8f529b5e62bfa730d74725bc0531b648: Status 404 returned error can't find the container with id 706218b4279f5a4116d5e3fff97ece6d8f529b5e62bfa730d74725bc0531b648
Jul 28 22:09:55.507013 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:09:55.506931    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod934b288b_0f1f_4bd5_a697_d41516122892.slice/crio-98f23cdb18d3bcde36b06197459a9e246aa84f19c39cf75345228848b1a3a8af WatchSource:0}: Error finding container 98f23cdb18d3bcde36b06197459a9e246aa84f19c39cf75345228848b1a3a8af: Status 404 returned error can't find the container with id 98f23cdb18d3bcde36b06197459a9e246aa84f19c39cf75345228848b1a3a8af
Jul 28 22:09:55.920996 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:55.919951    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v4p4\" (UniqueName: \"kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4\") pod \"network-check-target-zb7gv\" (UID: \"9a7c5df8-e808-4a53-9697-7dc091ca7660\") " pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:09:55.920996 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:55.920266    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs\") pod \"network-metrics-daemon-k6pcj\" (UID: \"c54b2692-1620-47a1-b04e-7b7f6b5df90a\") " pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:09:55.920996 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:55.920202    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 22:09:55.920996 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:55.920465    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 22:09:55.920996 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:55.920481    2551 projected.go:196] Error preparing data for projected volume kube-api-access-7v4p4 for pod openshift-network-diagnostics/network-check-target-zb7gv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:09:55.920996 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:55.920545    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4 podName:9a7c5df8-e808-4a53-9697-7dc091ca7660 nodeName:}" failed. No retries permitted until 2026-07-28 22:09:57.920524905 +0000 UTC m=+5.938490594 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7v4p4" (UniqueName: "kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4") pod "network-check-target-zb7gv" (UID: "9a7c5df8-e808-4a53-9697-7dc091ca7660") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:09:55.920996 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:55.920438    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:09:55.920996 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:55.920959    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs podName:c54b2692-1620-47a1-b04e-7b7f6b5df90a nodeName:}" failed. No retries permitted until 2026-07-28 22:09:57.920940957 +0000 UTC m=+5.938906632 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs") pod "network-metrics-daemon-k6pcj" (UID: "c54b2692-1620-47a1-b04e-7b7f6b5df90a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:09:56.021505 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:56.020754    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret\") pod \"global-pull-secret-syncer-7cn74\" (UID: \"be77a0c4-e19b-49d2-b547-c36c3d79dd33\") " pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:09:56.021505 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:56.020913    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 22:09:56.021505 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:56.020983    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret podName:be77a0c4-e19b-49d2-b547-c36c3d79dd33 nodeName:}" failed. No retries permitted until 2026-07-28 22:09:58.020963654 +0000 UTC m=+6.038929347 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret") pod "global-pull-secret-syncer-7cn74" (UID: "be77a0c4-e19b-49d2-b547-c36c3d79dd33") : object "kube-system"/"original-pull-secret" not registered
Jul 28 22:09:56.356994 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:56.356897    2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 22:04:54 +0000 UTC" deadline="2028-01-13 10:41:26.518330963 +0000 UTC"
Jul 28 22:09:56.356994 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:56.356943    2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12804h31m30.161391924s"
Jul 28 22:09:56.459710 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:56.459664    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:09:56.459915 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:56.459826    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6pcj" podUID="c54b2692-1620-47a1-b04e-7b7f6b5df90a"
Jul 28 22:09:56.477925 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:56.477880    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sm9g2" event={"ID":"934b288b-0f1f-4bd5-a697-d41516122892","Type":"ContainerStarted","Data":"98f23cdb18d3bcde36b06197459a9e246aa84f19c39cf75345228848b1a3a8af"}
Jul 28 22:09:56.484722 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:56.484683    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mbcll" event={"ID":"46fd1bad-a9c6-4872-aa09-48a8d38fc005","Type":"ContainerStarted","Data":"706218b4279f5a4116d5e3fff97ece6d8f529b5e62bfa730d74725bc0531b648"}
Jul 28 22:09:56.512098 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:56.512051    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g6dq5" event={"ID":"e7cd9a57-9827-497b-a1fa-5c168e415621","Type":"ContainerStarted","Data":"1cf6f43fc065bd27a0cd1b237109f8b0aa20e1d9c0d72c7a2500ede37764d240"}
Jul 28 22:09:56.529047 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:56.528623    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt" event={"ID":"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87","Type":"ContainerStarted","Data":"4168505a361be36c79de01a94f102302679a8c2a001c124cdc7ab58f2f1b19a7"}
Jul 28 22:09:56.537980 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:56.537944    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-222qs" event={"ID":"481b2f34-6f30-4e75-986c-c7eba23f6214","Type":"ContainerStarted","Data":"de46bb4209e472bd94c79d7fcadf9953b693a11970f992f9e234483bfdb2d046"}
Jul 28 22:09:56.546565 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:56.546527    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-trrlj" event={"ID":"03112039-98fd-478f-b873-2504ce8aae02","Type":"ContainerStarted","Data":"7a36f43a333b47a3798bab3f51980f1a0c39ac2dfa1a32365c741367fb5bfb9c"}
Jul 28 22:09:56.553440 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:56.553407    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mw9qg" event={"ID":"1077d04a-ecfd-4764-bc8c-2eaa924f7df3","Type":"ContainerStarted","Data":"682774124e5a97db169226ae317e419c9888623d75bc560acac0a6685cf4491e"}
Jul 28 22:09:56.575025 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:56.574988    2551 generic.go:356] "Generic (PLEG): container finished" podID="8165253f3124970b10c0b16ca151ca09" containerID="de5420d1937825879629cf97dcad498d42d26f00a562b61bf444d37b7f56f8bd" exitCode=0
Jul 28 22:09:56.575161 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:56.575086    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-102.ec2.internal" event={"ID":"8165253f3124970b10c0b16ca151ca09","Type":"ContainerDied","Data":"de5420d1937825879629cf97dcad498d42d26f00a562b61bf444d37b7f56f8bd"}
Jul 28 22:09:56.610334 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:56.610242    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-102.ec2.internal" event={"ID":"76decd8d884af4bc21a6a287671d52e8","Type":"ContainerStarted","Data":"ff0dc2d5badfd52055ae089a1935fae0969827876788f1f68184a345fa493c44"}
Jul 28 22:09:56.632796 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:56.632759    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dfgvv" event={"ID":"1399c32f-c1f8-4d3e-8350-b7900e68a30c","Type":"ContainerStarted","Data":"b7c0020e0cff595ec373640ff77bca793de845df2599bed6a97042d5f7b91689"}
Jul 28 22:09:56.651301 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:56.644883    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-25fc9" event={"ID":"9c7c65c8-313a-4e85-9c0b-4c190cf1cbea","Type":"ContainerStarted","Data":"5d877c42ab59365f1064b08de3e9c32daa366968d9ee0b96fee06796c5d20c60"}
Jul 28 22:09:57.457843 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:57.457162    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:09:57.457843 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:57.457303    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7cn74" podUID="be77a0c4-e19b-49d2-b547-c36c3d79dd33"
Jul 28 22:09:57.457843 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:57.457381    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:09:57.457843 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:57.457456    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zb7gv" podUID="9a7c5df8-e808-4a53-9697-7dc091ca7660"
Jul 28 22:09:57.672760 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:57.672577    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-102.ec2.internal" event={"ID":"8165253f3124970b10c0b16ca151ca09","Type":"ContainerStarted","Data":"86fe607dbdce7bdd3eae64653b4cd203a11026c2c68628ff10908126f358e58c"}
Jul 28 22:09:57.686462 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:57.686404    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-102.ec2.internal" podStartSLOduration=3.6863820560000002 podStartE2EDuration="3.686382056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:09:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:09:56.629973523 +0000 UTC m=+4.647939221" watchObservedRunningTime="2026-07-28 22:09:57.686382056 +0000 UTC m=+5.704347746"
Jul 28 22:09:57.938853 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:57.938760    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v4p4\" (UniqueName: \"kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4\") pod \"network-check-target-zb7gv\" (UID: \"9a7c5df8-e808-4a53-9697-7dc091ca7660\") " pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:09:57.938853 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:57.938828    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs\") pod \"network-metrics-daemon-k6pcj\" (UID: \"c54b2692-1620-47a1-b04e-7b7f6b5df90a\") " pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:09:57.939093 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:57.938993    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:09:57.939093 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:57.939062    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs podName:c54b2692-1620-47a1-b04e-7b7f6b5df90a nodeName:}" failed. No retries permitted until 2026-07-28 22:10:01.939041622 +0000 UTC m=+9.957007312 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs") pod "network-metrics-daemon-k6pcj" (UID: "c54b2692-1620-47a1-b04e-7b7f6b5df90a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:09:57.939552 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:57.939525    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 22:09:57.939552 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:57.939551    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 22:09:57.939729 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:57.939567    2551 projected.go:196] Error preparing data for projected volume kube-api-access-7v4p4 for pod openshift-network-diagnostics/network-check-target-zb7gv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:09:57.939729 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:57.939634    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4 podName:9a7c5df8-e808-4a53-9697-7dc091ca7660 nodeName:}" failed. No retries permitted until 2026-07-28 22:10:01.939602294 +0000 UTC m=+9.957567983 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7v4p4" (UniqueName: "kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4") pod "network-check-target-zb7gv" (UID: "9a7c5df8-e808-4a53-9697-7dc091ca7660") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:09:58.039950 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:58.039911    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret\") pod \"global-pull-secret-syncer-7cn74\" (UID: \"be77a0c4-e19b-49d2-b547-c36c3d79dd33\") " pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:09:58.040167 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:58.040071    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 22:09:58.040167 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:58.040142    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret podName:be77a0c4-e19b-49d2-b547-c36c3d79dd33 nodeName:}" failed. No retries permitted until 2026-07-28 22:10:02.040122895 +0000 UTC m=+10.058088573 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret") pod "global-pull-secret-syncer-7cn74" (UID: "be77a0c4-e19b-49d2-b547-c36c3d79dd33") : object "kube-system"/"original-pull-secret" not registered
Jul 28 22:09:58.459528 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:58.459491    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:09:58.460028 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:58.459685    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6pcj" podUID="c54b2692-1620-47a1-b04e-7b7f6b5df90a"
Jul 28 22:09:59.457096 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:59.457064    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:09:59.457299 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:09:59.457107    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:09:59.457299 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:59.457206    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7cn74" podUID="be77a0c4-e19b-49d2-b547-c36c3d79dd33"
Jul 28 22:09:59.457442 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:09:59.457288    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zb7gv" podUID="9a7c5df8-e808-4a53-9697-7dc091ca7660"
Jul 28 22:10:00.456943 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:00.456909    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:10:00.457416 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:00.457065    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6pcj" podUID="c54b2692-1620-47a1-b04e-7b7f6b5df90a"
Jul 28 22:10:01.457281 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:01.457243    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:10:01.457747 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:01.457308    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:10:01.457747 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:01.457392    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7cn74" podUID="be77a0c4-e19b-49d2-b547-c36c3d79dd33"
Jul 28 22:10:01.457747 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:01.457543    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zb7gv" podUID="9a7c5df8-e808-4a53-9697-7dc091ca7660"
Jul 28 22:10:01.975736 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:01.975005    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs\") pod \"network-metrics-daemon-k6pcj\" (UID: \"c54b2692-1620-47a1-b04e-7b7f6b5df90a\") " pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:10:01.975736 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:01.975082    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v4p4\" (UniqueName: \"kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4\") pod \"network-check-target-zb7gv\" (UID: \"9a7c5df8-e808-4a53-9697-7dc091ca7660\") " pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:10:01.975736 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:01.975214    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 22:10:01.975736 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:01.975236    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 22:10:01.975736 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:01.975247    2551 projected.go:196] Error preparing data for projected volume kube-api-access-7v4p4 for pod openshift-network-diagnostics/network-check-target-zb7gv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:10:01.975736 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:01.975300    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4 podName:9a7c5df8-e808-4a53-9697-7dc091ca7660 nodeName:}" failed. No retries permitted until 2026-07-28 22:10:09.97528014 +0000 UTC m=+17.993245812 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7v4p4" (UniqueName: "kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4") pod "network-check-target-zb7gv" (UID: "9a7c5df8-e808-4a53-9697-7dc091ca7660") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:10:01.975736 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:01.975646    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:10:01.975736 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:01.975691    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs podName:c54b2692-1620-47a1-b04e-7b7f6b5df90a nodeName:}" failed. No retries permitted until 2026-07-28 22:10:09.975679283 +0000 UTC m=+17.993644955 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs") pod "network-metrics-daemon-k6pcj" (UID: "c54b2692-1620-47a1-b04e-7b7f6b5df90a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:10:02.075457 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:02.075417    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret\") pod \"global-pull-secret-syncer-7cn74\" (UID: \"be77a0c4-e19b-49d2-b547-c36c3d79dd33\") " pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:10:02.075661 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:02.075610    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 22:10:02.075744 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:02.075685    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret podName:be77a0c4-e19b-49d2-b547-c36c3d79dd33 nodeName:}" failed. No retries permitted until 2026-07-28 22:10:10.07566902 +0000 UTC m=+18.093634702 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret") pod "global-pull-secret-syncer-7cn74" (UID: "be77a0c4-e19b-49d2-b547-c36c3d79dd33") : object "kube-system"/"original-pull-secret" not registered
Jul 28 22:10:02.457683 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:02.457598    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:10:02.458127 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:02.457737    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6pcj" podUID="c54b2692-1620-47a1-b04e-7b7f6b5df90a"
Jul 28 22:10:03.456893 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:03.456856    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:10:03.457061 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:03.456991    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zb7gv" podUID="9a7c5df8-e808-4a53-9697-7dc091ca7660"
Jul 28 22:10:03.457136 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:03.457068    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:10:03.457213 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:03.457188    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7cn74" podUID="be77a0c4-e19b-49d2-b547-c36c3d79dd33"
Jul 28 22:10:04.456579 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:04.456545    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:10:04.457016 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:04.456678    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6pcj" podUID="c54b2692-1620-47a1-b04e-7b7f6b5df90a"
Jul 28 22:10:05.457300 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:05.457263    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:10:05.457671 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:05.457262    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:10:05.457671 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:05.457378    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zb7gv" podUID="9a7c5df8-e808-4a53-9697-7dc091ca7660"
Jul 28 22:10:05.457671 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:05.457440    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7cn74" podUID="be77a0c4-e19b-49d2-b547-c36c3d79dd33"
Jul 28 22:10:06.456633 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:06.456600    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:10:06.456784 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:06.456748    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6pcj" podUID="c54b2692-1620-47a1-b04e-7b7f6b5df90a"
Jul 28 22:10:07.457241 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:07.457201    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:10:07.457764 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:07.457255    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:10:07.457764 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:07.457351    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7cn74" podUID="be77a0c4-e19b-49d2-b547-c36c3d79dd33"
Jul 28 22:10:07.457764 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:07.457683    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zb7gv" podUID="9a7c5df8-e808-4a53-9697-7dc091ca7660"
Jul 28 22:10:08.457294 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:08.457253    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:10:08.457724 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:08.457390    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6pcj" podUID="c54b2692-1620-47a1-b04e-7b7f6b5df90a"
Jul 28 22:10:09.456334 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:09.456301    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:10:09.456587 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:09.456301    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:10:09.456587 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:09.456421    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zb7gv" podUID="9a7c5df8-e808-4a53-9697-7dc091ca7660"
Jul 28 22:10:09.456587 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:09.456538    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7cn74" podUID="be77a0c4-e19b-49d2-b547-c36c3d79dd33"
Jul 28 22:10:10.040196 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:10.040152    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs\") pod \"network-metrics-daemon-k6pcj\" (UID: \"c54b2692-1620-47a1-b04e-7b7f6b5df90a\") " pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:10:10.040682 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:10.040250    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v4p4\" (UniqueName: \"kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4\") pod \"network-check-target-zb7gv\" (UID: \"9a7c5df8-e808-4a53-9697-7dc091ca7660\") " pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:10:10.040682 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:10.040335    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:10:10.040682 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:10.040371    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 22:10:10.040682 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:10.040393    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 22:10:10.040682 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:10.040407    2551 projected.go:196] Error preparing data for projected volume kube-api-access-7v4p4 for pod openshift-network-diagnostics/network-check-target-zb7gv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:10:10.040682 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:10.040419    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs podName:c54b2692-1620-47a1-b04e-7b7f6b5df90a nodeName:}" failed. No retries permitted until 2026-07-28 22:10:26.040396631 +0000 UTC m=+34.058362319 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs") pod "network-metrics-daemon-k6pcj" (UID: "c54b2692-1620-47a1-b04e-7b7f6b5df90a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:10:10.040682 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:10.040449    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4 podName:9a7c5df8-e808-4a53-9697-7dc091ca7660 nodeName:}" failed. No retries permitted until 2026-07-28 22:10:26.040437448 +0000 UTC m=+34.058403121 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7v4p4" (UniqueName: "kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4") pod "network-check-target-zb7gv" (UID: "9a7c5df8-e808-4a53-9697-7dc091ca7660") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:10:10.141466 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:10.141425    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret\") pod \"global-pull-secret-syncer-7cn74\" (UID: \"be77a0c4-e19b-49d2-b547-c36c3d79dd33\") " pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:10:10.141640 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:10.141575    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 22:10:10.141720 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:10.141645    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret podName:be77a0c4-e19b-49d2-b547-c36c3d79dd33 nodeName:}" failed. No retries permitted until 2026-07-28 22:10:26.141627322 +0000 UTC m=+34.159592994 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret") pod "global-pull-secret-syncer-7cn74" (UID: "be77a0c4-e19b-49d2-b547-c36c3d79dd33") : object "kube-system"/"original-pull-secret" not registered
Jul 28 22:10:10.456942 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:10.456852    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:10:10.457102 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:10.456983    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6pcj" podUID="c54b2692-1620-47a1-b04e-7b7f6b5df90a"
Jul 28 22:10:11.456452 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:11.456407    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:10:11.456452 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:11.456444    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:10:11.456918 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:11.456553    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7cn74" podUID="be77a0c4-e19b-49d2-b547-c36c3d79dd33"
Jul 28 22:10:11.456918 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:11.456691    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zb7gv" podUID="9a7c5df8-e808-4a53-9697-7dc091ca7660"
Jul 28 22:10:12.457754 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:12.457720    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:10:12.458193 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:12.457832    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6pcj" podUID="c54b2692-1620-47a1-b04e-7b7f6b5df90a"
Jul 28 22:10:13.456995 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:13.456959    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:10:13.457168 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:13.456959    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:10:13.457168 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:13.457084    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7cn74" podUID="be77a0c4-e19b-49d2-b547-c36c3d79dd33"
Jul 28 22:10:13.457302 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:13.457185    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zb7gv" podUID="9a7c5df8-e808-4a53-9697-7dc091ca7660"
Jul 28 22:10:14.456862 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:14.456585    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:10:14.456862 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:14.456718    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6pcj" podUID="c54b2692-1620-47a1-b04e-7b7f6b5df90a"
Jul 28 22:10:14.705382 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:14.705149    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sm9g2" event={"ID":"934b288b-0f1f-4bd5-a697-d41516122892","Type":"ContainerStarted","Data":"37fa8fbe25c3b038205f2bf7a0afca1ad4b64f1fe48f39643da2bfc21f372c1a"}
Jul 28 22:10:14.706850 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:14.706817    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mbcll" event={"ID":"46fd1bad-a9c6-4872-aa09-48a8d38fc005","Type":"ContainerStarted","Data":"d65a69b048d85b6d1c3e3a29b24f369fd5bdcc021157f08b806b9c6494263160"}
Jul 28 22:10:14.708415 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:14.708392    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt" event={"ID":"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87","Type":"ContainerStarted","Data":"ba5bff68fb16ae20788434bacd34d9514eedd53cf93074a337a0b5ec7c9b3a0d"}
Jul 28 22:10:14.709860 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:14.709826    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-trrlj" event={"ID":"03112039-98fd-478f-b873-2504ce8aae02","Type":"ContainerStarted","Data":"6b414c4e59cee95e858d594d1fc7b16ab1fd53900f47596fb156670106d8a80f"}
Jul 28 22:10:14.711309 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:14.711285    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mw9qg" event={"ID":"1077d04a-ecfd-4764-bc8c-2eaa924f7df3","Type":"ContainerStarted","Data":"1cde036247b31a3f1c3cff59db47d5cfbdfa3139c174d0a72edb0cd0ec4842ae"}
Jul 28 22:10:14.712925 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:14.712904    2551 generic.go:356] "Generic (PLEG): container finished" podID="1399c32f-c1f8-4d3e-8350-b7900e68a30c" containerID="efe39b248e3221b32cf9b5addf9753d2c4f93cad890ce005a34ba4063122845c" exitCode=0
Jul 28 22:10:14.713015 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:14.712985    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dfgvv" event={"ID":"1399c32f-c1f8-4d3e-8350-b7900e68a30c","Type":"ContainerDied","Data":"efe39b248e3221b32cf9b5addf9753d2c4f93cad890ce005a34ba4063122845c"}
Jul 28 22:10:14.714431 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:14.714403    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-25fc9" event={"ID":"9c7c65c8-313a-4e85-9c0b-4c190cf1cbea","Type":"ContainerStarted","Data":"81992b18e8cabd72da685d79347a354cedf18e3c898fc7d6127cfee7385ada10"}
Jul 28 22:10:14.719745 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:14.719706    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-102.ec2.internal" podStartSLOduration=20.719693376 podStartE2EDuration="20.719693376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:09:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:09:57.687848145 +0000 UTC m=+5.705813856" watchObservedRunningTime="2026-07-28 22:10:14.719693376 +0000 UTC m=+22.737659069"
Jul 28 22:10:14.732208 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:14.732171    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-sm9g2" podStartSLOduration=2.899218437 podStartE2EDuration="21.732157036s" totalImagesPullingTime="18.832938599s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:09:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:14.71920929 +0000 UTC m=+22.737174978" watchObservedRunningTime="2026-07-28 22:10:14.732157036 +0000 UTC m=+22.750122730"
Jul 28 22:10:14.732296 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:14.732240    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-trrlj" podStartSLOduration=2.84936087 podStartE2EDuration="21.732235933s" totalImagesPullingTime="18.882875063s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:09:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:14.732051591 +0000 UTC m=+22.750017296" watchObservedRunningTime="2026-07-28 22:10:14.732235933 +0000 UTC m=+22.750201627"
Jul 28 22:10:14.766340 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:14.766292    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mw9qg" podStartSLOduration=2.877845581 podStartE2EDuration="21.766276026s" totalImagesPullingTime="18.888430445s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:09:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:14.766159665 +0000 UTC m=+22.784125359" watchObservedRunningTime="2026-07-28 22:10:14.766276026 +0000 UTC m=+22.784241719"
Jul 28 22:10:14.788811 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:14.788772    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mbcll" podStartSLOduration=2.909119986 podStartE2EDuration="21.788757508s" totalImagesPullingTime="18.879637522s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:09:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:14.788108908 +0000 UTC m=+22.806074603" watchObservedRunningTime="2026-07-28 22:10:14.788757508 +0000 UTC m=+22.806723201"
Jul 28 22:10:14.803738 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:14.803689    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-25fc9" podStartSLOduration=2.9175648560000003 podStartE2EDuration="21.803671914s" totalImagesPullingTime="18.886107058s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:09:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:14.803619341 +0000 UTC m=+22.821585036" watchObservedRunningTime="2026-07-28 22:10:14.803671914 +0000 UTC m=+22.821637609"
Jul 28 22:10:15.141507 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:15.141267    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-sm9g2"
Jul 28 22:10:15.142088 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:15.142070    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-sm9g2"
Jul 28 22:10:15.456511 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:15.456478    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:10:15.456718 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:15.456491    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:10:15.456791 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:15.456628    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7cn74" podUID="be77a0c4-e19b-49d2-b547-c36c3d79dd33"
Jul 28 22:10:15.456791 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:15.456692    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zb7gv" podUID="9a7c5df8-e808-4a53-9697-7dc091ca7660"
Jul 28 22:10:15.720666 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:15.720638    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:10:15.721520 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:15.721061    2551 generic.go:356] "Generic (PLEG): container finished" podID="481b2f34-6f30-4e75-986c-c7eba23f6214" containerID="6a5c99844515672290360ea47427f3c90c53a012cee60f07c9676b01c4902b50" exitCode=1
Jul 28 22:10:15.721520 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:15.721212    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-222qs" event={"ID":"481b2f34-6f30-4e75-986c-c7eba23f6214","Type":"ContainerStarted","Data":"4b34b013ee7e31010f83887d218316dd913d62a2ebf87997338f8b9c255d2bb6"}
Jul 28 22:10:15.721520 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:15.721487    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-sm9g2"
Jul 28 22:10:15.721703 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:15.721542    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-222qs" event={"ID":"481b2f34-6f30-4e75-986c-c7eba23f6214","Type":"ContainerStarted","Data":"53ecc90360c6310cb2261db8cab0d0b996d690d303740bc81ab7040cc4ee8a44"}
Jul 28 22:10:15.721703 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:15.721599    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-222qs" event={"ID":"481b2f34-6f30-4e75-986c-c7eba23f6214","Type":"ContainerStarted","Data":"b7a91c8b7fdb999a7bf0985e613e716decd4c4c90450234a566a987b7253d948"}
Jul 28 22:10:15.721703 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:15.721651    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-222qs" event={"ID":"481b2f34-6f30-4e75-986c-c7eba23f6214","Type":"ContainerStarted","Data":"3394c059093aa7134d0ddd7532ad1e591986b883a9e97adb0a8a166c0e8613bb"}
Jul 28 22:10:15.721703 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:15.721685    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-222qs" event={"ID":"481b2f34-6f30-4e75-986c-c7eba23f6214","Type":"ContainerDied","Data":"6a5c99844515672290360ea47427f3c90c53a012cee60f07c9676b01c4902b50"}
Jul 28 22:10:15.721893 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:15.721716    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-222qs" event={"ID":"481b2f34-6f30-4e75-986c-c7eba23f6214","Type":"ContainerStarted","Data":"a17dbf13c0e224f3e00f661f62e660f627eeb68dd085a684288acebac3997015"}
Jul 28 22:10:15.722127 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:15.722109    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-sm9g2"
Jul 28 22:10:15.919786 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:15.919757    2551 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 22:10:16.370860 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:16.370742    2551 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T22:10:15.919777697Z","UUID":"c2a2e473-fa01-4df7-a4b9-131425d7ec1c","Handler":null,"Name":"","Endpoint":""}
Jul 28 22:10:16.372431 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:16.372406    2551 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 22:10:16.372582 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:16.372453    2551 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 22:10:16.456550 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:16.456474    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:10:16.456723 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:16.456633    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6pcj" podUID="c54b2692-1620-47a1-b04e-7b7f6b5df90a"
Jul 28 22:10:16.726261 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:16.725977    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-g6dq5" event={"ID":"e7cd9a57-9827-497b-a1fa-5c168e415621","Type":"ContainerStarted","Data":"602e11bda86c339aaee6112d59b5349247c4acf0cb50f86cb1ae238798aa98e0"}
Jul 28 22:10:16.728605 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:16.728578    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt" event={"ID":"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87","Type":"ContainerStarted","Data":"21adf525094789f4947859874c82ff64547a08530acc64c4cbb3ecc699ea2027"}
Jul 28 22:10:16.741068 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:16.741015    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-g6dq5" podStartSLOduration=4.861762477 podStartE2EDuration="23.74099827s" totalImagesPullingTime="18.879235793s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:09:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:16.740540217 +0000 UTC m=+24.758505908" watchObservedRunningTime="2026-07-28 22:10:16.74099827 +0000 UTC m=+24.758963965"
Jul 28 22:10:17.457111 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:17.457019    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:10:17.457264 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:17.457155    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zb7gv" podUID="9a7c5df8-e808-4a53-9697-7dc091ca7660"
Jul 28 22:10:17.457264 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:17.457210    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:10:17.457368 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:17.457327    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7cn74" podUID="be77a0c4-e19b-49d2-b547-c36c3d79dd33"
Jul 28 22:10:17.733439 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:17.733354    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt" event={"ID":"4df53b69-f4b7-4cc9-beb6-ef1e20e2ff87","Type":"ContainerStarted","Data":"7b7de2a07022ffeb4739df635df1aff9aa66900f872d60b62be1af3791b3f793"}
Jul 28 22:10:17.751100 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:17.751053    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v9njt" podStartSLOduration=3.684564346 podStartE2EDuration="24.751032731s" totalImagesPullingTime="21.066468385s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:09:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:17.750434432 +0000 UTC m=+25.768400125" watchObservedRunningTime="2026-07-28 22:10:17.751032731 +0000 UTC m=+25.768998432"
Jul 28 22:10:18.456950 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:18.456771    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:10:18.457122 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:18.457066    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6pcj" podUID="c54b2692-1620-47a1-b04e-7b7f6b5df90a"
Jul 28 22:10:19.456347 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:19.456310    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:10:19.456347 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:19.456330    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:10:19.456911 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:19.456429    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7cn74" podUID="be77a0c4-e19b-49d2-b547-c36c3d79dd33"
Jul 28 22:10:19.456911 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:19.456555    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zb7gv" podUID="9a7c5df8-e808-4a53-9697-7dc091ca7660"
Jul 28 22:10:19.739150 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:19.739066    2551 generic.go:356] "Generic (PLEG): container finished" podID="1399c32f-c1f8-4d3e-8350-b7900e68a30c" containerID="1e670122d29cab13d83af9437f37136162208e5295e3b9da848297ecb66f62e1" exitCode=0
Jul 28 22:10:19.739150 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:19.739140    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dfgvv" event={"ID":"1399c32f-c1f8-4d3e-8350-b7900e68a30c","Type":"ContainerDied","Data":"1e670122d29cab13d83af9437f37136162208e5295e3b9da848297ecb66f62e1"}
Jul 28 22:10:19.742361 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:19.742343    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:10:19.742738 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:19.742717    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-222qs" event={"ID":"481b2f34-6f30-4e75-986c-c7eba23f6214","Type":"ContainerStarted","Data":"3b178b575a2a9ac71bee4d705fb5704e66355c84d62e2923d9f6f2f6069d2e38"}
Jul 28 22:10:20.456760 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:20.456735    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:10:20.457222 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:20.456843    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6pcj" podUID="c54b2692-1620-47a1-b04e-7b7f6b5df90a"
Jul 28 22:10:20.747208 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:20.747175    2551 generic.go:356] "Generic (PLEG): container finished" podID="1399c32f-c1f8-4d3e-8350-b7900e68a30c" containerID="deadbafc2f7dcfd9bbda6d8f168980d4e271f43ac37a30c634330ac657bae27c" exitCode=0
Jul 28 22:10:20.747341 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:20.747216    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dfgvv" event={"ID":"1399c32f-c1f8-4d3e-8350-b7900e68a30c","Type":"ContainerDied","Data":"deadbafc2f7dcfd9bbda6d8f168980d4e271f43ac37a30c634330ac657bae27c"}
Jul 28 22:10:21.456790 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:21.456612    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:10:21.457362 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:21.456625    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:10:21.457362 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:21.456893    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7cn74" podUID="be77a0c4-e19b-49d2-b547-c36c3d79dd33"
Jul 28 22:10:21.457362 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:21.456938    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zb7gv" podUID="9a7c5df8-e808-4a53-9697-7dc091ca7660"
Jul 28 22:10:21.752816 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:21.752795    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:10:21.753139 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:21.753119    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-222qs" event={"ID":"481b2f34-6f30-4e75-986c-c7eba23f6214","Type":"ContainerStarted","Data":"6dcde916d15944570f2cdc613979eb5c3b9721b434ddb4c8170dcaf7a152d997"}
Jul 28 22:10:21.753386 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:21.753355    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:10:21.753601 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:21.753447    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:10:21.753601 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:21.753558    2551 scope.go:117] "RemoveContainer" containerID="6a5c99844515672290360ea47427f3c90c53a012cee60f07c9676b01c4902b50"
Jul 28 22:10:21.768695 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:21.768673    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:10:22.457781 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:22.457753    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:10:22.458172 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:22.457878    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6pcj" podUID="c54b2692-1620-47a1-b04e-7b7f6b5df90a"
Jul 28 22:10:22.759270 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:22.759210    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:10:22.759622 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:22.759597    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-222qs" event={"ID":"481b2f34-6f30-4e75-986c-c7eba23f6214","Type":"ContainerStarted","Data":"e8876bee14dc7506cecb1fc004b14409c7e0dc5e7ef1ba0ad9f58350f133ba55"}
Jul 28 22:10:22.759889 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:22.759869    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:10:22.762179 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:22.762151    2551 generic.go:356] "Generic (PLEG): container finished" podID="1399c32f-c1f8-4d3e-8350-b7900e68a30c" containerID="33ee25ad6c75776b35fb5f2cb4a821bea067622cef32fd37a3fcf65dc0450691" exitCode=0
Jul 28 22:10:22.762275 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:22.762194    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dfgvv" event={"ID":"1399c32f-c1f8-4d3e-8350-b7900e68a30c","Type":"ContainerDied","Data":"33ee25ad6c75776b35fb5f2cb4a821bea067622cef32fd37a3fcf65dc0450691"}
Jul 28 22:10:22.780493 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:22.780278    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:10:22.789641 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:22.789577    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-222qs" podStartSLOduration=10.659494455 podStartE2EDuration="29.789556519s" totalImagesPullingTime="19.130062064s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:09:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:22.787632792 +0000 UTC m=+30.805598486" watchObservedRunningTime="2026-07-28 22:10:22.789556519 +0000 UTC m=+30.807522212"
Jul 28 22:10:22.946143 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:22.946083    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zb7gv"]
Jul 28 22:10:22.946519 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:22.946236    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:10:22.946519 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:22.946376    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zb7gv" podUID="9a7c5df8-e808-4a53-9697-7dc091ca7660"
Jul 28 22:10:22.947006 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:22.946980    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7cn74"]
Jul 28 22:10:22.947109 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:22.947092    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:10:22.947228 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:22.947208    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7cn74" podUID="be77a0c4-e19b-49d2-b547-c36c3d79dd33"
Jul 28 22:10:22.951450 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:22.951423    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k6pcj"]
Jul 28 22:10:22.951573 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:22.951511    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:10:22.951650 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:22.951601    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6pcj" podUID="c54b2692-1620-47a1-b04e-7b7f6b5df90a"
Jul 28 22:10:24.456729 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:24.456688    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:10:24.456729 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:24.456716    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:10:24.457341 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:24.456688    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:10:24.457341 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:24.456836    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6pcj" podUID="c54b2692-1620-47a1-b04e-7b7f6b5df90a"
Jul 28 22:10:24.457341 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:24.456953    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7cn74" podUID="be77a0c4-e19b-49d2-b547-c36c3d79dd33"
Jul 28 22:10:24.457341 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:24.457032    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zb7gv" podUID="9a7c5df8-e808-4a53-9697-7dc091ca7660"
Jul 28 22:10:26.056549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:26.056455    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v4p4\" (UniqueName: \"kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4\") pod \"network-check-target-zb7gv\" (UID: \"9a7c5df8-e808-4a53-9697-7dc091ca7660\") " pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:10:26.056549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:26.056497    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs\") pod \"network-metrics-daemon-k6pcj\" (UID: \"c54b2692-1620-47a1-b04e-7b7f6b5df90a\") " pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:10:26.057148 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:26.056622    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:10:26.057148 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:26.056669    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 22:10:26.057148 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:26.056693    2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 22:10:26.057148 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:26.056704    2551 projected.go:196] Error preparing data for projected volume kube-api-access-7v4p4 for pod openshift-network-diagnostics/network-check-target-zb7gv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:10:26.057148 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:26.056692    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs podName:c54b2692-1620-47a1-b04e-7b7f6b5df90a nodeName:}" failed. No retries permitted until 2026-07-28 22:10:58.056673196 +0000 UTC m=+66.074638873 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs") pod "network-metrics-daemon-k6pcj" (UID: "c54b2692-1620-47a1-b04e-7b7f6b5df90a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 22:10:26.057148 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:26.056756    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4 podName:9a7c5df8-e808-4a53-9697-7dc091ca7660 nodeName:}" failed. No retries permitted until 2026-07-28 22:10:58.056743931 +0000 UTC m=+66.074709608 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7v4p4" (UniqueName: "kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4") pod "network-check-target-zb7gv" (UID: "9a7c5df8-e808-4a53-9697-7dc091ca7660") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 22:10:26.157290 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:26.157244    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret\") pod \"global-pull-secret-syncer-7cn74\" (UID: \"be77a0c4-e19b-49d2-b547-c36c3d79dd33\") " pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:10:26.157460 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:26.157405    2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 22:10:26.157547 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:26.157478    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret podName:be77a0c4-e19b-49d2-b547-c36c3d79dd33 nodeName:}" failed. No retries permitted until 2026-07-28 22:10:58.157461568 +0000 UTC m=+66.175427244 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret") pod "global-pull-secret-syncer-7cn74" (UID: "be77a0c4-e19b-49d2-b547-c36c3d79dd33") : object "kube-system"/"original-pull-secret" not registered
Jul 28 22:10:26.456422 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:26.456386    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:10:26.456651 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:26.456514    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zb7gv" podUID="9a7c5df8-e808-4a53-9697-7dc091ca7660"
Jul 28 22:10:26.456651 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:26.456386    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:10:26.456651 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:26.456631    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7cn74" podUID="be77a0c4-e19b-49d2-b547-c36c3d79dd33"
Jul 28 22:10:26.456651 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:26.456386    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:10:26.456858 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:26.456727    2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k6pcj" podUID="c54b2692-1620-47a1-b04e-7b7f6b5df90a"
Jul 28 22:10:27.331376 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.331340    2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-102.ec2.internal" event="NodeReady"
Jul 28 22:10:27.331908 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.331467    2551 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 22:10:27.364037 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.363981    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"]
Jul 28 22:10:27.379399 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.379367    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-x8fct"]
Jul 28 22:10:27.379562 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.379471    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.381845 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.381821    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 22:10:27.381978 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.381822    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 22:10:27.382145 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.382126    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 22:10:27.382300 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.382172    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hf5kg\""
Jul 28 22:10:27.388290 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.388268    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 22:10:27.393490 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.393469    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-r6wpx"]
Jul 28 22:10:27.393693 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.393652    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x8fct"
Jul 28 22:10:27.395605 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.395590    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 22:10:27.395703 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.395616    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 22:10:27.395703 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.395635    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vcmq5\""
Jul 28 22:10:27.412386 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.412356    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"]
Jul 28 22:10:27.412562 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.412405    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x8fct"]
Jul 28 22:10:27.412562 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.412433    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r6wpx"]
Jul 28 22:10:27.412562 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.412449    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nmfk9"]
Jul 28 22:10:27.412562 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.412492    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r6wpx"
Jul 28 22:10:27.414355 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.414332    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 22:10:27.414476 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.414355    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 22:10:27.414476 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.414368    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lsdr6\""
Jul 28 22:10:27.414476 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.414463    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 22:10:27.433771 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.433694    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nmfk9"]
Jul 28 22:10:27.433924 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.433819    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:27.435760 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.435743    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 22:10:27.435918 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.435896    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 22:10:27.435918 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.435914    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 22:10:27.436078 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.435923    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-r6q89\""
Jul 28 22:10:27.436078 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.435936    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 22:10:27.468207 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.468172    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-image-registry-private-configuration\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.468363 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.468227    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-trusted-ca\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.468363 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.468270    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/31f5aa14-3e3d-445b-b646-1ac37987f977-config-volume\") pod \"dns-default-x8fct\" (UID: \"31f5aa14-3e3d-445b-b646-1ac37987f977\") " pod="openshift-dns/dns-default-x8fct"
Jul 28 22:10:27.468363 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.468290    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9zl5\" (UniqueName: \"kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-kube-api-access-h9zl5\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.468363 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.468307    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31f5aa14-3e3d-445b-b646-1ac37987f977-metrics-tls\") pod \"dns-default-x8fct\" (UID: \"31f5aa14-3e3d-445b-b646-1ac37987f977\") " pod="openshift-dns/dns-default-x8fct"
Jul 28 22:10:27.468363 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.468330    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-tls\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.468520 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.468400    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-certificates\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.468520 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.468428    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-ca-trust-extracted\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.468520 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.468452    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31f5aa14-3e3d-445b-b646-1ac37987f977-tmp-dir\") pod \"dns-default-x8fct\" (UID: \"31f5aa14-3e3d-445b-b646-1ac37987f977\") " pod="openshift-dns/dns-default-x8fct"
Jul 28 22:10:27.468520 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.468470    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhbhv\" (UniqueName: \"kubernetes.io/projected/31f5aa14-3e3d-445b-b646-1ac37987f977-kube-api-access-dhbhv\") pod \"dns-default-x8fct\" (UID: \"31f5aa14-3e3d-445b-b646-1ac37987f977\") " pod="openshift-dns/dns-default-x8fct"
Jul 28 22:10:27.468520 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.468486    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-bound-sa-token\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.468520 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.468513    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-installation-pull-secrets\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.568910 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.568868    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/31f5aa14-3e3d-445b-b646-1ac37987f977-config-volume\") pod \"dns-default-x8fct\" (UID: \"31f5aa14-3e3d-445b-b646-1ac37987f977\") " pod="openshift-dns/dns-default-x8fct"
Jul 28 22:10:27.568910 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.568911    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ff7f02a-ec84-4e1c-b29d-65063a1dd278-cert\") pod \"ingress-canary-r6wpx\" (UID: \"0ff7f02a-ec84-4e1c-b29d-65063a1dd278\") " pod="openshift-ingress-canary/ingress-canary-r6wpx"
Jul 28 22:10:27.569122 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.568938    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/511a4d5c-39cf-4a64-971d-1ee43a729c90-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nmfk9\" (UID: \"511a4d5c-39cf-4a64-971d-1ee43a729c90\") " pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:27.569122 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.568968    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9zl5\" (UniqueName: \"kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-kube-api-access-h9zl5\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.569212 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.569138    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31f5aa14-3e3d-445b-b646-1ac37987f977-metrics-tls\") pod \"dns-default-x8fct\" (UID: \"31f5aa14-3e3d-445b-b646-1ac37987f977\") " pod="openshift-dns/dns-default-x8fct"
Jul 28 22:10:27.569212 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.569166    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-tls\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.569212 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.569205    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-certificates\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.569360 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.569235    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-ca-trust-extracted\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.569360 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.569264    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31f5aa14-3e3d-445b-b646-1ac37987f977-tmp-dir\") pod \"dns-default-x8fct\" (UID: \"31f5aa14-3e3d-445b-b646-1ac37987f977\") " pod="openshift-dns/dns-default-x8fct"
Jul 28 22:10:27.569360 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.569292    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhbhv\" (UniqueName: \"kubernetes.io/projected/31f5aa14-3e3d-445b-b646-1ac37987f977-kube-api-access-dhbhv\") pod \"dns-default-x8fct\" (UID: \"31f5aa14-3e3d-445b-b646-1ac37987f977\") " pod="openshift-dns/dns-default-x8fct"
Jul 28 22:10:27.569360 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.569314    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-bound-sa-token\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.569574 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:27.569290    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 22:10:27.569574 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:27.569442    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31f5aa14-3e3d-445b-b646-1ac37987f977-metrics-tls podName:31f5aa14-3e3d-445b-b646-1ac37987f977 nodeName:}" failed. No retries permitted until 2026-07-28 22:10:28.069422912 +0000 UTC m=+36.087388587 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/31f5aa14-3e3d-445b-b646-1ac37987f977-metrics-tls") pod "dns-default-x8fct" (UID: "31f5aa14-3e3d-445b-b646-1ac37987f977") : secret "dns-default-metrics-tls" not found
Jul 28 22:10:27.569574 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:27.569368    2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 22:10:27.569574 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:27.569466    2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c5dc9fc45-ztdwr: secret "image-registry-tls" not found
Jul 28 22:10:27.569574 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.569549    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/31f5aa14-3e3d-445b-b646-1ac37987f977-config-volume\") pod \"dns-default-x8fct\" (UID: \"31f5aa14-3e3d-445b-b646-1ac37987f977\") " pod="openshift-dns/dns-default-x8fct"
Jul 28 22:10:27.569574 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:27.569570    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-tls podName:50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe nodeName:}" failed. No retries permitted until 2026-07-28 22:10:28.069520341 +0000 UTC m=+36.087486031 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-tls") pod "image-registry-7c5dc9fc45-ztdwr" (UID: "50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe") : secret "image-registry-tls" not found
Jul 28 22:10:27.569881 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.569615    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-installation-pull-secrets\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.569881 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.569624    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31f5aa14-3e3d-445b-b646-1ac37987f977-tmp-dir\") pod \"dns-default-x8fct\" (UID: \"31f5aa14-3e3d-445b-b646-1ac37987f977\") " pod="openshift-dns/dns-default-x8fct"
Jul 28 22:10:27.569881 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.569637    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/511a4d5c-39cf-4a64-971d-1ee43a729c90-crio-socket\") pod \"insights-runtime-extractor-nmfk9\" (UID: \"511a4d5c-39cf-4a64-971d-1ee43a729c90\") " pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:27.569881 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.569681    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrmc2\" (UniqueName: \"kubernetes.io/projected/511a4d5c-39cf-4a64-971d-1ee43a729c90-kube-api-access-qrmc2\") pod \"insights-runtime-extractor-nmfk9\" (UID: \"511a4d5c-39cf-4a64-971d-1ee43a729c90\") " pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:27.569881 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.569725    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-image-registry-private-configuration\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.569881 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.569727    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-ca-trust-extracted\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.569881 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.569761    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/511a4d5c-39cf-4a64-971d-1ee43a729c90-data-volume\") pod \"insights-runtime-extractor-nmfk9\" (UID: \"511a4d5c-39cf-4a64-971d-1ee43a729c90\") " pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:27.569881 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.569788    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/511a4d5c-39cf-4a64-971d-1ee43a729c90-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nmfk9\" (UID: \"511a4d5c-39cf-4a64-971d-1ee43a729c90\") " pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:27.569881 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.569847    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-trusted-ca\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.569881 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.569869    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-certificates\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.570364 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.569875    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4th7\" (UniqueName: \"kubernetes.io/projected/0ff7f02a-ec84-4e1c-b29d-65063a1dd278-kube-api-access-h4th7\") pod \"ingress-canary-r6wpx\" (UID: \"0ff7f02a-ec84-4e1c-b29d-65063a1dd278\") " pod="openshift-ingress-canary/ingress-canary-r6wpx"
Jul 28 22:10:27.570716 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.570691    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-trusted-ca\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.573509 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.573486    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-installation-pull-secrets\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.573617 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.573519    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-image-registry-private-configuration\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.577164 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.577144    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-bound-sa-token\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.577291 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.577201    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9zl5\" (UniqueName: \"kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-kube-api-access-h9zl5\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:27.577364 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.577346    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhbhv\" (UniqueName: \"kubernetes.io/projected/31f5aa14-3e3d-445b-b646-1ac37987f977-kube-api-access-dhbhv\") pod \"dns-default-x8fct\" (UID: \"31f5aa14-3e3d-445b-b646-1ac37987f977\") " pod="openshift-dns/dns-default-x8fct"
Jul 28 22:10:27.670864 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.670829    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/511a4d5c-39cf-4a64-971d-1ee43a729c90-crio-socket\") pod \"insights-runtime-extractor-nmfk9\" (UID: \"511a4d5c-39cf-4a64-971d-1ee43a729c90\") " pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:27.670864 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.670869    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrmc2\" (UniqueName: \"kubernetes.io/projected/511a4d5c-39cf-4a64-971d-1ee43a729c90-kube-api-access-qrmc2\") pod \"insights-runtime-extractor-nmfk9\" (UID: \"511a4d5c-39cf-4a64-971d-1ee43a729c90\") " pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:27.671109 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.670892    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/511a4d5c-39cf-4a64-971d-1ee43a729c90-data-volume\") pod \"insights-runtime-extractor-nmfk9\" (UID: \"511a4d5c-39cf-4a64-971d-1ee43a729c90\") " pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:27.671109 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.670910    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/511a4d5c-39cf-4a64-971d-1ee43a729c90-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nmfk9\" (UID: \"511a4d5c-39cf-4a64-971d-1ee43a729c90\") " pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:27.671109 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:27.670995    2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 22:10:27.671109 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:27.671048    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/511a4d5c-39cf-4a64-971d-1ee43a729c90-insights-runtime-extractor-tls podName:511a4d5c-39cf-4a64-971d-1ee43a729c90 nodeName:}" failed. No retries permitted until 2026-07-28 22:10:28.171034249 +0000 UTC m=+36.188999921 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/511a4d5c-39cf-4a64-971d-1ee43a729c90-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nmfk9" (UID: "511a4d5c-39cf-4a64-971d-1ee43a729c90") : secret "insights-runtime-extractor-tls" not found
Jul 28 22:10:27.671109 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.671061    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4th7\" (UniqueName: \"kubernetes.io/projected/0ff7f02a-ec84-4e1c-b29d-65063a1dd278-kube-api-access-h4th7\") pod \"ingress-canary-r6wpx\" (UID: \"0ff7f02a-ec84-4e1c-b29d-65063a1dd278\") " pod="openshift-ingress-canary/ingress-canary-r6wpx"
Jul 28 22:10:27.671109 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.671107    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ff7f02a-ec84-4e1c-b29d-65063a1dd278-cert\") pod \"ingress-canary-r6wpx\" (UID: \"0ff7f02a-ec84-4e1c-b29d-65063a1dd278\") " pod="openshift-ingress-canary/ingress-canary-r6wpx"
Jul 28 22:10:27.671380 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.671233    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/511a4d5c-39cf-4a64-971d-1ee43a729c90-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nmfk9\" (UID: \"511a4d5c-39cf-4a64-971d-1ee43a729c90\") " pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:27.671380 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.671250    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/511a4d5c-39cf-4a64-971d-1ee43a729c90-data-volume\") pod \"insights-runtime-extractor-nmfk9\" (UID: \"511a4d5c-39cf-4a64-971d-1ee43a729c90\") " pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:27.671380 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.671246    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/511a4d5c-39cf-4a64-971d-1ee43a729c90-crio-socket\") pod \"insights-runtime-extractor-nmfk9\" (UID: \"511a4d5c-39cf-4a64-971d-1ee43a729c90\") " pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:27.671380 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:27.671326    2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 22:10:27.671380 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:27.671376    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/511a4d5c-39cf-4a64-971d-1ee43a729c90-kube-rbac-proxy-cm podName:511a4d5c-39cf-4a64-971d-1ee43a729c90 nodeName:}" failed. No retries permitted until 2026-07-28 22:10:28.171361271 +0000 UTC m=+36.189326968 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/511a4d5c-39cf-4a64-971d-1ee43a729c90-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nmfk9" (UID: "511a4d5c-39cf-4a64-971d-1ee43a729c90") : configmap "kube-rbac-proxy" not found
Jul 28 22:10:27.673369 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.673352    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ff7f02a-ec84-4e1c-b29d-65063a1dd278-cert\") pod \"ingress-canary-r6wpx\" (UID: \"0ff7f02a-ec84-4e1c-b29d-65063a1dd278\") " pod="openshift-ingress-canary/ingress-canary-r6wpx"
Jul 28 22:10:27.678830 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.678807    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrmc2\" (UniqueName: \"kubernetes.io/projected/511a4d5c-39cf-4a64-971d-1ee43a729c90-kube-api-access-qrmc2\") pod \"insights-runtime-extractor-nmfk9\" (UID: \"511a4d5c-39cf-4a64-971d-1ee43a729c90\") " pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:27.678928 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.678891    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4th7\" (UniqueName: \"kubernetes.io/projected/0ff7f02a-ec84-4e1c-b29d-65063a1dd278-kube-api-access-h4th7\") pod \"ingress-canary-r6wpx\" (UID: \"0ff7f02a-ec84-4e1c-b29d-65063a1dd278\") " pod="openshift-ingress-canary/ingress-canary-r6wpx"
Jul 28 22:10:27.720285 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.720210    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-r6wpx"
Jul 28 22:10:27.776056 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.776027    2551 generic.go:356] "Generic (PLEG): container finished" podID="1399c32f-c1f8-4d3e-8350-b7900e68a30c" containerID="f49ab68744420af22717baca4a41be32dcead4ec59d9a582f7dfaf7748a5de5d" exitCode=0
Jul 28 22:10:27.776207 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.776093    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dfgvv" event={"ID":"1399c32f-c1f8-4d3e-8350-b7900e68a30c","Type":"ContainerDied","Data":"f49ab68744420af22717baca4a41be32dcead4ec59d9a582f7dfaf7748a5de5d"}
Jul 28 22:10:27.927275 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.926790    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-r6wpx"]
Jul 28 22:10:27.929914 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:10:27.929883    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ff7f02a_ec84_4e1c_b29d_65063a1dd278.slice/crio-6b4d6913c3bb4d41ac3d6a183e382ccd218647e67d850c4024fcf2c3e2a99b03 WatchSource:0}: Error finding container 6b4d6913c3bb4d41ac3d6a183e382ccd218647e67d850c4024fcf2c3e2a99b03: Status 404 returned error can't find the container with id 6b4d6913c3bb4d41ac3d6a183e382ccd218647e67d850c4024fcf2c3e2a99b03
Jul 28 22:10:27.950481 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:27.950460    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-trrlj_03112039-98fd-478f-b873-2504ce8aae02/dns-node-resolver/0.log"
Jul 28 22:10:28.074171 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.074143    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31f5aa14-3e3d-445b-b646-1ac37987f977-metrics-tls\") pod \"dns-default-x8fct\" (UID: \"31f5aa14-3e3d-445b-b646-1ac37987f977\") " pod="openshift-dns/dns-default-x8fct"
Jul 28 22:10:28.074318 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.074176    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-tls\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:28.074318 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:28.074278    2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 22:10:28.074318 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:28.074288    2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c5dc9fc45-ztdwr: secret "image-registry-tls" not found
Jul 28 22:10:28.074318 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:28.074290    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 22:10:28.074442 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:28.074336    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-tls podName:50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe nodeName:}" failed. No retries permitted until 2026-07-28 22:10:29.074323327 +0000 UTC m=+37.092288998 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-tls") pod "image-registry-7c5dc9fc45-ztdwr" (UID: "50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe") : secret "image-registry-tls" not found
Jul 28 22:10:28.074442 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:28.074350    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31f5aa14-3e3d-445b-b646-1ac37987f977-metrics-tls podName:31f5aa14-3e3d-445b-b646-1ac37987f977 nodeName:}" failed. No retries permitted until 2026-07-28 22:10:29.074344684 +0000 UTC m=+37.092310356 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/31f5aa14-3e3d-445b-b646-1ac37987f977-metrics-tls") pod "dns-default-x8fct" (UID: "31f5aa14-3e3d-445b-b646-1ac37987f977") : secret "dns-default-metrics-tls" not found
Jul 28 22:10:28.174752 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.174723    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/511a4d5c-39cf-4a64-971d-1ee43a729c90-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nmfk9\" (UID: \"511a4d5c-39cf-4a64-971d-1ee43a729c90\") " pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:28.174908 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.174801    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/511a4d5c-39cf-4a64-971d-1ee43a729c90-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nmfk9\" (UID: \"511a4d5c-39cf-4a64-971d-1ee43a729c90\") " pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:28.174908 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:28.174839    2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 22:10:28.174908 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:28.174905    2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 22:10:28.175004 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:28.174906    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/511a4d5c-39cf-4a64-971d-1ee43a729c90-kube-rbac-proxy-cm podName:511a4d5c-39cf-4a64-971d-1ee43a729c90 nodeName:}" failed. No retries permitted until 2026-07-28 22:10:29.174887876 +0000 UTC m=+37.192853548 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/511a4d5c-39cf-4a64-971d-1ee43a729c90-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nmfk9" (UID: "511a4d5c-39cf-4a64-971d-1ee43a729c90") : configmap "kube-rbac-proxy" not found
Jul 28 22:10:28.175004 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:28.174954    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/511a4d5c-39cf-4a64-971d-1ee43a729c90-insights-runtime-extractor-tls podName:511a4d5c-39cf-4a64-971d-1ee43a729c90 nodeName:}" failed. No retries permitted until 2026-07-28 22:10:29.174940542 +0000 UTC m=+37.192906214 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/511a4d5c-39cf-4a64-971d-1ee43a729c90-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nmfk9" (UID: "511a4d5c-39cf-4a64-971d-1ee43a729c90") : secret "insights-runtime-extractor-tls" not found
Jul 28 22:10:28.456702 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.456628    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:10:28.456702 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.456675    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:10:28.456702 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.456675    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:10:28.459350 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.459114    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 22:10:28.459350 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.459201    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 22:10:28.459350 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.459238    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 22:10:28.459350 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.459256    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7dq7d\""
Jul 28 22:10:28.459752 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.459734    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-q7z5n\""
Jul 28 22:10:28.459842 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.459829    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 22:10:28.558334 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.558305    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-fgt6c"]
Jul 28 22:10:28.582694 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.582660    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-fgt6c"]
Jul 28 22:10:28.582834 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.582763    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-fgt6c"
Jul 28 22:10:28.584893 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.584870    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 28 22:10:28.585031 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.584964    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-5qphs\""
Jul 28 22:10:28.585031 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.584974    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 28 22:10:28.585031 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.585015    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 28 22:10:28.585210 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.585102    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 28 22:10:28.678736 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.678688    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/233c2081-963b-4ed1-a399-84048e39a1fb-signing-key\") pod \"service-ca-658d7b7c78-fgt6c\" (UID: \"233c2081-963b-4ed1-a399-84048e39a1fb\") " pod="openshift-service-ca/service-ca-658d7b7c78-fgt6c"
Jul 28 22:10:28.678911 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.678813    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/233c2081-963b-4ed1-a399-84048e39a1fb-signing-cabundle\") pod \"service-ca-658d7b7c78-fgt6c\" (UID: \"233c2081-963b-4ed1-a399-84048e39a1fb\") " pod="openshift-service-ca/service-ca-658d7b7c78-fgt6c"
Jul 28 22:10:28.678911 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.678861    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7jrj\" (UniqueName: \"kubernetes.io/projected/233c2081-963b-4ed1-a399-84048e39a1fb-kube-api-access-s7jrj\") pod \"service-ca-658d7b7c78-fgt6c\" (UID: \"233c2081-963b-4ed1-a399-84048e39a1fb\") " pod="openshift-service-ca/service-ca-658d7b7c78-fgt6c"
Jul 28 22:10:28.724173 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.724098    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-25fc9_9c7c65c8-313a-4e85-9c0b-4c190cf1cbea/node-ca/0.log"
Jul 28 22:10:28.779467 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.779432    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/233c2081-963b-4ed1-a399-84048e39a1fb-signing-cabundle\") pod \"service-ca-658d7b7c78-fgt6c\" (UID: \"233c2081-963b-4ed1-a399-84048e39a1fb\") " pod="openshift-service-ca/service-ca-658d7b7c78-fgt6c"
Jul 28 22:10:28.779678 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.779516    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7jrj\" (UniqueName: \"kubernetes.io/projected/233c2081-963b-4ed1-a399-84048e39a1fb-kube-api-access-s7jrj\") pod \"service-ca-658d7b7c78-fgt6c\" (UID: \"233c2081-963b-4ed1-a399-84048e39a1fb\") " pod="openshift-service-ca/service-ca-658d7b7c78-fgt6c"
Jul 28 22:10:28.779678 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.779627    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/233c2081-963b-4ed1-a399-84048e39a1fb-signing-key\") pod \"service-ca-658d7b7c78-fgt6c\" (UID: \"233c2081-963b-4ed1-a399-84048e39a1fb\") " pod="openshift-service-ca/service-ca-658d7b7c78-fgt6c"
Jul 28 22:10:28.780249 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.780221    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/233c2081-963b-4ed1-a399-84048e39a1fb-signing-cabundle\") pod \"service-ca-658d7b7c78-fgt6c\" (UID: \"233c2081-963b-4ed1-a399-84048e39a1fb\") " pod="openshift-service-ca/service-ca-658d7b7c78-fgt6c"
Jul 28 22:10:28.780892 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.780825    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r6wpx" event={"ID":"0ff7f02a-ec84-4e1c-b29d-65063a1dd278","Type":"ContainerStarted","Data":"6b4d6913c3bb4d41ac3d6a183e382ccd218647e67d850c4024fcf2c3e2a99b03"}
Jul 28 22:10:28.784002 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.783979    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/233c2081-963b-4ed1-a399-84048e39a1fb-signing-key\") pod \"service-ca-658d7b7c78-fgt6c\" (UID: \"233c2081-963b-4ed1-a399-84048e39a1fb\") " pod="openshift-service-ca/service-ca-658d7b7c78-fgt6c"
Jul 28 22:10:28.784818 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.784796    2551 generic.go:356] "Generic (PLEG): container finished" podID="1399c32f-c1f8-4d3e-8350-b7900e68a30c" containerID="827054d32906b1c2ecb4ab2f54eb5d4d827675641e5a7eab781415e61a937c19" exitCode=0
Jul 28 22:10:28.784923 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.784831    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dfgvv" event={"ID":"1399c32f-c1f8-4d3e-8350-b7900e68a30c","Type":"ContainerDied","Data":"827054d32906b1c2ecb4ab2f54eb5d4d827675641e5a7eab781415e61a937c19"}
Jul 28 22:10:28.788046 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.787968    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7jrj\" (UniqueName: \"kubernetes.io/projected/233c2081-963b-4ed1-a399-84048e39a1fb-kube-api-access-s7jrj\") pod \"service-ca-658d7b7c78-fgt6c\" (UID: \"233c2081-963b-4ed1-a399-84048e39a1fb\") " pod="openshift-service-ca/service-ca-658d7b7c78-fgt6c"
Jul 28 22:10:28.891442 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:28.891417    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-fgt6c"
Jul 28 22:10:29.057428 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:29.057212    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-fgt6c"]
Jul 28 22:10:29.062010 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:10:29.061971    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod233c2081_963b_4ed1_a399_84048e39a1fb.slice/crio-110d85af5a9aa3a199c3bf5743b20a15059631ad701587f6913d8dd06194a9a5 WatchSource:0}: Error finding container 110d85af5a9aa3a199c3bf5743b20a15059631ad701587f6913d8dd06194a9a5: Status 404 returned error can't find the container with id 110d85af5a9aa3a199c3bf5743b20a15059631ad701587f6913d8dd06194a9a5
Jul 28 22:10:29.083507 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:29.083477    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31f5aa14-3e3d-445b-b646-1ac37987f977-metrics-tls\") pod \"dns-default-x8fct\" (UID: \"31f5aa14-3e3d-445b-b646-1ac37987f977\") " pod="openshift-dns/dns-default-x8fct"
Jul 28 22:10:29.083642 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:29.083518    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-tls\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:29.083711 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:29.083639    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 22:10:29.083711 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:29.083653    2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 22:10:29.083711 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:29.083666    2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c5dc9fc45-ztdwr: secret "image-registry-tls" not found
Jul 28 22:10:29.083853 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:29.083722    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31f5aa14-3e3d-445b-b646-1ac37987f977-metrics-tls podName:31f5aa14-3e3d-445b-b646-1ac37987f977 nodeName:}" failed. No retries permitted until 2026-07-28 22:10:31.083700233 +0000 UTC m=+39.101665919 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/31f5aa14-3e3d-445b-b646-1ac37987f977-metrics-tls") pod "dns-default-x8fct" (UID: "31f5aa14-3e3d-445b-b646-1ac37987f977") : secret "dns-default-metrics-tls" not found
Jul 28 22:10:29.083853 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:29.083743    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-tls podName:50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe nodeName:}" failed. No retries permitted until 2026-07-28 22:10:31.08373311 +0000 UTC m=+39.101698789 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-tls") pod "image-registry-7c5dc9fc45-ztdwr" (UID: "50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe") : secret "image-registry-tls" not found
Jul 28 22:10:29.184796 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:29.184753    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/511a4d5c-39cf-4a64-971d-1ee43a729c90-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nmfk9\" (UID: \"511a4d5c-39cf-4a64-971d-1ee43a729c90\") " pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:29.184964 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:29.184845    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/511a4d5c-39cf-4a64-971d-1ee43a729c90-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nmfk9\" (UID: \"511a4d5c-39cf-4a64-971d-1ee43a729c90\") " pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:29.184964 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:29.184938    2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 22:10:29.185084 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:29.185009    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/511a4d5c-39cf-4a64-971d-1ee43a729c90-insights-runtime-extractor-tls podName:511a4d5c-39cf-4a64-971d-1ee43a729c90 nodeName:}" failed. No retries permitted until 2026-07-28 22:10:31.184993761 +0000 UTC m=+39.202959432 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/511a4d5c-39cf-4a64-971d-1ee43a729c90-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nmfk9" (UID: "511a4d5c-39cf-4a64-971d-1ee43a729c90") : secret "insights-runtime-extractor-tls" not found
Jul 28 22:10:29.185084 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:29.184940    2551 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 22:10:29.185084 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:29.185050    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/511a4d5c-39cf-4a64-971d-1ee43a729c90-kube-rbac-proxy-cm podName:511a4d5c-39cf-4a64-971d-1ee43a729c90 nodeName:}" failed. No retries permitted until 2026-07-28 22:10:31.185041426 +0000 UTC m=+39.203007133 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/511a4d5c-39cf-4a64-971d-1ee43a729c90-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nmfk9" (UID: "511a4d5c-39cf-4a64-971d-1ee43a729c90") : configmap "kube-rbac-proxy" not found
Jul 28 22:10:29.788605 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:29.788546    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-fgt6c" event={"ID":"233c2081-963b-4ed1-a399-84048e39a1fb","Type":"ContainerStarted","Data":"110d85af5a9aa3a199c3bf5743b20a15059631ad701587f6913d8dd06194a9a5"}
Jul 28 22:10:29.792698 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:29.792668    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dfgvv" event={"ID":"1399c32f-c1f8-4d3e-8350-b7900e68a30c","Type":"ContainerStarted","Data":"66d349ee33c829e251aa7d43dbd5df4bfa4686df97e91cb3a221ef128ac50bf9"}
Jul 28 22:10:29.820547 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:29.820476    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dfgvv" podStartSLOduration=7.52345706 podStartE2EDuration="36.820455015s" totalImagesPullingTime="29.296997955s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:09:53 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:29.81931855 +0000 UTC m=+37.837284245" watchObservedRunningTime="2026-07-28 22:10:29.820455015 +0000 UTC m=+37.838420708"
Jul 28 22:10:31.101989 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:31.101955    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31f5aa14-3e3d-445b-b646-1ac37987f977-metrics-tls\") pod \"dns-default-x8fct\" (UID: \"31f5aa14-3e3d-445b-b646-1ac37987f977\") " pod="openshift-dns/dns-default-x8fct"
Jul 28 22:10:31.102369 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:31.102000    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-tls\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:31.102369 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:31.102091    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 22:10:31.102369 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:31.102145    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31f5aa14-3e3d-445b-b646-1ac37987f977-metrics-tls podName:31f5aa14-3e3d-445b-b646-1ac37987f977 nodeName:}" failed. No retries permitted until 2026-07-28 22:10:35.102130974 +0000 UTC m=+43.120096647 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/31f5aa14-3e3d-445b-b646-1ac37987f977-metrics-tls") pod "dns-default-x8fct" (UID: "31f5aa14-3e3d-445b-b646-1ac37987f977") : secret "dns-default-metrics-tls" not found
Jul 28 22:10:31.102369 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:31.102092    2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 22:10:31.102369 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:31.102183    2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c5dc9fc45-ztdwr: secret "image-registry-tls" not found
Jul 28 22:10:31.102369 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:31.102230    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-tls podName:50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe nodeName:}" failed. No retries permitted until 2026-07-28 22:10:35.102215232 +0000 UTC m=+43.120180906 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-tls") pod "image-registry-7c5dc9fc45-ztdwr" (UID: "50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe") : secret "image-registry-tls" not found
Jul 28 22:10:31.202711 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:31.202670    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/511a4d5c-39cf-4a64-971d-1ee43a729c90-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nmfk9\" (UID: \"511a4d5c-39cf-4a64-971d-1ee43a729c90\") " pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:31.202881 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:31.202792    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/511a4d5c-39cf-4a64-971d-1ee43a729c90-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nmfk9\" (UID: \"511a4d5c-39cf-4a64-971d-1ee43a729c90\") " pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:31.203000 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:31.202969    2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 22:10:31.203070 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:31.203060    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/511a4d5c-39cf-4a64-971d-1ee43a729c90-insights-runtime-extractor-tls podName:511a4d5c-39cf-4a64-971d-1ee43a729c90 nodeName:}" failed. No retries permitted until 2026-07-28 22:10:35.20303594 +0000 UTC m=+43.221001614 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/511a4d5c-39cf-4a64-971d-1ee43a729c90-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nmfk9" (UID: "511a4d5c-39cf-4a64-971d-1ee43a729c90") : secret "insights-runtime-extractor-tls" not found
Jul 28 22:10:31.203406 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:31.203342    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/511a4d5c-39cf-4a64-971d-1ee43a729c90-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nmfk9\" (UID: \"511a4d5c-39cf-4a64-971d-1ee43a729c90\") " pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:31.799415 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:31.799379    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-fgt6c" event={"ID":"233c2081-963b-4ed1-a399-84048e39a1fb","Type":"ContainerStarted","Data":"8fa9fc97f641052f10b7ce48a2227f766ff492c89f82bc87c6e24adeef20627f"}
Jul 28 22:10:31.800909 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:31.800878    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-r6wpx" event={"ID":"0ff7f02a-ec84-4e1c-b29d-65063a1dd278","Type":"ContainerStarted","Data":"e4597fb8dbad316eadbc8c5bdf3ee7645b9e1bc6b26de9c4f8ca83a53b6dc679"}
Jul 28 22:10:31.817420 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:31.817373    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-fgt6c" podStartSLOduration=1.789375384 podStartE2EDuration="3.817355904s" totalImagesPullingTime="2.02798052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:10:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:31.816841181 +0000 UTC m=+39.834806874" watchObservedRunningTime="2026-07-28 22:10:31.817355904 +0000 UTC m=+39.835321595"
Jul 28 22:10:31.831162 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:31.831110    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-r6wpx" podStartSLOduration=1.670993348 podStartE2EDuration="4.831087934s" totalImagesPullingTime="3.160094586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:10:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:31.830511839 +0000 UTC m=+39.848477531" watchObservedRunningTime="2026-07-28 22:10:31.831087934 +0000 UTC m=+39.849053628"
Jul 28 22:10:35.135725 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:35.135673    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31f5aa14-3e3d-445b-b646-1ac37987f977-metrics-tls\") pod \"dns-default-x8fct\" (UID: \"31f5aa14-3e3d-445b-b646-1ac37987f977\") " pod="openshift-dns/dns-default-x8fct"
Jul 28 22:10:35.135725 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:35.135719    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-tls\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:35.136446 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:35.135836    2551 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 22:10:35.136446 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:35.135853    2551 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7c5dc9fc45-ztdwr: secret "image-registry-tls" not found
Jul 28 22:10:35.136446 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:35.135909    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-tls podName:50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe nodeName:}" failed. No retries permitted until 2026-07-28 22:10:43.135892265 +0000 UTC m=+51.153857938 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-tls") pod "image-registry-7c5dc9fc45-ztdwr" (UID: "50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe") : secret "image-registry-tls" not found
Jul 28 22:10:35.136446 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:35.135833    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 22:10:35.136446 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:35.135981    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/31f5aa14-3e3d-445b-b646-1ac37987f977-metrics-tls podName:31f5aa14-3e3d-445b-b646-1ac37987f977 nodeName:}" failed. No retries permitted until 2026-07-28 22:10:43.135966034 +0000 UTC m=+51.153931705 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/31f5aa14-3e3d-445b-b646-1ac37987f977-metrics-tls") pod "dns-default-x8fct" (UID: "31f5aa14-3e3d-445b-b646-1ac37987f977") : secret "dns-default-metrics-tls" not found
Jul 28 22:10:35.236605 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:35.236570    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/511a4d5c-39cf-4a64-971d-1ee43a729c90-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nmfk9\" (UID: \"511a4d5c-39cf-4a64-971d-1ee43a729c90\") " pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:35.236840 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:35.236733    2551 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 22:10:35.236840 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:10:35.236797    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/511a4d5c-39cf-4a64-971d-1ee43a729c90-insights-runtime-extractor-tls podName:511a4d5c-39cf-4a64-971d-1ee43a729c90 nodeName:}" failed. No retries permitted until 2026-07-28 22:10:43.236781107 +0000 UTC m=+51.254746778 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/511a4d5c-39cf-4a64-971d-1ee43a729c90-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nmfk9" (UID: "511a4d5c-39cf-4a64-971d-1ee43a729c90") : secret "insights-runtime-extractor-tls" not found
Jul 28 22:10:43.198631 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:43.198595    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31f5aa14-3e3d-445b-b646-1ac37987f977-metrics-tls\") pod \"dns-default-x8fct\" (UID: \"31f5aa14-3e3d-445b-b646-1ac37987f977\") " pod="openshift-dns/dns-default-x8fct"
Jul 28 22:10:43.198631 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:43.198628    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-tls\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:43.201738 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:43.201716    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/31f5aa14-3e3d-445b-b646-1ac37987f977-metrics-tls\") pod \"dns-default-x8fct\" (UID: \"31f5aa14-3e3d-445b-b646-1ac37987f977\") " pod="openshift-dns/dns-default-x8fct"
Jul 28 22:10:43.201965 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:43.201943    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-tls\") pod \"image-registry-7c5dc9fc45-ztdwr\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") " pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:43.290197 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:43.290163    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:43.299534 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:43.299495    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/511a4d5c-39cf-4a64-971d-1ee43a729c90-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nmfk9\" (UID: \"511a4d5c-39cf-4a64-971d-1ee43a729c90\") " pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:43.301658 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:43.301631    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-x8fct"
Jul 28 22:10:43.301890 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:43.301870    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/511a4d5c-39cf-4a64-971d-1ee43a729c90-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nmfk9\" (UID: \"511a4d5c-39cf-4a64-971d-1ee43a729c90\") " pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:43.342629 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:43.342601    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nmfk9"
Jul 28 22:10:43.469360 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:43.469283    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"]
Jul 28 22:10:43.472401 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:10:43.472374    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50d5ff4d_4f55_4d51_9bb9_b6d1233a1cfe.slice/crio-c055342b634d2cdbc18ad167708ba570fb7b102bb907305c0d38b4f9a7caa543 WatchSource:0}: Error finding container c055342b634d2cdbc18ad167708ba570fb7b102bb907305c0d38b4f9a7caa543: Status 404 returned error can't find the container with id c055342b634d2cdbc18ad167708ba570fb7b102bb907305c0d38b4f9a7caa543
Jul 28 22:10:43.480882 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:43.480860    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-x8fct"]
Jul 28 22:10:43.483655 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:10:43.483625    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31f5aa14_3e3d_445b_b646_1ac37987f977.slice/crio-0cad247c5e2ec48ccb8e057a7345bf742abfea6a5917c70f93839f683a7078bd WatchSource:0}: Error finding container 0cad247c5e2ec48ccb8e057a7345bf742abfea6a5917c70f93839f683a7078bd: Status 404 returned error can't find the container with id 0cad247c5e2ec48ccb8e057a7345bf742abfea6a5917c70f93839f683a7078bd
Jul 28 22:10:43.522859 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:43.522829    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nmfk9"]
Jul 28 22:10:43.526685 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:10:43.526662    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod511a4d5c_39cf_4a64_971d_1ee43a729c90.slice/crio-6d3e103ea87ae6b922c562e00cd1491b3feb9bc78d019fa0ab27926cd81cadf2 WatchSource:0}: Error finding container 6d3e103ea87ae6b922c562e00cd1491b3feb9bc78d019fa0ab27926cd81cadf2: Status 404 returned error can't find the container with id 6d3e103ea87ae6b922c562e00cd1491b3feb9bc78d019fa0ab27926cd81cadf2
Jul 28 22:10:43.832678 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:43.832631    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nmfk9" event={"ID":"511a4d5c-39cf-4a64-971d-1ee43a729c90","Type":"ContainerStarted","Data":"6d3e103ea87ae6b922c562e00cd1491b3feb9bc78d019fa0ab27926cd81cadf2"}
Jul 28 22:10:43.834136 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:43.834106    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr" event={"ID":"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe","Type":"ContainerStarted","Data":"7c243e4a6fc4460558fbf3a6b4ab9e8f755b24dc766dbb65dcaef45c51341770"}
Jul 28 22:10:43.834270 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:43.834153    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr" event={"ID":"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe","Type":"ContainerStarted","Data":"c055342b634d2cdbc18ad167708ba570fb7b102bb907305c0d38b4f9a7caa543"}
Jul 28 22:10:43.834270 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:43.834219    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:10:43.835493 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:43.835471    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x8fct" event={"ID":"31f5aa14-3e3d-445b-b646-1ac37987f977","Type":"ContainerStarted","Data":"0cad247c5e2ec48ccb8e057a7345bf742abfea6a5917c70f93839f683a7078bd"}
Jul 28 22:10:43.851422 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:43.851375    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr" podStartSLOduration=44.851360396 podStartE2EDuration="44.851360396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:09:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:43.850993672 +0000 UTC m=+51.868959381" watchObservedRunningTime="2026-07-28 22:10:43.851360396 +0000 UTC m=+51.869326089"
Jul 28 22:10:49.856295 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:49.856257    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nmfk9" event={"ID":"511a4d5c-39cf-4a64-971d-1ee43a729c90","Type":"ContainerStarted","Data":"43e62789d13e42a306d8bf6bf8a219995ea992fa966b22d815395180adc265eb"}
Jul 28 22:10:49.858010 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:49.857984    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x8fct" event={"ID":"31f5aa14-3e3d-445b-b646-1ac37987f977","Type":"ContainerStarted","Data":"1560014bee081e404d06af27400663f5d115adbef9395c6fbe70779932bf3855"}
Jul 28 22:10:49.858119 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:49.858026    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-x8fct" event={"ID":"31f5aa14-3e3d-445b-b646-1ac37987f977","Type":"ContainerStarted","Data":"a1c2ebc98fae400d298e37c6636edd1df9e7f4e90b202b0686aceae0cfbb8d77"}
Jul 28 22:10:49.858157 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:49.858123    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-x8fct"
Jul 28 22:10:49.874247 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:49.874209    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-x8fct" podStartSLOduration=17.083016352 podStartE2EDuration="22.874196566s" totalImagesPullingTime="5.791180214s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:10:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:49.87393782 +0000 UTC m=+57.891903526" watchObservedRunningTime="2026-07-28 22:10:49.874196566 +0000 UTC m=+57.892162260"
Jul 28 22:10:50.761642 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.761604    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9c6c8748d-d92sm"]
Jul 28 22:10:50.763500 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.763479    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78869489fc-ztvlr"]
Jul 28 22:10:50.763659 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.763638    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9c6c8748d-d92sm"
Jul 28 22:10:50.765213 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.765191    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k"]
Jul 28 22:10:50.765330 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.765310    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78869489fc-ztvlr"
Jul 28 22:10:50.767179 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.767157    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k"
Jul 28 22:10:50.767985 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.767962    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 28 22:10:50.768373 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.768201    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 28 22:10:50.768373 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.768236    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 28 22:10:50.768373 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.768253    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 28 22:10:50.768373 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.768286    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-zh5hb\""
Jul 28 22:10:50.768649 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.768600    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 28 22:10:50.769752 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.769730    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 28 22:10:50.769849 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.769775    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 28 22:10:50.769849 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.769798    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 28 22:10:50.770065 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.770018    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 28 22:10:50.779237 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.779215    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9c6c8748d-d92sm"]
Jul 28 22:10:50.780293 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.780271    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78869489fc-ztvlr"]
Jul 28 22:10:50.783140 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.783118    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k"]
Jul 28 22:10:50.787963 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.787943    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"]
Jul 28 22:10:50.868736 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.868704    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0525ef00-1ad4-439f-8eb0-ce327d62940a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-d89464787-dm56k\" (UID: \"0525ef00-1ad4-439f-8eb0-ce327d62940a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k"
Jul 28 22:10:50.869129 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.868755    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66vcp\" (UniqueName: \"kubernetes.io/projected/ef5b5fc2-7b90-42c9-85ab-b1a56bb82ef2-kube-api-access-66vcp\") pod \"managed-serviceaccount-addon-agent-9c6c8748d-d92sm\" (UID: \"ef5b5fc2-7b90-42c9-85ab-b1a56bb82ef2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9c6c8748d-d92sm"
Jul 28 22:10:50.869129 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.868818    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49335e20-dd7c-4f6b-8f47-8cd9ba14f4a3-tmp\") pod \"klusterlet-addon-workmgr-78869489fc-ztvlr\" (UID: \"49335e20-dd7c-4f6b-8f47-8cd9ba14f4a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78869489fc-ztvlr"
Jul 28 22:10:50.869129 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.868840    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0525ef00-1ad4-439f-8eb0-ce327d62940a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-d89464787-dm56k\" (UID: \"0525ef00-1ad4-439f-8eb0-ce327d62940a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k"
Jul 28 22:10:50.869129 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.868864    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0525ef00-1ad4-439f-8eb0-ce327d62940a-hub\") pod \"cluster-proxy-proxy-agent-d89464787-dm56k\" (UID: \"0525ef00-1ad4-439f-8eb0-ce327d62940a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k"
Jul 28 22:10:50.869129 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.868935    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0525ef00-1ad4-439f-8eb0-ce327d62940a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-d89464787-dm56k\" (UID: \"0525ef00-1ad4-439f-8eb0-ce327d62940a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k"
Jul 28 22:10:50.869129 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.868974    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ef5b5fc2-7b90-42c9-85ab-b1a56bb82ef2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-9c6c8748d-d92sm\" (UID: \"ef5b5fc2-7b90-42c9-85ab-b1a56bb82ef2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9c6c8748d-d92sm"
Jul 28 22:10:50.869129 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.869014    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0525ef00-1ad4-439f-8eb0-ce327d62940a-ca\") pod \"cluster-proxy-proxy-agent-d89464787-dm56k\" (UID: \"0525ef00-1ad4-439f-8eb0-ce327d62940a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k"
Jul 28 22:10:50.869129 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.869046    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8s54\" (UniqueName: \"kubernetes.io/projected/0525ef00-1ad4-439f-8eb0-ce327d62940a-kube-api-access-r8s54\") pod \"cluster-proxy-proxy-agent-d89464787-dm56k\" (UID: \"0525ef00-1ad4-439f-8eb0-ce327d62940a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k"
Jul 28 22:10:50.869129 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.869075    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/49335e20-dd7c-4f6b-8f47-8cd9ba14f4a3-klusterlet-config\") pod \"klusterlet-addon-workmgr-78869489fc-ztvlr\" (UID: \"49335e20-dd7c-4f6b-8f47-8cd9ba14f4a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78869489fc-ztvlr"
Jul 28 22:10:50.869129 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.869102    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fprvk\" (UniqueName: \"kubernetes.io/projected/49335e20-dd7c-4f6b-8f47-8cd9ba14f4a3-kube-api-access-fprvk\") pod \"klusterlet-addon-workmgr-78869489fc-ztvlr\" (UID: \"49335e20-dd7c-4f6b-8f47-8cd9ba14f4a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78869489fc-ztvlr"
Jul 28 22:10:50.871174 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.871148    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-55dcc5fc8-nr27g"]
Jul 28 22:10:50.873236 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.873218    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:50.894180 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.894150    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55dcc5fc8-nr27g"]
Jul 28 22:10:50.970001 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.969961    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66vcp\" (UniqueName: \"kubernetes.io/projected/ef5b5fc2-7b90-42c9-85ab-b1a56bb82ef2-kube-api-access-66vcp\") pod \"managed-serviceaccount-addon-agent-9c6c8748d-d92sm\" (UID: \"ef5b5fc2-7b90-42c9-85ab-b1a56bb82ef2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9c6c8748d-d92sm"
Jul 28 22:10:50.970184 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.970037    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49335e20-dd7c-4f6b-8f47-8cd9ba14f4a3-tmp\") pod \"klusterlet-addon-workmgr-78869489fc-ztvlr\" (UID: \"49335e20-dd7c-4f6b-8f47-8cd9ba14f4a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78869489fc-ztvlr"
Jul 28 22:10:50.970184 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.970072    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0525ef00-1ad4-439f-8eb0-ce327d62940a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-d89464787-dm56k\" (UID: \"0525ef00-1ad4-439f-8eb0-ce327d62940a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k"
Jul 28 22:10:50.970184 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.970090    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0525ef00-1ad4-439f-8eb0-ce327d62940a-hub\") pod \"cluster-proxy-proxy-agent-d89464787-dm56k\" (UID: \"0525ef00-1ad4-439f-8eb0-ce327d62940a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k"
Jul 28 22:10:50.970184 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.970118    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0525ef00-1ad4-439f-8eb0-ce327d62940a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-d89464787-dm56k\" (UID: \"0525ef00-1ad4-439f-8eb0-ce327d62940a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k"
Jul 28 22:10:50.970184 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.970145    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ef5b5fc2-7b90-42c9-85ab-b1a56bb82ef2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-9c6c8748d-d92sm\" (UID: \"ef5b5fc2-7b90-42c9-85ab-b1a56bb82ef2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9c6c8748d-d92sm"
Jul 28 22:10:50.970184 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.970175    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tttvw\" (UniqueName: \"kubernetes.io/projected/e6b58763-25f6-42cf-ab80-50b19539628b-kube-api-access-tttvw\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:50.970504 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.970206    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0525ef00-1ad4-439f-8eb0-ce327d62940a-ca\") pod \"cluster-proxy-proxy-agent-d89464787-dm56k\" (UID: \"0525ef00-1ad4-439f-8eb0-ce327d62940a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k"
Jul 28 22:10:50.970504 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.970230    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e6b58763-25f6-42cf-ab80-50b19539628b-installation-pull-secrets\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:50.970504 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.970257    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e6b58763-25f6-42cf-ab80-50b19539628b-registry-certificates\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:50.970504 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.970286    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8s54\" (UniqueName: \"kubernetes.io/projected/0525ef00-1ad4-439f-8eb0-ce327d62940a-kube-api-access-r8s54\") pod \"cluster-proxy-proxy-agent-d89464787-dm56k\" (UID: \"0525ef00-1ad4-439f-8eb0-ce327d62940a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k"
Jul 28 22:10:50.970504 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.970306    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6b58763-25f6-42cf-ab80-50b19539628b-bound-sa-token\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:50.970504 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.970328    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/49335e20-dd7c-4f6b-8f47-8cd9ba14f4a3-klusterlet-config\") pod \"klusterlet-addon-workmgr-78869489fc-ztvlr\" (UID: \"49335e20-dd7c-4f6b-8f47-8cd9ba14f4a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78869489fc-ztvlr"
Jul 28 22:10:50.970504 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.970348    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6b58763-25f6-42cf-ab80-50b19539628b-registry-tls\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:50.970504 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.970383    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e6b58763-25f6-42cf-ab80-50b19539628b-image-registry-private-configuration\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:50.970504 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.970482    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/49335e20-dd7c-4f6b-8f47-8cd9ba14f4a3-tmp\") pod \"klusterlet-addon-workmgr-78869489fc-ztvlr\" (UID: \"49335e20-dd7c-4f6b-8f47-8cd9ba14f4a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78869489fc-ztvlr"
Jul 28 22:10:50.970964 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.970885    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fprvk\" (UniqueName: \"kubernetes.io/projected/49335e20-dd7c-4f6b-8f47-8cd9ba14f4a3-kube-api-access-fprvk\") pod \"klusterlet-addon-workmgr-78869489fc-ztvlr\" (UID: \"49335e20-dd7c-4f6b-8f47-8cd9ba14f4a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78869489fc-ztvlr"
Jul 28 22:10:50.970964 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.970955    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e6b58763-25f6-42cf-ab80-50b19539628b-ca-trust-extracted\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:50.971062 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.971000    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e6b58763-25f6-42cf-ab80-50b19539628b-trusted-ca\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:50.971293 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.971250    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0525ef00-1ad4-439f-8eb0-ce327d62940a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-d89464787-dm56k\" (UID: \"0525ef00-1ad4-439f-8eb0-ce327d62940a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k"
Jul 28 22:10:50.972159 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.972109    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/0525ef00-1ad4-439f-8eb0-ce327d62940a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-d89464787-dm56k\" (UID: \"0525ef00-1ad4-439f-8eb0-ce327d62940a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k"
Jul 28 22:10:50.973214 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.973170    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/0525ef00-1ad4-439f-8eb0-ce327d62940a-ca\") pod \"cluster-proxy-proxy-agent-d89464787-dm56k\" (UID: \"0525ef00-1ad4-439f-8eb0-ce327d62940a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k"
Jul 28 22:10:50.973355 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.973330    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ef5b5fc2-7b90-42c9-85ab-b1a56bb82ef2-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-9c6c8748d-d92sm\" (UID: \"ef5b5fc2-7b90-42c9-85ab-b1a56bb82ef2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9c6c8748d-d92sm"
Jul 28 22:10:50.973469 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.973361    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0525ef00-1ad4-439f-8eb0-ce327d62940a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-d89464787-dm56k\" (UID: \"0525ef00-1ad4-439f-8eb0-ce327d62940a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k"
Jul 28 22:10:50.973564 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.973511    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/0525ef00-1ad4-439f-8eb0-ce327d62940a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-d89464787-dm56k\" (UID: \"0525ef00-1ad4-439f-8eb0-ce327d62940a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k"
Jul 28 22:10:50.973619 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.973516    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/0525ef00-1ad4-439f-8eb0-ce327d62940a-hub\") pod \"cluster-proxy-proxy-agent-d89464787-dm56k\" (UID: \"0525ef00-1ad4-439f-8eb0-ce327d62940a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k"
Jul 28 22:10:50.974851 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.974831    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/49335e20-dd7c-4f6b-8f47-8cd9ba14f4a3-klusterlet-config\") pod \"klusterlet-addon-workmgr-78869489fc-ztvlr\" (UID: \"49335e20-dd7c-4f6b-8f47-8cd9ba14f4a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78869489fc-ztvlr"
Jul 28 22:10:50.978562 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.978516    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66vcp\" (UniqueName: \"kubernetes.io/projected/ef5b5fc2-7b90-42c9-85ab-b1a56bb82ef2-kube-api-access-66vcp\") pod \"managed-serviceaccount-addon-agent-9c6c8748d-d92sm\" (UID: \"ef5b5fc2-7b90-42c9-85ab-b1a56bb82ef2\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9c6c8748d-d92sm"
Jul 28 22:10:50.979718 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.979693    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8s54\" (UniqueName: \"kubernetes.io/projected/0525ef00-1ad4-439f-8eb0-ce327d62940a-kube-api-access-r8s54\") pod \"cluster-proxy-proxy-agent-d89464787-dm56k\" (UID: \"0525ef00-1ad4-439f-8eb0-ce327d62940a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k"
Jul 28 22:10:50.980574 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:50.980554    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fprvk\" (UniqueName: \"kubernetes.io/projected/49335e20-dd7c-4f6b-8f47-8cd9ba14f4a3-kube-api-access-fprvk\") pod \"klusterlet-addon-workmgr-78869489fc-ztvlr\" (UID: \"49335e20-dd7c-4f6b-8f47-8cd9ba14f4a3\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78869489fc-ztvlr"
Jul 28 22:10:51.072489 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.072458    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6b58763-25f6-42cf-ab80-50b19539628b-bound-sa-token\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:51.072666 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.072498    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6b58763-25f6-42cf-ab80-50b19539628b-registry-tls\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:51.072666 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.072518    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e6b58763-25f6-42cf-ab80-50b19539628b-image-registry-private-configuration\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:51.072666 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.072565    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e6b58763-25f6-42cf-ab80-50b19539628b-ca-trust-extracted\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:51.072666 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.072607    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e6b58763-25f6-42cf-ab80-50b19539628b-trusted-ca\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:51.072885 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.072694    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tttvw\" (UniqueName: \"kubernetes.io/projected/e6b58763-25f6-42cf-ab80-50b19539628b-kube-api-access-tttvw\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:51.072885 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.072726    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e6b58763-25f6-42cf-ab80-50b19539628b-installation-pull-secrets\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:51.072885 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.072755    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e6b58763-25f6-42cf-ab80-50b19539628b-registry-certificates\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:51.073282 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.073218    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e6b58763-25f6-42cf-ab80-50b19539628b-ca-trust-extracted\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:51.073694 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.073661    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e6b58763-25f6-42cf-ab80-50b19539628b-registry-certificates\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:51.073791 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.073669    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e6b58763-25f6-42cf-ab80-50b19539628b-trusted-ca\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:51.075286 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.075258    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e6b58763-25f6-42cf-ab80-50b19539628b-registry-tls\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:51.075419 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.075393    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e6b58763-25f6-42cf-ab80-50b19539628b-image-registry-private-configuration\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:51.075607 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.075591    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e6b58763-25f6-42cf-ab80-50b19539628b-installation-pull-secrets\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:51.082295 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.082271    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e6b58763-25f6-42cf-ab80-50b19539628b-bound-sa-token\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:51.082857 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.082835    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tttvw\" (UniqueName: \"kubernetes.io/projected/e6b58763-25f6-42cf-ab80-50b19539628b-kube-api-access-tttvw\") pod \"image-registry-55dcc5fc8-nr27g\" (UID: \"e6b58763-25f6-42cf-ab80-50b19539628b\") " pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:51.084519 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.084502    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9c6c8748d-d92sm"
Jul 28 22:10:51.093816 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.093793    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78869489fc-ztvlr"
Jul 28 22:10:51.099914 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.099891    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k"
Jul 28 22:10:51.197840 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.197791    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:51.251542 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.251452    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9c6c8748d-d92sm"]
Jul 28 22:10:51.255621 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:10:51.255590    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef5b5fc2_7b90_42c9_85ab_b1a56bb82ef2.slice/crio-e71a0d8c17807458fa8f917892d202ee430341093801e90600b18f267d24daab WatchSource:0}: Error finding container e71a0d8c17807458fa8f917892d202ee430341093801e90600b18f267d24daab: Status 404 returned error can't find the container with id e71a0d8c17807458fa8f917892d202ee430341093801e90600b18f267d24daab
Jul 28 22:10:51.354071 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.354000    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55dcc5fc8-nr27g"]
Jul 28 22:10:51.481316 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.481235    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k"]
Jul 28 22:10:51.487818 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.487785    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-78869489fc-ztvlr"]
Jul 28 22:10:51.868153 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:51.868123    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9c6c8748d-d92sm" event={"ID":"ef5b5fc2-7b90-42c9-85ab-b1a56bb82ef2","Type":"ContainerStarted","Data":"e71a0d8c17807458fa8f917892d202ee430341093801e90600b18f267d24daab"}
Jul 28 22:10:52.559543 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:10:52.559490    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0525ef00_1ad4_439f_8eb0_ce327d62940a.slice/crio-342e5561e94fe7847c52d7f16571c2333de232902acf4ba0189bd1f32db31cff WatchSource:0}: Error finding container 342e5561e94fe7847c52d7f16571c2333de232902acf4ba0189bd1f32db31cff: Status 404 returned error can't find the container with id 342e5561e94fe7847c52d7f16571c2333de232902acf4ba0189bd1f32db31cff
Jul 28 22:10:52.560139 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:10:52.560101    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49335e20_dd7c_4f6b_8f47_8cd9ba14f4a3.slice/crio-a1758c65e48c102021876ae0f7850bce14e0bc552aed07af809c5d19ad1a62ca WatchSource:0}: Error finding container a1758c65e48c102021876ae0f7850bce14e0bc552aed07af809c5d19ad1a62ca: Status 404 returned error can't find the container with id a1758c65e48c102021876ae0f7850bce14e0bc552aed07af809c5d19ad1a62ca
Jul 28 22:10:52.560975 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:10:52.560954    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b58763_25f6_42cf_ab80_50b19539628b.slice/crio-b0fd037fb3218916c645b1f76afc96843e416498d19ca1a64fadefab14c7cc97 WatchSource:0}: Error finding container b0fd037fb3218916c645b1f76afc96843e416498d19ca1a64fadefab14c7cc97: Status 404 returned error can't find the container with id b0fd037fb3218916c645b1f76afc96843e416498d19ca1a64fadefab14c7cc97
Jul 28 22:10:52.874588 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:52.874543    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k" event={"ID":"0525ef00-1ad4-439f-8eb0-ce327d62940a","Type":"ContainerStarted","Data":"342e5561e94fe7847c52d7f16571c2333de232902acf4ba0189bd1f32db31cff"}
Jul 28 22:10:52.876335 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:52.876302    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78869489fc-ztvlr" event={"ID":"49335e20-dd7c-4f6b-8f47-8cd9ba14f4a3","Type":"ContainerStarted","Data":"a1758c65e48c102021876ae0f7850bce14e0bc552aed07af809c5d19ad1a62ca"}
Jul 28 22:10:52.877518 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:52.877468    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g" event={"ID":"e6b58763-25f6-42cf-ab80-50b19539628b","Type":"ContainerStarted","Data":"b0fd037fb3218916c645b1f76afc96843e416498d19ca1a64fadefab14c7cc97"}
Jul 28 22:10:53.887894 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:53.887041    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g" event={"ID":"e6b58763-25f6-42cf-ab80-50b19539628b","Type":"ContainerStarted","Data":"ad6babf9da01e7380d1f34149cdada730daa868a6f0bf7ed6e79daa51bf5a13a"}
Jul 28 22:10:53.887894 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:53.887840    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:10:53.893249 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:53.893171    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nmfk9" event={"ID":"511a4d5c-39cf-4a64-971d-1ee43a729c90","Type":"ContainerStarted","Data":"bd644ab5b0685cf278385b455ea65988fcb1beb3b216a63f344d76350b8b3045"}
Jul 28 22:10:54.785883 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:54.784951    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-222qs"
Jul 28 22:10:54.817920 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:54.817230    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g" podStartSLOduration=4.817205948 podStartE2EDuration="4.817205948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:10:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:10:53.909816987 +0000 UTC m=+61.927782685" watchObservedRunningTime="2026-07-28 22:10:54.817205948 +0000 UTC m=+62.835171639"
Jul 28 22:10:58.145062 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:58.144637    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v4p4\" (UniqueName: \"kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4\") pod \"network-check-target-zb7gv\" (UID: \"9a7c5df8-e808-4a53-9697-7dc091ca7660\") " pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:10:58.145062 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:58.144701    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs\") pod \"network-metrics-daemon-k6pcj\" (UID: \"c54b2692-1620-47a1-b04e-7b7f6b5df90a\") " pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:10:58.147500 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:58.147470    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 22:10:58.147649 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:58.147581    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 22:10:58.157239 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:58.157216    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 22:10:58.157769 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:58.157745    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c54b2692-1620-47a1-b04e-7b7f6b5df90a-metrics-certs\") pod \"network-metrics-daemon-k6pcj\" (UID: \"c54b2692-1620-47a1-b04e-7b7f6b5df90a\") " pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:10:58.168173 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:58.168151    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v4p4\" (UniqueName: \"kubernetes.io/projected/9a7c5df8-e808-4a53-9697-7dc091ca7660-kube-api-access-7v4p4\") pod \"network-check-target-zb7gv\" (UID: \"9a7c5df8-e808-4a53-9697-7dc091ca7660\") " pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:10:58.175850 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:58.175818    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-q7z5n\""
Jul 28 22:10:58.181398 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:58.181376    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-7dq7d\""
Jul 28 22:10:58.184864 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:58.184846    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k6pcj"
Jul 28 22:10:58.189991 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:58.189972    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:10:58.245133 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:58.245095    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret\") pod \"global-pull-secret-syncer-7cn74\" (UID: \"be77a0c4-e19b-49d2-b547-c36c3d79dd33\") " pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:10:58.247592 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:58.247570    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 22:10:58.258705 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:58.258681    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/be77a0c4-e19b-49d2-b547-c36c3d79dd33-original-pull-secret\") pod \"global-pull-secret-syncer-7cn74\" (UID: \"be77a0c4-e19b-49d2-b547-c36c3d79dd33\") " pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:10:58.467395 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:58.467322    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7cn74"
Jul 28 22:10:59.866385 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:10:59.866355    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-x8fct"
Jul 28 22:11:00.600228 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:00.599680    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zb7gv"]
Jul 28 22:11:00.609785 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:11:00.609284    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a7c5df8_e808_4a53_9697_7dc091ca7660.slice/crio-7a3924cd833e0ca3d3d8fe226d5edba06340cfa42edb0c84cbf042bcea0ab12a WatchSource:0}: Error finding container 7a3924cd833e0ca3d3d8fe226d5edba06340cfa42edb0c84cbf042bcea0ab12a: Status 404 returned error can't find the container with id 7a3924cd833e0ca3d3d8fe226d5edba06340cfa42edb0c84cbf042bcea0ab12a
Jul 28 22:11:00.618130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:00.618106    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7cn74"]
Jul 28 22:11:00.626931 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:11:00.626900    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe77a0c4_e19b_49d2_b547_c36c3d79dd33.slice/crio-fa87e8a150787c5112d1272b472d2f8d76149b02c5caa440f5f36c4c339355b7 WatchSource:0}: Error finding container fa87e8a150787c5112d1272b472d2f8d76149b02c5caa440f5f36c4c339355b7: Status 404 returned error can't find the container with id fa87e8a150787c5112d1272b472d2f8d76149b02c5caa440f5f36c4c339355b7
Jul 28 22:11:00.642986 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:00.642518    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k6pcj"]
Jul 28 22:11:00.646251 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:11:00.646216    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc54b2692_1620_47a1_b04e_7b7f6b5df90a.slice/crio-c3d4ff3d5622265e0e47fd0eb9434607a01971d6d7e20bfc16b290b824b005c4 WatchSource:0}: Error finding container c3d4ff3d5622265e0e47fd0eb9434607a01971d6d7e20bfc16b290b824b005c4: Status 404 returned error can't find the container with id c3d4ff3d5622265e0e47fd0eb9434607a01971d6d7e20bfc16b290b824b005c4
Jul 28 22:11:00.794239 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:00.794188    2551 patch_prober.go:28] interesting pod/image-registry-7c5dc9fc45-ztdwr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 28 22:11:00.794388 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:00.794255    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr" podUID="50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 22:11:00.924962 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:00.924867    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nmfk9" event={"ID":"511a4d5c-39cf-4a64-971d-1ee43a729c90","Type":"ContainerStarted","Data":"c3e5af36a6b5eec03b865c20631dedaa14fd7a6e030751cc9dfda95d67f6934e"}
Jul 28 22:11:00.926158 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:00.926131    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k6pcj" event={"ID":"c54b2692-1620-47a1-b04e-7b7f6b5df90a","Type":"ContainerStarted","Data":"c3d4ff3d5622265e0e47fd0eb9434607a01971d6d7e20bfc16b290b824b005c4"}
Jul 28 22:11:00.928385 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:00.928350    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k" event={"ID":"0525ef00-1ad4-439f-8eb0-ce327d62940a","Type":"ContainerStarted","Data":"9bbebb6bdf42d6d24f0df6e8d127e19e7c2014bcbc92a868824dbea5de6e6f75"}
Jul 28 22:11:00.928846 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:00.928399    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k" event={"ID":"0525ef00-1ad4-439f-8eb0-ce327d62940a","Type":"ContainerStarted","Data":"5b96ccb6b0b5244b12697780451805813ab94006ec16666ddc96f074fcf23a8b"}
Jul 28 22:11:00.928846 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:00.928417    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k" event={"ID":"0525ef00-1ad4-439f-8eb0-ce327d62940a","Type":"ContainerStarted","Data":"7c5f2a7a1a6afdf26a9ff506d989cec8d32b7cd24f72c382489b3bb5d8cc4e35"}
Jul 28 22:11:00.929831 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:00.929806    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78869489fc-ztvlr" event={"ID":"49335e20-dd7c-4f6b-8f47-8cd9ba14f4a3","Type":"ContainerStarted","Data":"1c684e6de4ff886fa78e1a5e24e61a0d32511b989ecafc0f510c27adaaee10d2"}
Jul 28 22:11:00.930004 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:00.929970    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78869489fc-ztvlr"
Jul 28 22:11:00.930980 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:00.930948    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zb7gv" event={"ID":"9a7c5df8-e808-4a53-9697-7dc091ca7660","Type":"ContainerStarted","Data":"7a3924cd833e0ca3d3d8fe226d5edba06340cfa42edb0c84cbf042bcea0ab12a"}
Jul 28 22:11:00.932088 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:00.932068    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78869489fc-ztvlr"
Jul 28 22:11:00.932297 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:00.932277    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7cn74" event={"ID":"be77a0c4-e19b-49d2-b547-c36c3d79dd33","Type":"ContainerStarted","Data":"fa87e8a150787c5112d1272b472d2f8d76149b02c5caa440f5f36c4c339355b7"}
Jul 28 22:11:00.933588 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:00.933569    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9c6c8748d-d92sm" event={"ID":"ef5b5fc2-7b90-42c9-85ab-b1a56bb82ef2","Type":"ContainerStarted","Data":"8ff083b782a9deea82f0daf643bb7302886dc831b7deeb342aca6952ac4aa280"}
Jul 28 22:11:00.941550 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:00.941493    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nmfk9" podStartSLOduration=17.208986808 podStartE2EDuration="33.941478164s" totalImagesPullingTime="16.732491356s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:10:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:11:00.940844144 +0000 UTC m=+68.958809863" watchObservedRunningTime="2026-07-28 22:11:00.941478164 +0000 UTC m=+68.959443858"
Jul 28 22:11:00.955222 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:00.955171    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9c6c8748d-d92sm" podStartSLOduration=1.785781708 podStartE2EDuration="10.955153506s" totalImagesPullingTime="9.169371798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:10:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:11:00.954406783 +0000 UTC m=+68.972372478" watchObservedRunningTime="2026-07-28 22:11:00.955153506 +0000 UTC m=+68.973119198"
Jul 28 22:11:00.968710 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:00.968669    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-78869489fc-ztvlr" podStartSLOduration=3.090328384 podStartE2EDuration="10.968653932s" totalImagesPullingTime="7.878325548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:10:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:11:00.968384642 +0000 UTC m=+68.986350337" watchObservedRunningTime="2026-07-28 22:11:00.968653932 +0000 UTC m=+68.986619626"
Jul 28 22:11:00.985809 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:00.985743    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-d89464787-dm56k" podStartSLOduration=3.101121857 podStartE2EDuration="10.985728946s" totalImagesPullingTime="7.884607089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:10:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:11:00.985674563 +0000 UTC m=+69.003640258" watchObservedRunningTime="2026-07-28 22:11:00.985728946 +0000 UTC m=+69.003694630"
Jul 28 22:11:02.945047 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:02.944993    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k6pcj" event={"ID":"c54b2692-1620-47a1-b04e-7b7f6b5df90a","Type":"ContainerStarted","Data":"418f47ef43cb8df3812188046460262216ef64dbc26696626e1e8b4a81b5c248"}
Jul 28 22:11:02.945544 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:02.945102    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k6pcj" event={"ID":"c54b2692-1620-47a1-b04e-7b7f6b5df90a","Type":"ContainerStarted","Data":"304f9742044c94ff0461a98c1740f27aef4435d60c167b77d0565bc6fb530244"}
Jul 28 22:11:02.961319 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:02.961252    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-k6pcj" podStartSLOduration=68.341269955 podStartE2EDuration="1m9.96123254s" totalImagesPullingTime="1.619962585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:09:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:11:02.960537813 +0000 UTC m=+70.978503524" watchObservedRunningTime="2026-07-28 22:11:02.96123254 +0000 UTC m=+70.979198238"
Jul 28 22:11:05.957486 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:05.957449    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zb7gv" event={"ID":"9a7c5df8-e808-4a53-9697-7dc091ca7660","Type":"ContainerStarted","Data":"6a278119f8aaa5add8f7b7b032263226f675e988fd03314751d39c970b3a7f6f"}
Jul 28 22:11:05.957954 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:05.957552    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:11:05.958875 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:05.958850    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7cn74" event={"ID":"be77a0c4-e19b-49d2-b547-c36c3d79dd33","Type":"ContainerStarted","Data":"e5d7f2922cfd315f157296a426e19542cf94138bf42ac0f082fcf8b2e63af36b"}
Jul 28 22:11:05.972275 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:05.972232    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zb7gv" podStartSLOduration=68.284840046 podStartE2EDuration="1m12.972218174s" totalImagesPullingTime="4.687378128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:09:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:11:05.971548235 +0000 UTC m=+73.989513930" watchObservedRunningTime="2026-07-28 22:11:05.972218174 +0000 UTC m=+73.990183868"
Jul 28 22:11:10.795044 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:10.795004    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:11:10.816319 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:10.816261    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7cn74" podStartSLOduration=73.144087728 podStartE2EDuration="1m17.81624683s" totalImagesPullingTime="4.672159102s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 22:09:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:11:05.985337736 +0000 UTC m=+74.003303444" watchObservedRunningTime="2026-07-28 22:11:10.81624683 +0000 UTC m=+78.834212566"
Jul 28 22:11:11.025042 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.025004    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-76b4h"]
Jul 28 22:11:11.028718 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.028699    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.031018 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.030973    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 22:11:11.031209 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.031022    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 22:11:11.031340 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.031055    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 22:11:11.031340 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.031068    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 22:11:11.031454 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.031076    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-n8cxc\""
Jul 28 22:11:11.031454 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.031092    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 22:11:11.031454 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.031108    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 22:11:11.148581 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.148550    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed198125-3b0c-47de-b461-a0b8ee9083e6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.148581 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.148586    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ed198125-3b0c-47de-b461-a0b8ee9083e6-node-exporter-accelerators-collector-config\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.148830 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.148614    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ed198125-3b0c-47de-b461-a0b8ee9083e6-root\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.148830 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.148697    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed198125-3b0c-47de-b461-a0b8ee9083e6-metrics-client-ca\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.148830 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.148727    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ed198125-3b0c-47de-b461-a0b8ee9083e6-node-exporter-textfile\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.148830 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.148760    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ed198125-3b0c-47de-b461-a0b8ee9083e6-sys\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.148830 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.148784    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ed198125-3b0c-47de-b461-a0b8ee9083e6-node-exporter-tls\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.148830 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.148815    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ed198125-3b0c-47de-b461-a0b8ee9083e6-node-exporter-wtmp\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.149129 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.148874    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdzk2\" (UniqueName: \"kubernetes.io/projected/ed198125-3b0c-47de-b461-a0b8ee9083e6-kube-api-access-sdzk2\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.249807 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.249772    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed198125-3b0c-47de-b461-a0b8ee9083e6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.249807 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.249808    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ed198125-3b0c-47de-b461-a0b8ee9083e6-node-exporter-accelerators-collector-config\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.250060 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.249828    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ed198125-3b0c-47de-b461-a0b8ee9083e6-root\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.250060 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.249850    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed198125-3b0c-47de-b461-a0b8ee9083e6-metrics-client-ca\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.250060 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.249879    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ed198125-3b0c-47de-b461-a0b8ee9083e6-node-exporter-textfile\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.250060 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.249913    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ed198125-3b0c-47de-b461-a0b8ee9083e6-sys\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.250060 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.249925    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ed198125-3b0c-47de-b461-a0b8ee9083e6-root\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.250060 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.249936    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ed198125-3b0c-47de-b461-a0b8ee9083e6-node-exporter-tls\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.250060 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.249993    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ed198125-3b0c-47de-b461-a0b8ee9083e6-node-exporter-wtmp\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.250060 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.250045    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdzk2\" (UniqueName: \"kubernetes.io/projected/ed198125-3b0c-47de-b461-a0b8ee9083e6-kube-api-access-sdzk2\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.250060 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.250055    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ed198125-3b0c-47de-b461-a0b8ee9083e6-sys\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.250496 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.250206    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ed198125-3b0c-47de-b461-a0b8ee9083e6-node-exporter-wtmp\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.250496 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.250366    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ed198125-3b0c-47de-b461-a0b8ee9083e6-node-exporter-textfile\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.250618 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.250601    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed198125-3b0c-47de-b461-a0b8ee9083e6-metrics-client-ca\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.250670 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.250602    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ed198125-3b0c-47de-b461-a0b8ee9083e6-node-exporter-accelerators-collector-config\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.252263 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.252238    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed198125-3b0c-47de-b461-a0b8ee9083e6-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.252372 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.252336    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ed198125-3b0c-47de-b461-a0b8ee9083e6-node-exporter-tls\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.257864 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.257843    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdzk2\" (UniqueName: \"kubernetes.io/projected/ed198125-3b0c-47de-b461-a0b8ee9083e6-kube-api-access-sdzk2\") pod \"node-exporter-76b4h\" (UID: \"ed198125-3b0c-47de-b461-a0b8ee9083e6\") " pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.339195 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.339157    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-76b4h"
Jul 28 22:11:11.350538 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:11:11.350501    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded198125_3b0c_47de_b461_a0b8ee9083e6.slice/crio-3b82b7b0cffb4365b5f6c6a2b4cccfad5e9909926191645b847d3031934ca036 WatchSource:0}: Error finding container 3b82b7b0cffb4365b5f6c6a2b4cccfad5e9909926191645b847d3031934ca036: Status 404 returned error can't find the container with id 3b82b7b0cffb4365b5f6c6a2b4cccfad5e9909926191645b847d3031934ca036
Jul 28 22:11:11.983358 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:11.983317    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-76b4h" event={"ID":"ed198125-3b0c-47de-b461-a0b8ee9083e6","Type":"ContainerStarted","Data":"3b82b7b0cffb4365b5f6c6a2b4cccfad5e9909926191645b847d3031934ca036"}
Jul 28 22:11:12.988273 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:12.988233    2551 generic.go:356] "Generic (PLEG): container finished" podID="ed198125-3b0c-47de-b461-a0b8ee9083e6" containerID="df13405ef82b08d507d1a8e91990930d638d0c9377efeb3cd8273ada5a12033c" exitCode=0
Jul 28 22:11:12.988679 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:12.988290    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-76b4h" event={"ID":"ed198125-3b0c-47de-b461-a0b8ee9083e6","Type":"ContainerDied","Data":"df13405ef82b08d507d1a8e91990930d638d0c9377efeb3cd8273ada5a12033c"}
Jul 28 22:11:13.993975 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:13.993936    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-76b4h" event={"ID":"ed198125-3b0c-47de-b461-a0b8ee9083e6","Type":"ContainerStarted","Data":"32d8ea12f3e0b115e7db6a405e6f688c5555d147155579411d40da6e8529244d"}
Jul 28 22:11:13.994341 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:13.994001    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-76b4h" event={"ID":"ed198125-3b0c-47de-b461-a0b8ee9083e6","Type":"ContainerStarted","Data":"33f55af5398e89d02212d76a9f84498d8f983fd1d7f7f417c69b0cbb47f22c25"}
Jul 28 22:11:14.011865 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:14.011817    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-76b4h" podStartSLOduration=1.99789284 podStartE2EDuration="3.011800701s" totalImagesPullingTime="950.226998ms" totalInitContainerRuntime="63.680863ms" isStatefulPod=true podCreationTimestamp="2026-07-28 22:11:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:11:14.01033433 +0000 UTC m=+82.028300057" watchObservedRunningTime="2026-07-28 22:11:14.011800701 +0000 UTC m=+82.029766394"
Jul 28 22:11:15.809797 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:15.809731    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr" podUID="50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe" containerName="registry" containerID="cri-o://7c243e4a6fc4460558fbf3a6b4ab9e8f755b24dc766dbb65dcaef45c51341770" gracePeriod=30
Jul 28 22:11:15.909722 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:15.909691    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-55dcc5fc8-nr27g"
Jul 28 22:11:16.003127 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.003098    2551 generic.go:356] "Generic (PLEG): container finished" podID="50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe" containerID="7c243e4a6fc4460558fbf3a6b4ab9e8f755b24dc766dbb65dcaef45c51341770" exitCode=0
Jul 28 22:11:16.003251 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.003168    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr" event={"ID":"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe","Type":"ContainerDied","Data":"7c243e4a6fc4460558fbf3a6b4ab9e8f755b24dc766dbb65dcaef45c51341770"}
Jul 28 22:11:16.045712 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.045691    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:11:16.188285 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.188251    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-ca-trust-extracted\") pod \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") "
Jul 28 22:11:16.188485 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.188298    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-image-registry-private-configuration\") pod \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") "
Jul 28 22:11:16.188485 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.188322    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9zl5\" (UniqueName: \"kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-kube-api-access-h9zl5\") pod \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") "
Jul 28 22:11:16.188485 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.188338    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-bound-sa-token\") pod \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") "
Jul 28 22:11:16.188485 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.188360    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-certificates\") pod \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") "
Jul 28 22:11:16.188485 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.188391    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-tls\") pod \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") "
Jul 28 22:11:16.188485 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.188424    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-installation-pull-secrets\") pod \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") "
Jul 28 22:11:16.188485 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.188454    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-trusted-ca\") pod \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\" (UID: \"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe\") "
Jul 28 22:11:16.189137 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.189109    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe" (UID: "50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:11:16.189225 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.189174    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe" (UID: "50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:11:16.190916 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.190867    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-kube-api-access-h9zl5" (OuterVolumeSpecName: "kube-api-access-h9zl5") pod "50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe" (UID: "50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe"). InnerVolumeSpecName "kube-api-access-h9zl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:11:16.191244 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.191198    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe" (UID: "50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:11:16.191339 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.191234    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe" (UID: "50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:11:16.191339 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.191276    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe" (UID: "50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:11:16.191421 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.191333    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe" (UID: "50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:11:16.196663 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.196638    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe" (UID: "50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:11:16.289098 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.289060    2551 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-certificates\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:11:16.289098 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.289090    2551 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-registry-tls\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:11:16.289098 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.289104    2551 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-installation-pull-secrets\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:11:16.289351 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.289116    2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-trusted-ca\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:11:16.289351 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.289128    2551 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-ca-trust-extracted\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:11:16.289351 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.289140    2551 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-image-registry-private-configuration\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:11:16.289351 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.289152    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9zl5\" (UniqueName: \"kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-kube-api-access-h9zl5\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:11:16.289351 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:16.289163    2551 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe-bound-sa-token\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:11:17.007813 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:17.007779    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"
Jul 28 22:11:17.008295 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:17.007780    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c5dc9fc45-ztdwr" event={"ID":"50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe","Type":"ContainerDied","Data":"c055342b634d2cdbc18ad167708ba570fb7b102bb907305c0d38b4f9a7caa543"}
Jul 28 22:11:17.008295 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:17.007915    2551 scope.go:117] "RemoveContainer" containerID="7c243e4a6fc4460558fbf3a6b4ab9e8f755b24dc766dbb65dcaef45c51341770"
Jul 28 22:11:17.031555 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:17.031502    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"]
Jul 28 22:11:17.033506 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:17.033484    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7c5dc9fc45-ztdwr"]
Jul 28 22:11:18.460853 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:18.460816    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe" path="/var/lib/kubelet/pods/50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe/volumes"
Jul 28 22:11:36.964938 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:11:36.964908    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zb7gv"
Jul 28 22:14:28.060539 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.060486    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x"]
Jul 28 22:14:28.061009 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.060757    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe" containerName="registry"
Jul 28 22:14:28.061009 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.060769    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe" containerName="registry"
Jul 28 22:14:28.061009 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.060814    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="50d5ff4d-4f55-4d51-9bb9-b6d1233a1cfe" containerName="registry"
Jul 28 22:14:28.063664 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.063647    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x"
Jul 28 22:14:28.067448 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.067425    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 22:14:28.068335 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.068315    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 22:14:28.068399 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.068386    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 22:14:28.068490 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.068475    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zwhdk\""
Jul 28 22:14:28.068574 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.068484    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 22:14:28.088322 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.088295    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x"]
Jul 28 22:14:28.137259 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.137219    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1f196092-c49f-41f1-ac37-b92510ba4cd9-manifests\") pod \"opendatahub-operator-controller-manager-7b7ccc5769-hcb2x\" (UID: \"1f196092-c49f-41f1-ac37-b92510ba4cd9\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x"
Jul 28 22:14:28.137259 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.137265    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92ngf\" (UniqueName: \"kubernetes.io/projected/1f196092-c49f-41f1-ac37-b92510ba4cd9-kube-api-access-92ngf\") pod \"opendatahub-operator-controller-manager-7b7ccc5769-hcb2x\" (UID: \"1f196092-c49f-41f1-ac37-b92510ba4cd9\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x"
Jul 28 22:14:28.137472 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.137328    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1f196092-c49f-41f1-ac37-b92510ba4cd9-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b7ccc5769-hcb2x\" (UID: \"1f196092-c49f-41f1-ac37-b92510ba4cd9\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x"
Jul 28 22:14:28.137472 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.137371    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f196092-c49f-41f1-ac37-b92510ba4cd9-tmp\") pod \"opendatahub-operator-controller-manager-7b7ccc5769-hcb2x\" (UID: \"1f196092-c49f-41f1-ac37-b92510ba4cd9\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x"
Jul 28 22:14:28.137472 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.137447    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1f196092-c49f-41f1-ac37-b92510ba4cd9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b7ccc5769-hcb2x\" (UID: \"1f196092-c49f-41f1-ac37-b92510ba4cd9\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x"
Jul 28 22:14:28.238170 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.238131    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1f196092-c49f-41f1-ac37-b92510ba4cd9-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b7ccc5769-hcb2x\" (UID: \"1f196092-c49f-41f1-ac37-b92510ba4cd9\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x"
Jul 28 22:14:28.238170 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.238174    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f196092-c49f-41f1-ac37-b92510ba4cd9-tmp\") pod \"opendatahub-operator-controller-manager-7b7ccc5769-hcb2x\" (UID: \"1f196092-c49f-41f1-ac37-b92510ba4cd9\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x"
Jul 28 22:14:28.238365 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.238218    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1f196092-c49f-41f1-ac37-b92510ba4cd9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b7ccc5769-hcb2x\" (UID: \"1f196092-c49f-41f1-ac37-b92510ba4cd9\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x"
Jul 28 22:14:28.238365 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.238237    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1f196092-c49f-41f1-ac37-b92510ba4cd9-manifests\") pod \"opendatahub-operator-controller-manager-7b7ccc5769-hcb2x\" (UID: \"1f196092-c49f-41f1-ac37-b92510ba4cd9\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x"
Jul 28 22:14:28.238365 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.238259    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92ngf\" (UniqueName: \"kubernetes.io/projected/1f196092-c49f-41f1-ac37-b92510ba4cd9-kube-api-access-92ngf\") pod \"opendatahub-operator-controller-manager-7b7ccc5769-hcb2x\" (UID: \"1f196092-c49f-41f1-ac37-b92510ba4cd9\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x"
Jul 28 22:14:28.238717 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.238699    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1f196092-c49f-41f1-ac37-b92510ba4cd9-manifests\") pod \"opendatahub-operator-controller-manager-7b7ccc5769-hcb2x\" (UID: \"1f196092-c49f-41f1-ac37-b92510ba4cd9\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x"
Jul 28 22:14:28.240582 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.240560    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f196092-c49f-41f1-ac37-b92510ba4cd9-tmp\") pod \"opendatahub-operator-controller-manager-7b7ccc5769-hcb2x\" (UID: \"1f196092-c49f-41f1-ac37-b92510ba4cd9\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x"
Jul 28 22:14:28.240699 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.240659    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1f196092-c49f-41f1-ac37-b92510ba4cd9-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b7ccc5769-hcb2x\" (UID: \"1f196092-c49f-41f1-ac37-b92510ba4cd9\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x"
Jul 28 22:14:28.240762 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.240745    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1f196092-c49f-41f1-ac37-b92510ba4cd9-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b7ccc5769-hcb2x\" (UID: \"1f196092-c49f-41f1-ac37-b92510ba4cd9\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x"
Jul 28 22:14:28.248382 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.248359    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92ngf\" (UniqueName: \"kubernetes.io/projected/1f196092-c49f-41f1-ac37-b92510ba4cd9-kube-api-access-92ngf\") pod \"opendatahub-operator-controller-manager-7b7ccc5769-hcb2x\" (UID: \"1f196092-c49f-41f1-ac37-b92510ba4cd9\") " pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x"
Jul 28 22:14:28.373342 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.373310    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x"
Jul 28 22:14:28.509385 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.509351    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x"]
Jul 28 22:14:28.511934 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:14:28.511899    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f196092_c49f_41f1_ac37_b92510ba4cd9.slice/crio-d9eb611f68644d345f99527c48e466112e04a0f055987efeb307ebd78f1ed44e WatchSource:0}: Error finding container d9eb611f68644d345f99527c48e466112e04a0f055987efeb307ebd78f1ed44e: Status 404 returned error can't find the container with id d9eb611f68644d345f99527c48e466112e04a0f055987efeb307ebd78f1ed44e
Jul 28 22:14:28.625896 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:28.625811    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x" event={"ID":"1f196092-c49f-41f1-ac37-b92510ba4cd9","Type":"ContainerStarted","Data":"d9eb611f68644d345f99527c48e466112e04a0f055987efeb307ebd78f1ed44e"}
Jul 28 22:14:32.642364 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:32.642328    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x" event={"ID":"1f196092-c49f-41f1-ac37-b92510ba4cd9","Type":"ContainerStarted","Data":"5fdcee7a0bce566c6043e1ce33537275f16b7041f37b577042d3bf78e9b44433"}
Jul 28 22:14:33.646475 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:33.646442    2551 generic.go:356] "Generic (PLEG): container finished" podID="1f196092-c49f-41f1-ac37-b92510ba4cd9" containerID="5fdcee7a0bce566c6043e1ce33537275f16b7041f37b577042d3bf78e9b44433" exitCode=0
Jul 28 22:14:33.646986 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:33.646490    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x" event={"ID":"1f196092-c49f-41f1-ac37-b92510ba4cd9","Type":"ContainerDied","Data":"5fdcee7a0bce566c6043e1ce33537275f16b7041f37b577042d3bf78e9b44433"}
Jul 28 22:14:34.651990 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:34.651952    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x" event={"ID":"1f196092-c49f-41f1-ac37-b92510ba4cd9","Type":"ContainerStarted","Data":"abcb08df2b7166c81eec618f4d59847a2005c97d941bf71a83895fe2bb60e82c"}
Jul 28 22:14:34.652464 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:34.652136    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x"
Jul 28 22:14:34.672773 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:34.672729    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x" podStartSLOduration=1.967592824 podStartE2EDuration="6.672713229s" totalImagesPullingTime="3.673964524s" totalInitContainerRuntime="1.031155881s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:14:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:14:34.671379062 +0000 UTC m=+282.689344765" watchObservedRunningTime="2026-07-28 22:14:34.672713229 +0000 UTC m=+282.690678922"
Jul 28 22:14:45.657598 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:45.657566    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7b7ccc5769-hcb2x"
Jul 28 22:14:49.090595 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.090558    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5d4df5cb96-q778h"]
Jul 28 22:14:49.095420 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.095396    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5d4df5cb96-q778h"
Jul 28 22:14:49.097933 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.097913    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 22:14:49.097933 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.097927    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 22:14:49.098158 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.098039    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 28 22:14:49.098158 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.098043    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 22:14:49.108058 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.108035    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5d4df5cb96-q778h"]
Jul 28 22:14:49.195266 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.195229    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fe78ca4-ba3e-4377-accc-d2056694218b-tmp\") pod \"kube-auth-proxy-5d4df5cb96-q778h\" (UID: \"6fe78ca4-ba3e-4377-accc-d2056694218b\") " pod="openshift-ingress/kube-auth-proxy-5d4df5cb96-q778h"
Jul 28 22:14:49.195430 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.195269    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hl2g\" (UniqueName: \"kubernetes.io/projected/6fe78ca4-ba3e-4377-accc-d2056694218b-kube-api-access-7hl2g\") pod \"kube-auth-proxy-5d4df5cb96-q778h\" (UID: \"6fe78ca4-ba3e-4377-accc-d2056694218b\") " pod="openshift-ingress/kube-auth-proxy-5d4df5cb96-q778h"
Jul 28 22:14:49.195430 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.195362    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fe78ca4-ba3e-4377-accc-d2056694218b-tls-certs\") pod \"kube-auth-proxy-5d4df5cb96-q778h\" (UID: \"6fe78ca4-ba3e-4377-accc-d2056694218b\") " pod="openshift-ingress/kube-auth-proxy-5d4df5cb96-q778h"
Jul 28 22:14:49.223855 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.223818    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-lfcxw"]
Jul 28 22:14:49.226240 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.226219    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-lfcxw"
Jul 28 22:14:49.231965 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.231944    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-j5nlw\""
Jul 28 22:14:49.232400 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.232381    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 28 22:14:49.242924 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.242900    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-lfcxw"]
Jul 28 22:14:49.296333 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.296290    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fe78ca4-ba3e-4377-accc-d2056694218b-tmp\") pod \"kube-auth-proxy-5d4df5cb96-q778h\" (UID: \"6fe78ca4-ba3e-4377-accc-d2056694218b\") " pod="openshift-ingress/kube-auth-proxy-5d4df5cb96-q778h"
Jul 28 22:14:49.296333 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.296334    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hl2g\" (UniqueName: \"kubernetes.io/projected/6fe78ca4-ba3e-4377-accc-d2056694218b-kube-api-access-7hl2g\") pod \"kube-auth-proxy-5d4df5cb96-q778h\" (UID: \"6fe78ca4-ba3e-4377-accc-d2056694218b\") " pod="openshift-ingress/kube-auth-proxy-5d4df5cb96-q778h"
Jul 28 22:14:49.296612 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.296363    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fe78ca4-ba3e-4377-accc-d2056694218b-tls-certs\") pod \"kube-auth-proxy-5d4df5cb96-q778h\" (UID: \"6fe78ca4-ba3e-4377-accc-d2056694218b\") " pod="openshift-ingress/kube-auth-proxy-5d4df5cb96-q778h"
Jul 28 22:14:49.296612 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.296391    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/220c022c-101e-4f2e-8bfc-79514b31a768-cert\") pod \"odh-model-controller-7998c67dc4-lfcxw\" (UID: \"220c022c-101e-4f2e-8bfc-79514b31a768\") " pod="opendatahub/odh-model-controller-7998c67dc4-lfcxw"
Jul 28 22:14:49.296612 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.296418    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlf47\" (UniqueName: \"kubernetes.io/projected/220c022c-101e-4f2e-8bfc-79514b31a768-kube-api-access-qlf47\") pod \"odh-model-controller-7998c67dc4-lfcxw\" (UID: \"220c022c-101e-4f2e-8bfc-79514b31a768\") " pod="opendatahub/odh-model-controller-7998c67dc4-lfcxw"
Jul 28 22:14:49.298734 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.298705    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6fe78ca4-ba3e-4377-accc-d2056694218b-tmp\") pod \"kube-auth-proxy-5d4df5cb96-q778h\" (UID: \"6fe78ca4-ba3e-4377-accc-d2056694218b\") " pod="openshift-ingress/kube-auth-proxy-5d4df5cb96-q778h"
Jul 28 22:14:49.298849 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.298828    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fe78ca4-ba3e-4377-accc-d2056694218b-tls-certs\") pod \"kube-auth-proxy-5d4df5cb96-q778h\" (UID: \"6fe78ca4-ba3e-4377-accc-d2056694218b\") " pod="openshift-ingress/kube-auth-proxy-5d4df5cb96-q778h"
Jul 28 22:14:49.306861 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.306837    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hl2g\" (UniqueName: \"kubernetes.io/projected/6fe78ca4-ba3e-4377-accc-d2056694218b-kube-api-access-7hl2g\") pod \"kube-auth-proxy-5d4df5cb96-q778h\" (UID: \"6fe78ca4-ba3e-4377-accc-d2056694218b\") " pod="openshift-ingress/kube-auth-proxy-5d4df5cb96-q778h"
Jul 28 22:14:49.397302 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.397208    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/220c022c-101e-4f2e-8bfc-79514b31a768-cert\") pod \"odh-model-controller-7998c67dc4-lfcxw\" (UID: \"220c022c-101e-4f2e-8bfc-79514b31a768\") " pod="opendatahub/odh-model-controller-7998c67dc4-lfcxw"
Jul 28 22:14:49.397302 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.397262    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlf47\" (UniqueName: \"kubernetes.io/projected/220c022c-101e-4f2e-8bfc-79514b31a768-kube-api-access-qlf47\") pod \"odh-model-controller-7998c67dc4-lfcxw\" (UID: \"220c022c-101e-4f2e-8bfc-79514b31a768\") " pod="opendatahub/odh-model-controller-7998c67dc4-lfcxw"
Jul 28 22:14:49.397516 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:14:49.397359    2551 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 28 22:14:49.397516 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:14:49.397426    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/220c022c-101e-4f2e-8bfc-79514b31a768-cert podName:220c022c-101e-4f2e-8bfc-79514b31a768 nodeName:}" failed. No retries permitted until 2026-07-28 22:14:49.89740806 +0000 UTC m=+297.915373734 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/220c022c-101e-4f2e-8bfc-79514b31a768-cert") pod "odh-model-controller-7998c67dc4-lfcxw" (UID: "220c022c-101e-4f2e-8bfc-79514b31a768") : secret "odh-model-controller-webhook-cert" not found
Jul 28 22:14:49.405167 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.405147    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5d4df5cb96-q778h"
Jul 28 22:14:49.412586 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.412554    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlf47\" (UniqueName: \"kubernetes.io/projected/220c022c-101e-4f2e-8bfc-79514b31a768-kube-api-access-qlf47\") pod \"odh-model-controller-7998c67dc4-lfcxw\" (UID: \"220c022c-101e-4f2e-8bfc-79514b31a768\") " pod="opendatahub/odh-model-controller-7998c67dc4-lfcxw"
Jul 28 22:14:49.556795 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.556706    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5d4df5cb96-q778h"]
Jul 28 22:14:49.561450 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:14:49.561417    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fe78ca4_ba3e_4377_accc_d2056694218b.slice/crio-a6e50262d6381fd2d945ee5729116f37a254f299a2c1550302aae073afb4d576 WatchSource:0}: Error finding container a6e50262d6381fd2d945ee5729116f37a254f299a2c1550302aae073afb4d576: Status 404 returned error can't find the container with id a6e50262d6381fd2d945ee5729116f37a254f299a2c1550302aae073afb4d576
Jul 28 22:14:49.703066 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.702976    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5d4df5cb96-q778h" event={"ID":"6fe78ca4-ba3e-4377-accc-d2056694218b","Type":"ContainerStarted","Data":"a6e50262d6381fd2d945ee5729116f37a254f299a2c1550302aae073afb4d576"}
Jul 28 22:14:49.900942 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.900903    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/220c022c-101e-4f2e-8bfc-79514b31a768-cert\") pod \"odh-model-controller-7998c67dc4-lfcxw\" (UID: \"220c022c-101e-4f2e-8bfc-79514b31a768\") " pod="opendatahub/odh-model-controller-7998c67dc4-lfcxw"
Jul 28 22:14:49.903226 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:49.903206    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/220c022c-101e-4f2e-8bfc-79514b31a768-cert\") pod \"odh-model-controller-7998c67dc4-lfcxw\" (UID: \"220c022c-101e-4f2e-8bfc-79514b31a768\") " pod="opendatahub/odh-model-controller-7998c67dc4-lfcxw"
Jul 28 22:14:50.137196 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.137162    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-lfcxw"
Jul 28 22:14:50.220281 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.219065    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-w2q5c"]
Jul 28 22:14:50.222273 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.221555    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-w2q5c"
Jul 28 22:14:50.224192 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.224156    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 28 22:14:50.245408 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.244837    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-w2q5c"]
Jul 28 22:14:50.297241 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.297197    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-lfcxw"]
Jul 28 22:14:50.302675 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:14:50.302444    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod220c022c_101e_4f2e_8bfc_79514b31a768.slice/crio-ac4012479f458b862aac52c4e3c3b570c193a230aa3b75dd3e88437011a3c075 WatchSource:0}: Error finding container ac4012479f458b862aac52c4e3c3b570c193a230aa3b75dd3e88437011a3c075: Status 404 returned error can't find the container with id ac4012479f458b862aac52c4e3c3b570c193a230aa3b75dd3e88437011a3c075
Jul 28 22:14:50.303971 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.303944    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkzlw\" (UniqueName: \"kubernetes.io/projected/32913f56-4ade-44a2-a3ff-c137229ddea5-kube-api-access-hkzlw\") pod \"model-serving-api-54d89bd5fc-w2q5c\" (UID: \"32913f56-4ade-44a2-a3ff-c137229ddea5\") " pod="opendatahub/model-serving-api-54d89bd5fc-w2q5c"
Jul 28 22:14:50.304093 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.304064    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32913f56-4ade-44a2-a3ff-c137229ddea5-tls-certs\") pod \"model-serving-api-54d89bd5fc-w2q5c\" (UID: \"32913f56-4ade-44a2-a3ff-c137229ddea5\") " pod="opendatahub/model-serving-api-54d89bd5fc-w2q5c"
Jul 28 22:14:50.405520 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.405428    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32913f56-4ade-44a2-a3ff-c137229ddea5-tls-certs\") pod \"model-serving-api-54d89bd5fc-w2q5c\" (UID: \"32913f56-4ade-44a2-a3ff-c137229ddea5\") " pod="opendatahub/model-serving-api-54d89bd5fc-w2q5c"
Jul 28 22:14:50.405702 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.405520    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkzlw\" (UniqueName: \"kubernetes.io/projected/32913f56-4ade-44a2-a3ff-c137229ddea5-kube-api-access-hkzlw\") pod \"model-serving-api-54d89bd5fc-w2q5c\" (UID: \"32913f56-4ade-44a2-a3ff-c137229ddea5\") " pod="opendatahub/model-serving-api-54d89bd5fc-w2q5c"
Jul 28 22:14:50.405702 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:14:50.405620    2551 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 28 22:14:50.405808 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:14:50.405706    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32913f56-4ade-44a2-a3ff-c137229ddea5-tls-certs podName:32913f56-4ade-44a2-a3ff-c137229ddea5 nodeName:}" failed. No retries permitted until 2026-07-28 22:14:50.905682318 +0000 UTC m=+298.923647993 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/32913f56-4ade-44a2-a3ff-c137229ddea5-tls-certs") pod "model-serving-api-54d89bd5fc-w2q5c" (UID: "32913f56-4ade-44a2-a3ff-c137229ddea5") : secret "model-serving-api-tls" not found
Jul 28 22:14:50.415781 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.415752    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkzlw\" (UniqueName: \"kubernetes.io/projected/32913f56-4ade-44a2-a3ff-c137229ddea5-kube-api-access-hkzlw\") pod \"model-serving-api-54d89bd5fc-w2q5c\" (UID: \"32913f56-4ade-44a2-a3ff-c137229ddea5\") " pod="opendatahub/model-serving-api-54d89bd5fc-w2q5c"
Jul 28 22:14:50.481343 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.481304    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"]
Jul 28 22:14:50.484435 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.484414    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:50.486361 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.486338    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 28 22:14:50.486454 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.486385    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 28 22:14:50.486648 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.486629    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 28 22:14:50.486882 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.486848    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-g2wcz\""
Jul 28 22:14:50.486992 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.486906    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 28 22:14:50.497599 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.497573    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"]
Jul 28 22:14:50.608356 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.608325    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-9rkqc\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:50.608356 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.608370    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-9rkqc\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:50.608611 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.608425    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd805c57-e107-4ca5-8dd4-254ba9dd8587-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-9rkqc\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:50.608611 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.608440    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fd805c57-e107-4ca5-8dd4-254ba9dd8587-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-9rkqc\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:50.608611 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.608459    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4hbz\" (UniqueName: \"kubernetes.io/projected/fd805c57-e107-4ca5-8dd4-254ba9dd8587-kube-api-access-b4hbz\") pod \"istiod-openshift-gateway-7b7995d5cd-9rkqc\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:50.608611 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.608553    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-9rkqc\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:50.608611 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.608600    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-9rkqc\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:50.709145 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.709055    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-9rkqc\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:50.709145 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.709121    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-9rkqc\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:50.709374 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.709155    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd805c57-e107-4ca5-8dd4-254ba9dd8587-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-9rkqc\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:50.709374 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.709177    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fd805c57-e107-4ca5-8dd4-254ba9dd8587-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-9rkqc\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:50.709374 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.709199    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4hbz\" (UniqueName: \"kubernetes.io/projected/fd805c57-e107-4ca5-8dd4-254ba9dd8587-kube-api-access-b4hbz\") pod \"istiod-openshift-gateway-7b7995d5cd-9rkqc\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:50.709374 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.709251    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-9rkqc\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:50.709374 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.709283    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-9rkqc\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:50.710448 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.710417    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-9rkqc\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:50.710655 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.710616    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-lfcxw" event={"ID":"220c022c-101e-4f2e-8bfc-79514b31a768","Type":"ContainerStarted","Data":"ac4012479f458b862aac52c4e3c3b570c193a230aa3b75dd3e88437011a3c075"}
Jul 28 22:14:50.712181 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.712157    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-9rkqc\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:50.712918 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.712895    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd805c57-e107-4ca5-8dd4-254ba9dd8587-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-9rkqc\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:50.713171 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.713152    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fd805c57-e107-4ca5-8dd4-254ba9dd8587-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-9rkqc\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:50.713547 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.713512    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-9rkqc\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:50.718077 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.718052    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-9rkqc\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:50.718715 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.718673    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4hbz\" (UniqueName: \"kubernetes.io/projected/fd805c57-e107-4ca5-8dd4-254ba9dd8587-kube-api-access-b4hbz\") pod \"istiod-openshift-gateway-7b7995d5cd-9rkqc\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:50.795262 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.795220    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:50.911347 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.911314    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32913f56-4ade-44a2-a3ff-c137229ddea5-tls-certs\") pod \"model-serving-api-54d89bd5fc-w2q5c\" (UID: \"32913f56-4ade-44a2-a3ff-c137229ddea5\") " pod="opendatahub/model-serving-api-54d89bd5fc-w2q5c"
Jul 28 22:14:50.911493 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:14:50.911454    2551 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 28 22:14:50.911578 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:14:50.911516    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32913f56-4ade-44a2-a3ff-c137229ddea5-tls-certs podName:32913f56-4ade-44a2-a3ff-c137229ddea5 nodeName:}" failed. No retries permitted until 2026-07-28 22:14:51.911497055 +0000 UTC m=+299.929462730 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/32913f56-4ade-44a2-a3ff-c137229ddea5-tls-certs") pod "model-serving-api-54d89bd5fc-w2q5c" (UID: "32913f56-4ade-44a2-a3ff-c137229ddea5") : secret "model-serving-api-tls" not found
Jul 28 22:14:50.967485 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:50.967347    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"]
Jul 28 22:14:50.970748 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:14:50.970718    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd805c57_e107_4ca5_8dd4_254ba9dd8587.slice/crio-7fc01db37e549a16186ff7ff11a28d1ff0d6128a51c242fb291fa18139c6a2e4 WatchSource:0}: Error finding container 7fc01db37e549a16186ff7ff11a28d1ff0d6128a51c242fb291fa18139c6a2e4: Status 404 returned error can't find the container with id 7fc01db37e549a16186ff7ff11a28d1ff0d6128a51c242fb291fa18139c6a2e4
Jul 28 22:14:51.719156 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:51.719068    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc" event={"ID":"fd805c57-e107-4ca5-8dd4-254ba9dd8587","Type":"ContainerStarted","Data":"7fc01db37e549a16186ff7ff11a28d1ff0d6128a51c242fb291fa18139c6a2e4"}
Jul 28 22:14:51.920909 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:51.920870    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32913f56-4ade-44a2-a3ff-c137229ddea5-tls-certs\") pod \"model-serving-api-54d89bd5fc-w2q5c\" (UID: \"32913f56-4ade-44a2-a3ff-c137229ddea5\") " pod="opendatahub/model-serving-api-54d89bd5fc-w2q5c"
Jul 28 22:14:51.924930 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:51.924889    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32913f56-4ade-44a2-a3ff-c137229ddea5-tls-certs\") pod \"model-serving-api-54d89bd5fc-w2q5c\" (UID: \"32913f56-4ade-44a2-a3ff-c137229ddea5\") " pod="opendatahub/model-serving-api-54d89bd5fc-w2q5c"
Jul 28 22:14:52.047266 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:52.047173    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-w2q5c"
Jul 28 22:14:54.846415 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:54.846384    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 28 22:14:54.876574 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:54.876544    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:14:54.877075 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:54.877054    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:14:54.882159 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:54.882131    2551 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 22:14:54.924330 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:54.923977    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 22:14:54.924330 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:54.924042    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 22:14:55.020156 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:55.020041    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-w2q5c"]
Jul 28 22:14:55.025864 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:14:55.025819    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32913f56_4ade_44a2_a3ff_c137229ddea5.slice/crio-ca6bbd48acd01f147dbeb83473d93f91251482b073bca5b19e432ec61aa8fdc1 WatchSource:0}: Error finding container ca6bbd48acd01f147dbeb83473d93f91251482b073bca5b19e432ec61aa8fdc1: Status 404 returned error can't find the container with id ca6bbd48acd01f147dbeb83473d93f91251482b073bca5b19e432ec61aa8fdc1
Jul 28 22:14:55.028698 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:55.028394    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 22:14:55.737945 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:55.737901    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-lfcxw" event={"ID":"220c022c-101e-4f2e-8bfc-79514b31a768","Type":"ContainerStarted","Data":"f66d99e4dd4f127dbe2ef97804802b5081b869180c75cd478fc40676a1085e45"}
Jul 28 22:14:55.738134 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:55.737990    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-lfcxw"
Jul 28 22:14:55.739518 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:55.739492    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5d4df5cb96-q778h" event={"ID":"6fe78ca4-ba3e-4377-accc-d2056694218b","Type":"ContainerStarted","Data":"b3a070864399d9cca5c0119a8781b2042d171067ab3ab6c5fb2cc9d9a57c7d01"}
Jul 28 22:14:55.741138 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:55.741100    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc" event={"ID":"fd805c57-e107-4ca5-8dd4-254ba9dd8587","Type":"ContainerStarted","Data":"f8719fb3180827292cc4742c314be835b5ee80052f5e04858f08b3c75c061ffe"}
Jul 28 22:14:55.741295 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:55.741159    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:55.742500 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:55.742476    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-w2q5c" event={"ID":"32913f56-4ade-44a2-a3ff-c137229ddea5","Type":"ContainerStarted","Data":"ca6bbd48acd01f147dbeb83473d93f91251482b073bca5b19e432ec61aa8fdc1"}
Jul 28 22:14:55.754793 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:55.754707    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-lfcxw" podStartSLOduration=2.194145129 podStartE2EDuration="6.754690239s" totalImagesPullingTime="4.56054511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:14:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:14:55.754462092 +0000 UTC m=+303.772427815" watchObservedRunningTime="2026-07-28 22:14:55.754690239 +0000 UTC m=+303.772655934"
Jul 28 22:14:55.769751 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:55.769689    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5d4df5cb96-q778h" podStartSLOduration=1.48931499 podStartE2EDuration="6.769668302s" totalImagesPullingTime="5.280353312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:14:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:14:55.769064143 +0000 UTC m=+303.787029837" watchObservedRunningTime="2026-07-28 22:14:55.769668302 +0000 UTC m=+303.787633996"
Jul 28 22:14:55.789553 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:55.789269    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc" podStartSLOduration=1.8395233819999999 podStartE2EDuration="5.789246316s" totalImagesPullingTime="3.949722934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:14:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:14:55.787682639 +0000 UTC m=+303.805648390" watchObservedRunningTime="2026-07-28 22:14:55.789246316 +0000 UTC m=+303.807212010"
Jul 28 22:14:56.557407 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.557359    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"]
Jul 28 22:14:56.564136 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.564100    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.567095 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.567068    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-7n8wj\""
Jul 28 22:14:56.573667 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.573643    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"]
Jul 28 22:14:56.666345 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.666306    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.666506 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.666406    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.666506 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.666452    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.666506 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.666479    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.666506 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.666502    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7gbm\" (UniqueName: \"kubernetes.io/projected/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-kube-api-access-l7gbm\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.666718 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.666544    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.666718 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.666606    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.666718 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.666648    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.666718 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.666668    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.748953 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.748930    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:14:56.767448 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.767414    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.767571 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.767460    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.767571 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.767493    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.767571 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.767516    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7gbm\" (UniqueName: \"kubernetes.io/projected/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-kube-api-access-l7gbm\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.767571 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.767560    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.767807 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.767595    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.767807 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.767613    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.767807 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.767636    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.767807 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.767683    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.768192 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.768058    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.768290 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.768212    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.768290 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.768275    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.768404 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.768300    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.768842 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.768820    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.770814 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.770793    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.771142 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.771119    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.781761 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.781718    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.786034 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.786000    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7gbm\" (UniqueName: \"kubernetes.io/projected/b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8-kube-api-access-l7gbm\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6\" (UID: \"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:56.881190 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:56.881157    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:14:57.020487 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:57.020457    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"]
Jul 28 22:14:57.023465 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:14:57.023437    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1b84c4f_f3ba_41ed_92d6_1c58aa9dadc8.slice/crio-be81505a2092be8b0fa70be2ea132f5b8c1efc025b21c0c87f8d10af3fba1b6f WatchSource:0}: Error finding container be81505a2092be8b0fa70be2ea132f5b8c1efc025b21c0c87f8d10af3fba1b6f: Status 404 returned error can't find the container with id be81505a2092be8b0fa70be2ea132f5b8c1efc025b21c0c87f8d10af3fba1b6f
Jul 28 22:14:57.753345 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:57.753310    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-w2q5c" event={"ID":"32913f56-4ade-44a2-a3ff-c137229ddea5","Type":"ContainerStarted","Data":"1ad4a3f4d2dfea6daf72803d73558c47fb21dced947f955e72fa1626a7327dea"}
Jul 28 22:14:57.753847 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:57.753487    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-w2q5c"
Jul 28 22:14:57.754842 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:57.754816    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6" event={"ID":"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8","Type":"ContainerStarted","Data":"be81505a2092be8b0fa70be2ea132f5b8c1efc025b21c0c87f8d10af3fba1b6f"}
Jul 28 22:14:57.792204 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:57.792156    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-w2q5c" podStartSLOduration=6.107512376 podStartE2EDuration="7.792140089s" totalImagesPullingTime="1.684627713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:14:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:14:57.790712691 +0000 UTC m=+305.808678386" watchObservedRunningTime="2026-07-28 22:14:57.792140089 +0000 UTC m=+305.810105782"
Jul 28 22:14:59.747540 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:59.747482    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 22:14:59.747915 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:59.747587    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 22:14:59.747915 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:14:59.747626    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 22:15:00.770110 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:00.770072    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6" event={"ID":"b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8","Type":"ContainerStarted","Data":"ba32490cb1ea42de5a2781a28c541ca7f32397044bee508f82f08522a43c24a0"}
Jul 28 22:15:00.793875 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:00.793823    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6" podStartSLOduration=2.072738505 podStartE2EDuration="4.793805264s" totalImagesPullingTime="2.721066759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:14:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:15:00.792361963 +0000 UTC m=+308.810327660" watchObservedRunningTime="2026-07-28 22:15:00.793805264 +0000 UTC m=+308.811770989"
Jul 28 22:15:00.881476 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:00.881438    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:15:00.886510 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:00.886478    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:15:00.918350 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:00.918312    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2nrlv"]
Jul 28 22:15:00.921296 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:00.921272    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nrlv"
Jul 28 22:15:00.923293 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:00.923270    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-j9cjs\""
Jul 28 22:15:00.923446 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:00.923417    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 28 22:15:00.933045 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:00.933021    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vcdcb"]
Jul 28 22:15:00.936565 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:00.936517    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2nrlv"]
Jul 28 22:15:00.936732 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:00.936627    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vcdcb"
Jul 28 22:15:00.939580 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:00.939312    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-n48m9\""
Jul 28 22:15:00.940036 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:00.939960    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 28 22:15:00.953739 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:00.953708    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vcdcb"]
Jul 28 22:15:01.101121 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:01.101083    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsmkx\" (UniqueName: \"kubernetes.io/projected/6ee65165-d6a4-423b-8595-fc5f815a8041-kube-api-access-bsmkx\") pod \"kserve-controller-manager-d6f4b65b8-2nrlv\" (UID: \"6ee65165-d6a4-423b-8595-fc5f815a8041\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nrlv"
Jul 28 22:15:01.101426 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:01.101406    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ee65165-d6a4-423b-8595-fc5f815a8041-cert\") pod \"kserve-controller-manager-d6f4b65b8-2nrlv\" (UID: \"6ee65165-d6a4-423b-8595-fc5f815a8041\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nrlv"
Jul 28 22:15:01.101626 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:01.101611    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b429457-3746-4fe4-9991-df23a899369b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vcdcb\" (UID: \"5b429457-3746-4fe4-9991-df23a899369b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vcdcb"
Jul 28 22:15:01.101771 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:01.101747    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6t7h\" (UniqueName: \"kubernetes.io/projected/5b429457-3746-4fe4-9991-df23a899369b-kube-api-access-t6t7h\") pod \"llmisvc-controller-manager-777c6bd7d5-vcdcb\" (UID: \"5b429457-3746-4fe4-9991-df23a899369b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vcdcb"
Jul 28 22:15:01.202390 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:01.202352    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ee65165-d6a4-423b-8595-fc5f815a8041-cert\") pod \"kserve-controller-manager-d6f4b65b8-2nrlv\" (UID: \"6ee65165-d6a4-423b-8595-fc5f815a8041\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nrlv"
Jul 28 22:15:01.202390 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:01.202400    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b429457-3746-4fe4-9991-df23a899369b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vcdcb\" (UID: \"5b429457-3746-4fe4-9991-df23a899369b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vcdcb"
Jul 28 22:15:01.202641 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:01.202443    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6t7h\" (UniqueName: \"kubernetes.io/projected/5b429457-3746-4fe4-9991-df23a899369b-kube-api-access-t6t7h\") pod \"llmisvc-controller-manager-777c6bd7d5-vcdcb\" (UID: \"5b429457-3746-4fe4-9991-df23a899369b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vcdcb"
Jul 28 22:15:01.202641 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:01.202499    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsmkx\" (UniqueName: \"kubernetes.io/projected/6ee65165-d6a4-423b-8595-fc5f815a8041-kube-api-access-bsmkx\") pod \"kserve-controller-manager-d6f4b65b8-2nrlv\" (UID: \"6ee65165-d6a4-423b-8595-fc5f815a8041\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nrlv"
Jul 28 22:15:01.202641 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:15:01.202516    2551 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 28 22:15:01.202641 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:15:01.202602    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ee65165-d6a4-423b-8595-fc5f815a8041-cert podName:6ee65165-d6a4-423b-8595-fc5f815a8041 nodeName:}" failed. No retries permitted until 2026-07-28 22:15:01.702578792 +0000 UTC m=+309.720544477 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6ee65165-d6a4-423b-8595-fc5f815a8041-cert") pod "kserve-controller-manager-d6f4b65b8-2nrlv" (UID: "6ee65165-d6a4-423b-8595-fc5f815a8041") : secret "kserve-webhook-server-cert" not found
Jul 28 22:15:01.204904 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:01.204880    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b429457-3746-4fe4-9991-df23a899369b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-vcdcb\" (UID: \"5b429457-3746-4fe4-9991-df23a899369b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vcdcb"
Jul 28 22:15:01.230201 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:01.230169    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6t7h\" (UniqueName: \"kubernetes.io/projected/5b429457-3746-4fe4-9991-df23a899369b-kube-api-access-t6t7h\") pod \"llmisvc-controller-manager-777c6bd7d5-vcdcb\" (UID: \"5b429457-3746-4fe4-9991-df23a899369b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vcdcb"
Jul 28 22:15:01.230352 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:01.230233    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsmkx\" (UniqueName: \"kubernetes.io/projected/6ee65165-d6a4-423b-8595-fc5f815a8041-kube-api-access-bsmkx\") pod \"kserve-controller-manager-d6f4b65b8-2nrlv\" (UID: \"6ee65165-d6a4-423b-8595-fc5f815a8041\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nrlv"
Jul 28 22:15:01.247829 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:01.247794    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vcdcb"
Jul 28 22:15:01.382450 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:01.382348    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-vcdcb"]
Jul 28 22:15:01.385847 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:15:01.385816    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5b429457_3746_4fe4_9991_df23a899369b.slice/crio-836241ffe653f8de69f95bd93e3b1e8166d40fd7e5e64e2ef8c8f55b2f18722d WatchSource:0}: Error finding container 836241ffe653f8de69f95bd93e3b1e8166d40fd7e5e64e2ef8c8f55b2f18722d: Status 404 returned error can't find the container with id 836241ffe653f8de69f95bd93e3b1e8166d40fd7e5e64e2ef8c8f55b2f18722d
Jul 28 22:15:01.706256 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:01.706162    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ee65165-d6a4-423b-8595-fc5f815a8041-cert\") pod \"kserve-controller-manager-d6f4b65b8-2nrlv\" (UID: \"6ee65165-d6a4-423b-8595-fc5f815a8041\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nrlv"
Jul 28 22:15:01.708503 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:01.708485    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ee65165-d6a4-423b-8595-fc5f815a8041-cert\") pod \"kserve-controller-manager-d6f4b65b8-2nrlv\" (UID: \"6ee65165-d6a4-423b-8595-fc5f815a8041\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nrlv"
Jul 28 22:15:01.774704 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:01.774665    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vcdcb" event={"ID":"5b429457-3746-4fe4-9991-df23a899369b","Type":"ContainerStarted","Data":"836241ffe653f8de69f95bd93e3b1e8166d40fd7e5e64e2ef8c8f55b2f18722d"}
Jul 28 22:15:01.775076 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:01.774732    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:15:01.775618 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:01.775603    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6"
Jul 28 22:15:01.831538 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:01.831500    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nrlv"
Jul 28 22:15:01.968105 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:01.968021    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2nrlv"]
Jul 28 22:15:01.971188 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:15:01.971160    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ee65165_d6a4_423b_8595_fc5f815a8041.slice/crio-0441b2aa12ef1db1443edc14b96c5783b5176ba19c1fda055dada2372c8458f3 WatchSource:0}: Error finding container 0441b2aa12ef1db1443edc14b96c5783b5176ba19c1fda055dada2372c8458f3: Status 404 returned error can't find the container with id 0441b2aa12ef1db1443edc14b96c5783b5176ba19c1fda055dada2372c8458f3
Jul 28 22:15:02.781434 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:02.781402    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nrlv" event={"ID":"6ee65165-d6a4-423b-8595-fc5f815a8041","Type":"ContainerStarted","Data":"0441b2aa12ef1db1443edc14b96c5783b5176ba19c1fda055dada2372c8458f3"}
Jul 28 22:15:04.027265 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:04.027231    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"]
Jul 28 22:15:04.027746 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:04.027442    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc" podUID="fd805c57-e107-4ca5-8dd4-254ba9dd8587" containerName="discovery" containerID="cri-o://f8719fb3180827292cc4742c314be835b5ee80052f5e04858f08b3c75c061ffe" gracePeriod=30
Jul 28 22:15:04.792108 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:04.792073    2551 generic.go:356] "Generic (PLEG): container finished" podID="fd805c57-e107-4ca5-8dd4-254ba9dd8587" containerID="f8719fb3180827292cc4742c314be835b5ee80052f5e04858f08b3c75c061ffe" exitCode=0
Jul 28 22:15:04.792268 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:04.792143    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc" event={"ID":"fd805c57-e107-4ca5-8dd4-254ba9dd8587","Type":"ContainerDied","Data":"f8719fb3180827292cc4742c314be835b5ee80052f5e04858f08b3c75c061ffe"}
Jul 28 22:15:04.970269 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:04.970239    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:15:05.139091 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.139055    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fd805c57-e107-4ca5-8dd4-254ba9dd8587-cacerts\") pod \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") "
Jul 28 22:15:05.139622 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.139110    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4hbz\" (UniqueName: \"kubernetes.io/projected/fd805c57-e107-4ca5-8dd4-254ba9dd8587-kube-api-access-b4hbz\") pod \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") "
Jul 28 22:15:05.139622 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.139130    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-kubeconfig\") pod \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") "
Jul 28 22:15:05.139622 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.139169    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd805c57-e107-4ca5-8dd4-254ba9dd8587-local-certs\") pod \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") "
Jul 28 22:15:05.139622 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.139325    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-csr-dns-cert\") pod \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") "
Jul 28 22:15:05.139622 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.139359    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-token\") pod \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") "
Jul 28 22:15:05.139622 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.139401    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-csr-ca-configmap\") pod \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\" (UID: \"fd805c57-e107-4ca5-8dd4-254ba9dd8587\") "
Jul 28 22:15:05.140021 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.139992    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "fd805c57-e107-4ca5-8dd4-254ba9dd8587" (UID: "fd805c57-e107-4ca5-8dd4-254ba9dd8587"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:15:05.142080 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.142022    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd805c57-e107-4ca5-8dd4-254ba9dd8587-local-certs" (OuterVolumeSpecName: "local-certs") pod "fd805c57-e107-4ca5-8dd4-254ba9dd8587" (UID: "fd805c57-e107-4ca5-8dd4-254ba9dd8587"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:15:05.142080 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.142041    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd805c57-e107-4ca5-8dd4-254ba9dd8587-cacerts" (OuterVolumeSpecName: "cacerts") pod "fd805c57-e107-4ca5-8dd4-254ba9dd8587" (UID: "fd805c57-e107-4ca5-8dd4-254ba9dd8587"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:15:05.142080 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.142053    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "fd805c57-e107-4ca5-8dd4-254ba9dd8587" (UID: "fd805c57-e107-4ca5-8dd4-254ba9dd8587"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:15:05.142080 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.142022    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "fd805c57-e107-4ca5-8dd4-254ba9dd8587" (UID: "fd805c57-e107-4ca5-8dd4-254ba9dd8587"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:15:05.142080 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.142028    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd805c57-e107-4ca5-8dd4-254ba9dd8587-kube-api-access-b4hbz" (OuterVolumeSpecName: "kube-api-access-b4hbz") pod "fd805c57-e107-4ca5-8dd4-254ba9dd8587" (UID: "fd805c57-e107-4ca5-8dd4-254ba9dd8587"). InnerVolumeSpecName "kube-api-access-b4hbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:15:05.142318 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.142116    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-token" (OuterVolumeSpecName: "istio-token") pod "fd805c57-e107-4ca5-8dd4-254ba9dd8587" (UID: "fd805c57-e107-4ca5-8dd4-254ba9dd8587"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:15:05.240468 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.240432    2551 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd805c57-e107-4ca5-8dd4-254ba9dd8587-local-certs\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:15:05.240468 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.240465    2551 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-csr-dns-cert\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:15:05.240468 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.240476    2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-token\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:15:05.240708 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.240484    2551 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-csr-ca-configmap\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:15:05.240708 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.240494    2551 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fd805c57-e107-4ca5-8dd4-254ba9dd8587-cacerts\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:15:05.240708 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.240503    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4hbz\" (UniqueName: \"kubernetes.io/projected/fd805c57-e107-4ca5-8dd4-254ba9dd8587-kube-api-access-b4hbz\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:15:05.240708 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.240511    2551 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd805c57-e107-4ca5-8dd4-254ba9dd8587-istio-kubeconfig\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:15:05.797210 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.797175    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"
Jul 28 22:15:05.797210 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.797202    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc" event={"ID":"fd805c57-e107-4ca5-8dd4-254ba9dd8587","Type":"ContainerDied","Data":"7fc01db37e549a16186ff7ff11a28d1ff0d6128a51c242fb291fa18139c6a2e4"}
Jul 28 22:15:05.797464 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.797282    2551 scope.go:117] "RemoveContainer" containerID="f8719fb3180827292cc4742c314be835b5ee80052f5e04858f08b3c75c061ffe"
Jul 28 22:15:05.798942 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.798904    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vcdcb" event={"ID":"5b429457-3746-4fe4-9991-df23a899369b","Type":"ContainerStarted","Data":"fd04c4c94a6c1464623e598e601494531a5940cba25d6efb90a6d149dc05ac27"}
Jul 28 22:15:05.799051 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.799011    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vcdcb"
Jul 28 22:15:05.800454 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.800430    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nrlv" event={"ID":"6ee65165-d6a4-423b-8595-fc5f815a8041","Type":"ContainerStarted","Data":"0429adafc5e7f1a113a157e4881c4e08e0fcdb38ad595ad27f991def75ca0d3f"}
Jul 28 22:15:05.800574 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.800513    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nrlv"
Jul 28 22:15:05.818953 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.818900    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nrlv" podStartSLOduration=2.933174459 podStartE2EDuration="5.818880831s" totalImagesPullingTime="2.885706372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:15:05.816336297 +0000 UTC m=+313.834301992" watchObservedRunningTime="2026-07-28 22:15:05.818880831 +0000 UTC m=+313.836846527"
Jul 28 22:15:05.831456 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.831407    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vcdcb" podStartSLOduration=2.357548176 podStartE2EDuration="5.831392661s" totalImagesPullingTime="3.473844485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:15:05.830612689 +0000 UTC m=+313.848578404" watchObservedRunningTime="2026-07-28 22:15:05.831392661 +0000 UTC m=+313.849358354"
Jul 28 22:15:05.844297 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.844266    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"]
Jul 28 22:15:05.848204 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:05.848180    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-9rkqc"]
Jul 28 22:15:06.462844 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:06.462805    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd805c57-e107-4ca5-8dd4-254ba9dd8587" path="/var/lib/kubelet/pods/fd805c57-e107-4ca5-8dd4-254ba9dd8587/volumes"
Jul 28 22:15:06.749317 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:06.749232    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-lfcxw"
Jul 28 22:15:08.764271 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:08.764242    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-w2q5c"
Jul 28 22:15:36.809069 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:36.809040    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-vcdcb"
Jul 28 22:15:36.812063 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:36.812042    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2nrlv"
Jul 28 22:15:49.921655 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:49.921580    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j4z7x"]
Jul 28 22:15:49.922013 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:49.921864    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd805c57-e107-4ca5-8dd4-254ba9dd8587" containerName="discovery"
Jul 28 22:15:49.922013 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:49.921874    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd805c57-e107-4ca5-8dd4-254ba9dd8587" containerName="discovery"
Jul 28 22:15:49.922013 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:49.921920    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd805c57-e107-4ca5-8dd4-254ba9dd8587" containerName="discovery"
Jul 28 22:15:49.924717 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:49.924698    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j4z7x"
Jul 28 22:15:49.926905 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:49.926882    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-rl6xl\""
Jul 28 22:15:49.927380 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:49.927360    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 22:15:49.927568 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:49.927547    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 22:15:49.931999 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:49.931972    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j4z7x"]
Jul 28 22:15:49.996774 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:49.996741    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nvdz\" (UniqueName: \"kubernetes.io/projected/39610ad9-b434-4139-8a37-6ab3ec5f4379-kube-api-access-7nvdz\") pod \"kuadrant-operator-catalog-j4z7x\" (UID: \"39610ad9-b434-4139-8a37-6ab3ec5f4379\") " pod="kuadrant-system/kuadrant-operator-catalog-j4z7x"
Jul 28 22:15:50.097734 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:50.097693    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nvdz\" (UniqueName: \"kubernetes.io/projected/39610ad9-b434-4139-8a37-6ab3ec5f4379-kube-api-access-7nvdz\") pod \"kuadrant-operator-catalog-j4z7x\" (UID: \"39610ad9-b434-4139-8a37-6ab3ec5f4379\") " pod="kuadrant-system/kuadrant-operator-catalog-j4z7x"
Jul 28 22:15:50.105039 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:50.105009    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nvdz\" (UniqueName: \"kubernetes.io/projected/39610ad9-b434-4139-8a37-6ab3ec5f4379-kube-api-access-7nvdz\") pod \"kuadrant-operator-catalog-j4z7x\" (UID: \"39610ad9-b434-4139-8a37-6ab3ec5f4379\") " pod="kuadrant-system/kuadrant-operator-catalog-j4z7x"
Jul 28 22:15:50.235869 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:50.235782    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j4z7x"
Jul 28 22:15:50.293316 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:50.293208    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j4z7x"]
Jul 28 22:15:50.362221 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:50.362196    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j4z7x"]
Jul 28 22:15:50.364580 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:15:50.364556    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39610ad9_b434_4139_8a37_6ab3ec5f4379.slice/crio-4ec9201e367e1f64610b3bcdf5ca9e081ed2cb38f836e8826b0748d82a7b1988 WatchSource:0}: Error finding container 4ec9201e367e1f64610b3bcdf5ca9e081ed2cb38f836e8826b0748d82a7b1988: Status 404 returned error can't find the container with id 4ec9201e367e1f64610b3bcdf5ca9e081ed2cb38f836e8826b0748d82a7b1988
Jul 28 22:15:50.502585 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:50.502501    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dd5hz"]
Jul 28 22:15:50.506732 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:50.506716    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dd5hz"
Jul 28 22:15:50.513598 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:50.513568    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dd5hz"]
Jul 28 22:15:50.602488 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:50.602456    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk8gq\" (UniqueName: \"kubernetes.io/projected/f6718c70-80d3-43f8-a684-aa08b0ce7a5c-kube-api-access-kk8gq\") pod \"kuadrant-operator-catalog-dd5hz\" (UID: \"f6718c70-80d3-43f8-a684-aa08b0ce7a5c\") " pod="kuadrant-system/kuadrant-operator-catalog-dd5hz"
Jul 28 22:15:50.702879 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:50.702847    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk8gq\" (UniqueName: \"kubernetes.io/projected/f6718c70-80d3-43f8-a684-aa08b0ce7a5c-kube-api-access-kk8gq\") pod \"kuadrant-operator-catalog-dd5hz\" (UID: \"f6718c70-80d3-43f8-a684-aa08b0ce7a5c\") " pod="kuadrant-system/kuadrant-operator-catalog-dd5hz"
Jul 28 22:15:50.710926 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:50.710904    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk8gq\" (UniqueName: \"kubernetes.io/projected/f6718c70-80d3-43f8-a684-aa08b0ce7a5c-kube-api-access-kk8gq\") pod \"kuadrant-operator-catalog-dd5hz\" (UID: \"f6718c70-80d3-43f8-a684-aa08b0ce7a5c\") " pod="kuadrant-system/kuadrant-operator-catalog-dd5hz"
Jul 28 22:15:50.816916 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:50.816884    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dd5hz"
Jul 28 22:15:50.945877 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:50.945835    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dd5hz"]
Jul 28 22:15:50.948333 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:15:50.948308    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6718c70_80d3_43f8_a684_aa08b0ce7a5c.slice/crio-b8044a1902776318a6c0069ce02a533578eb16490846faebb50dd846c5e229bd WatchSource:0}: Error finding container b8044a1902776318a6c0069ce02a533578eb16490846faebb50dd846c5e229bd: Status 404 returned error can't find the container with id b8044a1902776318a6c0069ce02a533578eb16490846faebb50dd846c5e229bd
Jul 28 22:15:50.978085 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:50.978049    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j4z7x" event={"ID":"39610ad9-b434-4139-8a37-6ab3ec5f4379","Type":"ContainerStarted","Data":"4ec9201e367e1f64610b3bcdf5ca9e081ed2cb38f836e8826b0748d82a7b1988"}
Jul 28 22:15:50.979292 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:50.979264    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dd5hz" event={"ID":"f6718c70-80d3-43f8-a684-aa08b0ce7a5c","Type":"ContainerStarted","Data":"b8044a1902776318a6c0069ce02a533578eb16490846faebb50dd846c5e229bd"}
Jul 28 22:15:53.994013 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:53.993968    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dd5hz" event={"ID":"f6718c70-80d3-43f8-a684-aa08b0ce7a5c","Type":"ContainerStarted","Data":"97ad31d91d921c4aacaa1aa6849f3b9a9c254fa8d4b7e88cf4c69cb6eb4aad44"}
Jul 28 22:15:53.995270 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:53.995246    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j4z7x" event={"ID":"39610ad9-b434-4139-8a37-6ab3ec5f4379","Type":"ContainerStarted","Data":"6925be8c985f0be5010ba67b1d0d4b9ed8aa2f03a7a64db3e09a1fb45b77ed46"}
Jul 28 22:15:53.995357 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:53.995323    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-j4z7x" podUID="39610ad9-b434-4139-8a37-6ab3ec5f4379" containerName="registry-server" containerID="cri-o://6925be8c985f0be5010ba67b1d0d4b9ed8aa2f03a7a64db3e09a1fb45b77ed46" gracePeriod=2
Jul 28 22:15:54.008309 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:54.008158    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-dd5hz" podStartSLOduration=1.713988304 podStartE2EDuration="4.008142663s" totalImagesPullingTime="2.294154359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:15:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:15:54.006971558 +0000 UTC m=+362.024937272" watchObservedRunningTime="2026-07-28 22:15:54.008142663 +0000 UTC m=+362.026108356"
Jul 28 22:15:54.020787 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:54.020740    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-j4z7x" podStartSLOduration=2.3199750740000002 podStartE2EDuration="5.020722468s" totalImagesPullingTime="2.700747394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:15:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:15:54.019811265 +0000 UTC m=+362.037776956" watchObservedRunningTime="2026-07-28 22:15:54.020722468 +0000 UTC m=+362.038688162"
Jul 28 22:15:54.251072 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:54.251014    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j4z7x"
Jul 28 22:15:54.335386 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:54.335350    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7nvdz\" (UniqueName: \"kubernetes.io/projected/39610ad9-b434-4139-8a37-6ab3ec5f4379-kube-api-access-7nvdz\") pod \"39610ad9-b434-4139-8a37-6ab3ec5f4379\" (UID: \"39610ad9-b434-4139-8a37-6ab3ec5f4379\") "
Jul 28 22:15:54.337486 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:54.337453    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39610ad9-b434-4139-8a37-6ab3ec5f4379-kube-api-access-7nvdz" (OuterVolumeSpecName: "kube-api-access-7nvdz") pod "39610ad9-b434-4139-8a37-6ab3ec5f4379" (UID: "39610ad9-b434-4139-8a37-6ab3ec5f4379"). InnerVolumeSpecName "kube-api-access-7nvdz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:15:54.436193 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:54.436158    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nvdz\" (UniqueName: \"kubernetes.io/projected/39610ad9-b434-4139-8a37-6ab3ec5f4379-kube-api-access-7nvdz\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:15:55.005608 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:55.005573    2551 generic.go:356] "Generic (PLEG): container finished" podID="39610ad9-b434-4139-8a37-6ab3ec5f4379" containerID="6925be8c985f0be5010ba67b1d0d4b9ed8aa2f03a7a64db3e09a1fb45b77ed46" exitCode=0
Jul 28 22:15:55.006016 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:55.005629    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-j4z7x"
Jul 28 22:15:55.006016 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:55.005660    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j4z7x" event={"ID":"39610ad9-b434-4139-8a37-6ab3ec5f4379","Type":"ContainerDied","Data":"6925be8c985f0be5010ba67b1d0d4b9ed8aa2f03a7a64db3e09a1fb45b77ed46"}
Jul 28 22:15:55.006016 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:55.005726    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-j4z7x" event={"ID":"39610ad9-b434-4139-8a37-6ab3ec5f4379","Type":"ContainerDied","Data":"4ec9201e367e1f64610b3bcdf5ca9e081ed2cb38f836e8826b0748d82a7b1988"}
Jul 28 22:15:55.006016 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:55.005747    2551 scope.go:117] "RemoveContainer" containerID="6925be8c985f0be5010ba67b1d0d4b9ed8aa2f03a7a64db3e09a1fb45b77ed46"
Jul 28 22:15:55.013992 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:55.013976    2551 scope.go:117] "RemoveContainer" containerID="6925be8c985f0be5010ba67b1d0d4b9ed8aa2f03a7a64db3e09a1fb45b77ed46"
Jul 28 22:15:55.014252 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:15:55.014232    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6925be8c985f0be5010ba67b1d0d4b9ed8aa2f03a7a64db3e09a1fb45b77ed46\": container with ID starting with 6925be8c985f0be5010ba67b1d0d4b9ed8aa2f03a7a64db3e09a1fb45b77ed46 not found: ID does not exist" containerID="6925be8c985f0be5010ba67b1d0d4b9ed8aa2f03a7a64db3e09a1fb45b77ed46"
Jul 28 22:15:55.014321 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:55.014262    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6925be8c985f0be5010ba67b1d0d4b9ed8aa2f03a7a64db3e09a1fb45b77ed46"} err="failed to get container status \"6925be8c985f0be5010ba67b1d0d4b9ed8aa2f03a7a64db3e09a1fb45b77ed46\": rpc error: code = NotFound desc = could not find container \"6925be8c985f0be5010ba67b1d0d4b9ed8aa2f03a7a64db3e09a1fb45b77ed46\": container with ID starting with 6925be8c985f0be5010ba67b1d0d4b9ed8aa2f03a7a64db3e09a1fb45b77ed46 not found: ID does not exist"
Jul 28 22:15:55.019167 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:55.019144    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j4z7x"]
Jul 28 22:15:55.025338 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:55.025318    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-j4z7x"]
Jul 28 22:15:56.462004 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:15:56.461974    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39610ad9-b434-4139-8a37-6ab3ec5f4379" path="/var/lib/kubelet/pods/39610ad9-b434-4139-8a37-6ab3ec5f4379/volumes"
Jul 28 22:16:00.817667 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:00.817627    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-dd5hz"
Jul 28 22:16:00.818086 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:00.817689    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-dd5hz"
Jul 28 22:16:00.839276 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:00.839245    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-dd5hz"
Jul 28 22:16:01.055632 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:01.055592    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-dd5hz"
Jul 28 22:16:16.189637 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:16.189605    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh"]
Jul 28 22:16:16.190114 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:16.189886    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39610ad9-b434-4139-8a37-6ab3ec5f4379" containerName="registry-server"
Jul 28 22:16:16.190114 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:16.189898    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="39610ad9-b434-4139-8a37-6ab3ec5f4379" containerName="registry-server"
Jul 28 22:16:16.190114 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:16.189951    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="39610ad9-b434-4139-8a37-6ab3ec5f4379" containerName="registry-server"
Jul 28 22:16:16.192868 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:16.192849    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh"
Jul 28 22:16:16.194838 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:16.194816    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-nhsl4\""
Jul 28 22:16:16.203175 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:16.203156    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd29f\" (UniqueName: \"kubernetes.io/projected/45556f1d-16ae-4f00-8029-4d65d32f1a22-kube-api-access-sd29f\") pod \"limitador-operator-controller-manager-859d7b78bd-gctwh\" (UID: \"45556f1d-16ae-4f00-8029-4d65d32f1a22\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh"
Jul 28 22:16:16.206197 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:16.206174    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh"]
Jul 28 22:16:16.304363 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:16.304328    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd29f\" (UniqueName: \"kubernetes.io/projected/45556f1d-16ae-4f00-8029-4d65d32f1a22-kube-api-access-sd29f\") pod \"limitador-operator-controller-manager-859d7b78bd-gctwh\" (UID: \"45556f1d-16ae-4f00-8029-4d65d32f1a22\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh"
Jul 28 22:16:16.320815 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:16.320775    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd29f\" (UniqueName: \"kubernetes.io/projected/45556f1d-16ae-4f00-8029-4d65d32f1a22-kube-api-access-sd29f\") pod \"limitador-operator-controller-manager-859d7b78bd-gctwh\" (UID: \"45556f1d-16ae-4f00-8029-4d65d32f1a22\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh"
Jul 28 22:16:16.502838 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:16.502745    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh"
Jul 28 22:16:16.653606 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:16.653575    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh"]
Jul 28 22:16:16.656844 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:16:16.656817    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45556f1d_16ae_4f00_8029_4d65d32f1a22.slice/crio-ae691e804e23a2a47a53569b5af04fb90e4c1434abaaaa62c32fdc2577813ae7 WatchSource:0}: Error finding container ae691e804e23a2a47a53569b5af04fb90e4c1434abaaaa62c32fdc2577813ae7: Status 404 returned error can't find the container with id ae691e804e23a2a47a53569b5af04fb90e4c1434abaaaa62c32fdc2577813ae7
Jul 28 22:16:17.099359 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:17.099324    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh" event={"ID":"45556f1d-16ae-4f00-8029-4d65d32f1a22","Type":"ContainerStarted","Data":"ae691e804e23a2a47a53569b5af04fb90e4c1434abaaaa62c32fdc2577813ae7"}
Jul 28 22:16:19.114300 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:19.114257    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh" event={"ID":"45556f1d-16ae-4f00-8029-4d65d32f1a22","Type":"ContainerStarted","Data":"fdb72674c6b7d3a6c7d6f2195a5b53c7ce66f47d53f7527468cb572fe5434063"}
Jul 28 22:16:19.114746 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:19.114411    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh"
Jul 28 22:16:19.133177 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:19.133124    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh" podStartSLOduration=1.7060391639999999 podStartE2EDuration="3.133109512s" totalImagesPullingTime="1.427070348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:16:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:16:19.131784556 +0000 UTC m=+387.149750285" watchObservedRunningTime="2026-07-28 22:16:19.133109512 +0000 UTC m=+387.151075203"
Jul 28 22:16:21.581727 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:21.581689    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"]
Jul 28 22:16:21.584948 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:21.584932    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"
Jul 28 22:16:21.589077 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:21.589054    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-smk6t\""
Jul 28 22:16:21.595692 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:21.595670    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"]
Jul 28 22:16:21.651589 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:21.651521    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpbjz\" (UniqueName: \"kubernetes.io/projected/f6d85cc8-8e76-4b6f-988e-b05727efaed8-kube-api-access-qpbjz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9d8qp\" (UID: \"f6d85cc8-8e76-4b6f-988e-b05727efaed8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"
Jul 28 22:16:21.651760 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:21.651603    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6d85cc8-8e76-4b6f-988e-b05727efaed8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9d8qp\" (UID: \"f6d85cc8-8e76-4b6f-988e-b05727efaed8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"
Jul 28 22:16:21.753055 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:21.753007    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpbjz\" (UniqueName: \"kubernetes.io/projected/f6d85cc8-8e76-4b6f-988e-b05727efaed8-kube-api-access-qpbjz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9d8qp\" (UID: \"f6d85cc8-8e76-4b6f-988e-b05727efaed8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"
Jul 28 22:16:21.753055 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:21.753060    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6d85cc8-8e76-4b6f-988e-b05727efaed8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9d8qp\" (UID: \"f6d85cc8-8e76-4b6f-988e-b05727efaed8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"
Jul 28 22:16:21.753455 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:21.753432    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6d85cc8-8e76-4b6f-988e-b05727efaed8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9d8qp\" (UID: \"f6d85cc8-8e76-4b6f-988e-b05727efaed8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"
Jul 28 22:16:21.764187 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:21.764158    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpbjz\" (UniqueName: \"kubernetes.io/projected/f6d85cc8-8e76-4b6f-988e-b05727efaed8-kube-api-access-qpbjz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-9d8qp\" (UID: \"f6d85cc8-8e76-4b6f-988e-b05727efaed8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"
Jul 28 22:16:21.896107 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:21.896023    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"
Jul 28 22:16:22.029484 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:22.029456    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"]
Jul 28 22:16:22.032265 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:16:22.032236    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6d85cc8_8e76_4b6f_988e_b05727efaed8.slice/crio-6bfd0ff5a662b44df54ac46313ce54944de121d1d547f6ab5f8182ac1de9c9c6 WatchSource:0}: Error finding container 6bfd0ff5a662b44df54ac46313ce54944de121d1d547f6ab5f8182ac1de9c9c6: Status 404 returned error can't find the container with id 6bfd0ff5a662b44df54ac46313ce54944de121d1d547f6ab5f8182ac1de9c9c6
Jul 28 22:16:22.128133 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:22.128097    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp" event={"ID":"f6d85cc8-8e76-4b6f-988e-b05727efaed8","Type":"ContainerStarted","Data":"6bfd0ff5a662b44df54ac46313ce54944de121d1d547f6ab5f8182ac1de9c9c6"}
Jul 28 22:16:24.725508 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:24.725468    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-68n4l"]
Jul 28 22:16:24.729028 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:24.729004    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-68n4l"
Jul 28 22:16:24.732479 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:24.732436    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-47mrg\""
Jul 28 22:16:24.732687 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:24.732672    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 28 22:16:24.739923 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:24.739900    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-68n4l"]
Jul 28 22:16:24.780781 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:24.780750    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxphb\" (UniqueName: \"kubernetes.io/projected/7bf1110c-d048-42ce-a459-3c18aadb8c09-kube-api-access-vxphb\") pod \"dns-operator-controller-manager-864dd78c4c-68n4l\" (UID: \"7bf1110c-d048-42ce-a459-3c18aadb8c09\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-68n4l"
Jul 28 22:16:24.882204 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:24.882165    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxphb\" (UniqueName: \"kubernetes.io/projected/7bf1110c-d048-42ce-a459-3c18aadb8c09-kube-api-access-vxphb\") pod \"dns-operator-controller-manager-864dd78c4c-68n4l\" (UID: \"7bf1110c-d048-42ce-a459-3c18aadb8c09\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-68n4l"
Jul 28 22:16:24.895988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:24.895954    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxphb\" (UniqueName: \"kubernetes.io/projected/7bf1110c-d048-42ce-a459-3c18aadb8c09-kube-api-access-vxphb\") pod \"dns-operator-controller-manager-864dd78c4c-68n4l\" (UID: \"7bf1110c-d048-42ce-a459-3c18aadb8c09\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-68n4l"
Jul 28 22:16:25.041619 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:25.041513    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-68n4l"
Jul 28 22:16:25.991587 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:25.991563    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-68n4l"]
Jul 28 22:16:25.994013 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:16:25.993987    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bf1110c_d048_42ce_a459_3c18aadb8c09.slice/crio-52d5e625694641d35b88b5e26618bd92667aa837305405e21613a32a5277d256 WatchSource:0}: Error finding container 52d5e625694641d35b88b5e26618bd92667aa837305405e21613a32a5277d256: Status 404 returned error can't find the container with id 52d5e625694641d35b88b5e26618bd92667aa837305405e21613a32a5277d256
Jul 28 22:16:26.149252 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:26.149150    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp" event={"ID":"f6d85cc8-8e76-4b6f-988e-b05727efaed8","Type":"ContainerStarted","Data":"28198d8d8f9496d5e21254b99173f6ea32caabdd5fe637a1aede4634f905459d"}
Jul 28 22:16:26.149252 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:26.149221    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"
Jul 28 22:16:26.150408 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:26.150387    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-68n4l" event={"ID":"7bf1110c-d048-42ce-a459-3c18aadb8c09","Type":"ContainerStarted","Data":"52d5e625694641d35b88b5e26618bd92667aa837305405e21613a32a5277d256"}
Jul 28 22:16:26.167715 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:26.167669    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp" podStartSLOduration=1.282259158 podStartE2EDuration="5.167653585s" totalImagesPullingTime="3.885394427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:16:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:16:26.166644773 +0000 UTC m=+394.184610467" watchObservedRunningTime="2026-07-28 22:16:26.167653585 +0000 UTC m=+394.185619282"
Jul 28 22:16:28.161610 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:28.161569    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-68n4l" event={"ID":"7bf1110c-d048-42ce-a459-3c18aadb8c09","Type":"ContainerStarted","Data":"2137a6d94ad650d0928e71576af2a7cbb82f877a8afcef6a6ccabd4f27777f4e"}
Jul 28 22:16:28.162059 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:28.161678    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-68n4l"
Jul 28 22:16:28.179414 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:28.179358    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-68n4l" podStartSLOduration=2.202792697 podStartE2EDuration="4.179337264s" totalImagesPullingTime="1.976544567s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:16:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:16:28.178200806 +0000 UTC m=+396.196166502" watchObservedRunningTime="2026-07-28 22:16:28.179337264 +0000 UTC m=+396.197302958"
Jul 28 22:16:28.234592 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:28.234499    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5mgx4"]
Jul 28 22:16:28.238304 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:28.238286    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5mgx4"
Jul 28 22:16:28.240479 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:28.240459    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-lkb8d\""
Jul 28 22:16:28.247039 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:28.247015    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5mgx4"]
Jul 28 22:16:28.310979 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:28.310940    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95r9r\" (UniqueName: \"kubernetes.io/projected/3b6a0f70-a698-484a-b6c9-4f69b5cde601-kube-api-access-95r9r\") pod \"authorino-operator-75cc8d5cb7-5mgx4\" (UID: \"3b6a0f70-a698-484a-b6c9-4f69b5cde601\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5mgx4"
Jul 28 22:16:28.412018 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:28.411984    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95r9r\" (UniqueName: \"kubernetes.io/projected/3b6a0f70-a698-484a-b6c9-4f69b5cde601-kube-api-access-95r9r\") pod \"authorino-operator-75cc8d5cb7-5mgx4\" (UID: \"3b6a0f70-a698-484a-b6c9-4f69b5cde601\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5mgx4"
Jul 28 22:16:28.427569 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:28.427518    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95r9r\" (UniqueName: \"kubernetes.io/projected/3b6a0f70-a698-484a-b6c9-4f69b5cde601-kube-api-access-95r9r\") pod \"authorino-operator-75cc8d5cb7-5mgx4\" (UID: \"3b6a0f70-a698-484a-b6c9-4f69b5cde601\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5mgx4"
Jul 28 22:16:28.549074 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:28.548982    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5mgx4"
Jul 28 22:16:28.677905 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:28.677880    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5mgx4"]
Jul 28 22:16:28.680689 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:16:28.680661    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b6a0f70_a698_484a_b6c9_4f69b5cde601.slice/crio-6f47fbcfea0036b47a915bf25b7e7d5b47addc55c5225c879921efdbdc9102ae WatchSource:0}: Error finding container 6f47fbcfea0036b47a915bf25b7e7d5b47addc55c5225c879921efdbdc9102ae: Status 404 returned error can't find the container with id 6f47fbcfea0036b47a915bf25b7e7d5b47addc55c5225c879921efdbdc9102ae
Jul 28 22:16:29.167742 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:29.167704    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5mgx4" event={"ID":"3b6a0f70-a698-484a-b6c9-4f69b5cde601","Type":"ContainerStarted","Data":"6f47fbcfea0036b47a915bf25b7e7d5b47addc55c5225c879921efdbdc9102ae"}
Jul 28 22:16:30.121691 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:30.121659    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh"
Jul 28 22:16:31.178714 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:31.178621    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5mgx4" event={"ID":"3b6a0f70-a698-484a-b6c9-4f69b5cde601","Type":"ContainerStarted","Data":"ac5770efd28b0bbe23b7551724584e8b075ad86e818b7c132f07acf0b441e24b"}
Jul 28 22:16:31.179066 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:31.178765    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5mgx4"
Jul 28 22:16:31.198150 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:31.198051    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5mgx4" podStartSLOduration=1.054640971 podStartE2EDuration="3.198033458s" totalImagesPullingTime="2.143392487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:16:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:16:31.197709732 +0000 UTC m=+399.215675427" watchObservedRunningTime="2026-07-28 22:16:31.198033458 +0000 UTC m=+399.215999154"
Jul 28 22:16:37.158273 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:37.158239    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"
Jul 28 22:16:38.923958 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:38.923913    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"]
Jul 28 22:16:38.924427 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:38.924149    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp" podUID="f6d85cc8-8e76-4b6f-988e-b05727efaed8" containerName="manager" containerID="cri-o://28198d8d8f9496d5e21254b99173f6ea32caabdd5fe637a1aede4634f905459d" gracePeriod=2
Jul 28 22:16:38.952366 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:38.952340    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"]
Jul 28 22:16:38.956890 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:16:38.956854    2551 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f6d85cc8-8e76-4b6f-988e-b05727efaed8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-9d8qp\": pods \"kuadrant-operator-controller-manager-8684c9ff49-9d8qp\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"
Jul 28 22:16:38.970862 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:38.970828    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6"]
Jul 28 22:16:38.971242 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:38.971224    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6d85cc8-8e76-4b6f-988e-b05727efaed8" containerName="manager"
Jul 28 22:16:38.971407 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:38.971273    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6d85cc8-8e76-4b6f-988e-b05727efaed8" containerName="manager"
Jul 28 22:16:38.971407 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:38.971376    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6d85cc8-8e76-4b6f-988e-b05727efaed8" containerName="manager"
Jul 28 22:16:38.974620 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:38.974589    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh"]
Jul 28 22:16:38.974756 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:38.974737    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6"
Jul 28 22:16:38.974845 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:38.974818    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh" podUID="45556f1d-16ae-4f00-8029-4d65d32f1a22" containerName="manager" containerID="cri-o://fdb72674c6b7d3a6c7d6f2195a5b53c7ce66f47d53f7527468cb572fe5434063" gracePeriod=2
Jul 28 22:16:38.976377 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:38.976348    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh"]
Jul 28 22:16:38.977255 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:16:38.977224    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9d8qp\" is forbidden: User \"system:node:ip-10-0-141-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-102.ec2.internal' and this object" podUID="f6d85cc8-8e76-4b6f-988e-b05727efaed8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"
Jul 28 22:16:38.979851 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:16:38.979815    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gctwh\" is forbidden: User \"system:node:ip-10-0-141-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-102.ec2.internal' and this object" podUID="45556f1d-16ae-4f00-8029-4d65d32f1a22" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh"
Jul 28 22:16:38.981546 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:16:38.981502    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9d8qp\" is forbidden: User \"system:node:ip-10-0-141-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-102.ec2.internal' and this object" podUID="f6d85cc8-8e76-4b6f-988e-b05727efaed8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"
Jul 28 22:16:38.998003 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:38.997966    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6"]
Jul 28 22:16:39.017003 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.016976    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dk88"]
Jul 28 22:16:39.017310 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.017297    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45556f1d-16ae-4f00-8029-4d65d32f1a22" containerName="manager"
Jul 28 22:16:39.017403 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.017312    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="45556f1d-16ae-4f00-8029-4d65d32f1a22" containerName="manager"
Jul 28 22:16:39.017403 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.017370    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="45556f1d-16ae-4f00-8029-4d65d32f1a22" containerName="manager"
Jul 28 22:16:39.019174 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:16:39.019136    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gctwh\" is forbidden: User \"system:node:ip-10-0-141-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-102.ec2.internal' and this object" podUID="45556f1d-16ae-4f00-8029-4d65d32f1a22" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh"
Jul 28 22:16:39.020646 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.020628    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dk88"
Jul 28 22:16:39.022416 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:16:39.022395    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gctwh\" is forbidden: User \"system:node:ip-10-0-141-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-102.ec2.internal' and this object" podUID="45556f1d-16ae-4f00-8029-4d65d32f1a22" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh"
Jul 28 22:16:39.035181 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.035134    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dk88"]
Jul 28 22:16:39.049336 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:16:39.049300    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9d8qp\" is forbidden: User \"system:node:ip-10-0-141-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-102.ec2.internal' and this object" podUID="f6d85cc8-8e76-4b6f-988e-b05727efaed8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"
Jul 28 22:16:39.103937 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.103901    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jgj4\" (UniqueName: \"kubernetes.io/projected/7947ae81-e535-495f-b66f-72510cf91f65-kube-api-access-4jgj4\") pod \"limitador-operator-controller-manager-859d7b78bd-7dk88\" (UID: \"7947ae81-e535-495f-b66f-72510cf91f65\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dk88"
Jul 28 22:16:39.104090 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.103970    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skfps\" (UniqueName: \"kubernetes.io/projected/18bee0e3-5641-4281-918c-8c46ca84019d-kube-api-access-skfps\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4k5b6\" (UID: \"18bee0e3-5641-4281-918c-8c46ca84019d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6"
Jul 28 22:16:39.104090 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.104037    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/18bee0e3-5641-4281-918c-8c46ca84019d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4k5b6\" (UID: \"18bee0e3-5641-4281-918c-8c46ca84019d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6"
Jul 28 22:16:39.169830 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.169805    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-68n4l"
Jul 28 22:16:39.190057 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:16:39.189979    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9d8qp\" is forbidden: User \"system:node:ip-10-0-141-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-102.ec2.internal' and this object" podUID="f6d85cc8-8e76-4b6f-988e-b05727efaed8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"
Jul 28 22:16:39.191382 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:16:39.191356    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gctwh\" is forbidden: User \"system:node:ip-10-0-141-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-102.ec2.internal' and this object" podUID="45556f1d-16ae-4f00-8029-4d65d32f1a22" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh"
Jul 28 22:16:39.204601 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.204568    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skfps\" (UniqueName: \"kubernetes.io/projected/18bee0e3-5641-4281-918c-8c46ca84019d-kube-api-access-skfps\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4k5b6\" (UID: \"18bee0e3-5641-4281-918c-8c46ca84019d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6"
Jul 28 22:16:39.204715 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.204670    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/18bee0e3-5641-4281-918c-8c46ca84019d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4k5b6\" (UID: \"18bee0e3-5641-4281-918c-8c46ca84019d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6"
Jul 28 22:16:39.204765 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.204751    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jgj4\" (UniqueName: \"kubernetes.io/projected/7947ae81-e535-495f-b66f-72510cf91f65-kube-api-access-4jgj4\") pod \"limitador-operator-controller-manager-859d7b78bd-7dk88\" (UID: \"7947ae81-e535-495f-b66f-72510cf91f65\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dk88"
Jul 28 22:16:39.205334 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.205312    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/18bee0e3-5641-4281-918c-8c46ca84019d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4k5b6\" (UID: \"18bee0e3-5641-4281-918c-8c46ca84019d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6"
Jul 28 22:16:39.214430 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.214407    2551 generic.go:356] "Generic (PLEG): container finished" podID="f6d85cc8-8e76-4b6f-988e-b05727efaed8" containerID="28198d8d8f9496d5e21254b99173f6ea32caabdd5fe637a1aede4634f905459d" exitCode=0
Jul 28 22:16:39.214552 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.214491    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6bfd0ff5a662b44df54ac46313ce54944de121d1d547f6ab5f8182ac1de9c9c6"
Jul 28 22:16:39.216568 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.216407    2551 generic.go:356] "Generic (PLEG): container finished" podID="45556f1d-16ae-4f00-8029-4d65d32f1a22" containerID="fdb72674c6b7d3a6c7d6f2195a5b53c7ce66f47d53f7527468cb572fe5434063" exitCode=0
Jul 28 22:16:39.216568 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.216483    2551 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae691e804e23a2a47a53569b5af04fb90e4c1434abaaaa62c32fdc2577813ae7"
Jul 28 22:16:39.219734 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.219708    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jgj4\" (UniqueName: \"kubernetes.io/projected/7947ae81-e535-495f-b66f-72510cf91f65-kube-api-access-4jgj4\") pod \"limitador-operator-controller-manager-859d7b78bd-7dk88\" (UID: \"7947ae81-e535-495f-b66f-72510cf91f65\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dk88"
Jul 28 22:16:39.220128 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.220103    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skfps\" (UniqueName: \"kubernetes.io/projected/18bee0e3-5641-4281-918c-8c46ca84019d-kube-api-access-skfps\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4k5b6\" (UID: \"18bee0e3-5641-4281-918c-8c46ca84019d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6"
Jul 28 22:16:39.220626 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.220610    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"
Jul 28 22:16:39.222204 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:16:39.222184    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gctwh\" is forbidden: User \"system:node:ip-10-0-141-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-102.ec2.internal' and this object" podUID="45556f1d-16ae-4f00-8029-4d65d32f1a22" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh"
Jul 28 22:16:39.223412 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:16:39.223393    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9d8qp\" is forbidden: User \"system:node:ip-10-0-141-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-102.ec2.internal' and this object" podUID="f6d85cc8-8e76-4b6f-988e-b05727efaed8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"
Jul 28 22:16:39.223581 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.223568    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh"
Jul 28 22:16:39.224981 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:16:39.224961    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9d8qp\" is forbidden: User \"system:node:ip-10-0-141-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-102.ec2.internal' and this object" podUID="f6d85cc8-8e76-4b6f-988e-b05727efaed8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"
Jul 28 22:16:39.226215 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:16:39.226192    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gctwh\" is forbidden: User \"system:node:ip-10-0-141-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-102.ec2.internal' and this object" podUID="45556f1d-16ae-4f00-8029-4d65d32f1a22" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh"
Jul 28 22:16:39.306130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.306095    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sd29f\" (UniqueName: \"kubernetes.io/projected/45556f1d-16ae-4f00-8029-4d65d32f1a22-kube-api-access-sd29f\") pod \"45556f1d-16ae-4f00-8029-4d65d32f1a22\" (UID: \"45556f1d-16ae-4f00-8029-4d65d32f1a22\") "
Jul 28 22:16:39.306130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.306132    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpbjz\" (UniqueName: \"kubernetes.io/projected/f6d85cc8-8e76-4b6f-988e-b05727efaed8-kube-api-access-qpbjz\") pod \"f6d85cc8-8e76-4b6f-988e-b05727efaed8\" (UID: \"f6d85cc8-8e76-4b6f-988e-b05727efaed8\") "
Jul 28 22:16:39.306369 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.306277    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6d85cc8-8e76-4b6f-988e-b05727efaed8-extensions-socket-volume\") pod \"f6d85cc8-8e76-4b6f-988e-b05727efaed8\" (UID: \"f6d85cc8-8e76-4b6f-988e-b05727efaed8\") "
Jul 28 22:16:39.306747 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.306716    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6d85cc8-8e76-4b6f-988e-b05727efaed8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f6d85cc8-8e76-4b6f-988e-b05727efaed8" (UID: "f6d85cc8-8e76-4b6f-988e-b05727efaed8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:16:39.308177 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.308153    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45556f1d-16ae-4f00-8029-4d65d32f1a22-kube-api-access-sd29f" (OuterVolumeSpecName: "kube-api-access-sd29f") pod "45556f1d-16ae-4f00-8029-4d65d32f1a22" (UID: "45556f1d-16ae-4f00-8029-4d65d32f1a22"). InnerVolumeSpecName "kube-api-access-sd29f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:16:39.308281 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.308192    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6d85cc8-8e76-4b6f-988e-b05727efaed8-kube-api-access-qpbjz" (OuterVolumeSpecName: "kube-api-access-qpbjz") pod "f6d85cc8-8e76-4b6f-988e-b05727efaed8" (UID: "f6d85cc8-8e76-4b6f-988e-b05727efaed8"). InnerVolumeSpecName "kube-api-access-qpbjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:16:39.367570 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.367504    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6"
Jul 28 22:16:39.373605 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.373583    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dk88"
Jul 28 22:16:39.407949 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.407913    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sd29f\" (UniqueName: \"kubernetes.io/projected/45556f1d-16ae-4f00-8029-4d65d32f1a22-kube-api-access-sd29f\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:16:39.408167 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.408150    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpbjz\" (UniqueName: \"kubernetes.io/projected/f6d85cc8-8e76-4b6f-988e-b05727efaed8-kube-api-access-qpbjz\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:16:39.408265 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.408253    2551 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f6d85cc8-8e76-4b6f-988e-b05727efaed8-extensions-socket-volume\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:16:39.515962 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.515933    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6"]
Jul 28 22:16:39.518607 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:16:39.518579    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18bee0e3_5641_4281_918c_8c46ca84019d.slice/crio-3a42ea9b1ceb3680a5b8fe5b288d12383d7000328a572fedde2ae69ad15449bf WatchSource:0}: Error finding container 3a42ea9b1ceb3680a5b8fe5b288d12383d7000328a572fedde2ae69ad15449bf: Status 404 returned error can't find the container with id 3a42ea9b1ceb3680a5b8fe5b288d12383d7000328a572fedde2ae69ad15449bf
Jul 28 22:16:39.538278 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:39.538256    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dk88"]
Jul 28 22:16:39.540330 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:16:39.540302    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7947ae81_e535_495f_b66f_72510cf91f65.slice/crio-98daa59ff97484f7363f72f8e9b1c24583366261b637f7523a185e1ac83c1bd3 WatchSource:0}: Error finding container 98daa59ff97484f7363f72f8e9b1c24583366261b637f7523a185e1ac83c1bd3: Status 404 returned error can't find the container with id 98daa59ff97484f7363f72f8e9b1c24583366261b637f7523a185e1ac83c1bd3
Jul 28 22:16:40.222870 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:40.222769    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dk88" event={"ID":"7947ae81-e535-495f-b66f-72510cf91f65","Type":"ContainerStarted","Data":"c795d4f4ac3fd2cbdb26f3304c845a741561b5c5b0c6992dd7047ca38f2f26f1"}
Jul 28 22:16:40.222870 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:40.222825    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dk88"
Jul 28 22:16:40.222870 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:40.222849    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dk88" event={"ID":"7947ae81-e535-495f-b66f-72510cf91f65","Type":"ContainerStarted","Data":"98daa59ff97484f7363f72f8e9b1c24583366261b637f7523a185e1ac83c1bd3"}
Jul 28 22:16:40.224364 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:40.224337    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6" event={"ID":"18bee0e3-5641-4281-918c-8c46ca84019d","Type":"ContainerStarted","Data":"f12cb6ed03d4757a2cf31ef5aceca51a6a68b5597b1fa99b4447700faabd5828"}
Jul 28 22:16:40.224481 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:40.224371    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh"
Jul 28 22:16:40.224481 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:40.224371    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"
Jul 28 22:16:40.224481 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:40.224382    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6" event={"ID":"18bee0e3-5641-4281-918c-8c46ca84019d","Type":"ContainerStarted","Data":"3a42ea9b1ceb3680a5b8fe5b288d12383d7000328a572fedde2ae69ad15449bf"}
Jul 28 22:16:40.224632 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:40.224518    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6"
Jul 28 22:16:40.225309 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:16:40.224914    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gctwh\" is forbidden: User \"system:node:ip-10-0-141-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-102.ec2.internal' and this object" podUID="45556f1d-16ae-4f00-8029-4d65d32f1a22" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh"
Jul 28 22:16:40.246700 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:16:40.246670    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9d8qp\" is forbidden: User \"system:node:ip-10-0-141-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-102.ec2.internal' and this object" podUID="f6d85cc8-8e76-4b6f-988e-b05727efaed8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"
Jul 28 22:16:40.247276 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:40.247240    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dk88" podStartSLOduration=2.247226224 podStartE2EDuration="2.247226224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:16:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:16:40.245098414 +0000 UTC m=+408.263064108" watchObservedRunningTime="2026-07-28 22:16:40.247226224 +0000 UTC m=+408.265191918"
Jul 28 22:16:40.250730 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:16:40.250707    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-9d8qp\" is forbidden: User \"system:node:ip-10-0-141-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-102.ec2.internal' and this object" podUID="f6d85cc8-8e76-4b6f-988e-b05727efaed8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-9d8qp"
Jul 28 22:16:40.273307 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:40.273259    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6" podStartSLOduration=2.273243897 podStartE2EDuration="2.273243897s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:16:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:16:40.271825465 +0000 UTC m=+408.289791160" watchObservedRunningTime="2026-07-28 22:16:40.273243897 +0000 UTC m=+408.291209591"
Jul 28 22:16:40.274034 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:16:40.274010    2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-gctwh\" is forbidden: User \"system:node:ip-10-0-141-102.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-102.ec2.internal' and this object" podUID="45556f1d-16ae-4f00-8029-4d65d32f1a22" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-gctwh"
Jul 28 22:16:40.462085 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:40.462048    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45556f1d-16ae-4f00-8029-4d65d32f1a22" path="/var/lib/kubelet/pods/45556f1d-16ae-4f00-8029-4d65d32f1a22/volumes"
Jul 28 22:16:40.462385 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:40.462373    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6d85cc8-8e76-4b6f-988e-b05727efaed8" path="/var/lib/kubelet/pods/f6d85cc8-8e76-4b6f-988e-b05727efaed8/volumes"
Jul 28 22:16:42.184387 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:42.184355    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5mgx4"
Jul 28 22:16:51.231472 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:51.231434    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6"
Jul 28 22:16:51.231989 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:16:51.231511    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-7dk88"
Jul 28 22:17:09.545193 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:09.545155    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6"]
Jul 28 22:17:09.545837 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:09.545441    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6" podUID="18bee0e3-5641-4281-918c-8c46ca84019d" containerName="manager" containerID="cri-o://f12cb6ed03d4757a2cf31ef5aceca51a6a68b5597b1fa99b4447700faabd5828" gracePeriod=10
Jul 28 22:17:09.774682 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:09.774649    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p"]
Jul 28 22:17:09.778198 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:09.778175    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p"
Jul 28 22:17:09.797564 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:09.797470    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p"]
Jul 28 22:17:09.809873 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:09.809844    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6"
Jul 28 22:17:09.863957 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:09.863929    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/18bee0e3-5641-4281-918c-8c46ca84019d-extensions-socket-volume\") pod \"18bee0e3-5641-4281-918c-8c46ca84019d\" (UID: \"18bee0e3-5641-4281-918c-8c46ca84019d\") "
Jul 28 22:17:09.864128 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:09.863964    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skfps\" (UniqueName: \"kubernetes.io/projected/18bee0e3-5641-4281-918c-8c46ca84019d-kube-api-access-skfps\") pod \"18bee0e3-5641-4281-918c-8c46ca84019d\" (UID: \"18bee0e3-5641-4281-918c-8c46ca84019d\") "
Jul 28 22:17:09.864128 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:09.864105    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6bhx\" (UniqueName: \"kubernetes.io/projected/f449c93a-3f7d-4562-a461-d814bd9a46b4-kube-api-access-l6bhx\") pod \"kuadrant-operator-controller-manager-86878c769f-djw5p\" (UID: \"f449c93a-3f7d-4562-a461-d814bd9a46b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p"
Jul 28 22:17:09.864225 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:09.864146    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f449c93a-3f7d-4562-a461-d814bd9a46b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-djw5p\" (UID: \"f449c93a-3f7d-4562-a461-d814bd9a46b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p"
Jul 28 22:17:09.864391 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:09.864360    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18bee0e3-5641-4281-918c-8c46ca84019d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "18bee0e3-5641-4281-918c-8c46ca84019d" (UID: "18bee0e3-5641-4281-918c-8c46ca84019d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:17:09.866018 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:09.865989    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18bee0e3-5641-4281-918c-8c46ca84019d-kube-api-access-skfps" (OuterVolumeSpecName: "kube-api-access-skfps") pod "18bee0e3-5641-4281-918c-8c46ca84019d" (UID: "18bee0e3-5641-4281-918c-8c46ca84019d"). InnerVolumeSpecName "kube-api-access-skfps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:17:09.964737 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:09.964697    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6bhx\" (UniqueName: \"kubernetes.io/projected/f449c93a-3f7d-4562-a461-d814bd9a46b4-kube-api-access-l6bhx\") pod \"kuadrant-operator-controller-manager-86878c769f-djw5p\" (UID: \"f449c93a-3f7d-4562-a461-d814bd9a46b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p"
Jul 28 22:17:09.964905 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:09.964754    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f449c93a-3f7d-4562-a461-d814bd9a46b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-djw5p\" (UID: \"f449c93a-3f7d-4562-a461-d814bd9a46b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p"
Jul 28 22:17:09.964905 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:09.964782    2551 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/18bee0e3-5641-4281-918c-8c46ca84019d-extensions-socket-volume\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:17:09.964905 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:09.964793    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skfps\" (UniqueName: \"kubernetes.io/projected/18bee0e3-5641-4281-918c-8c46ca84019d-kube-api-access-skfps\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:17:09.965104 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:09.965088    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f449c93a-3f7d-4562-a461-d814bd9a46b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-djw5p\" (UID: \"f449c93a-3f7d-4562-a461-d814bd9a46b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p"
Jul 28 22:17:09.972537 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:09.972492    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6bhx\" (UniqueName: \"kubernetes.io/projected/f449c93a-3f7d-4562-a461-d814bd9a46b4-kube-api-access-l6bhx\") pod \"kuadrant-operator-controller-manager-86878c769f-djw5p\" (UID: \"f449c93a-3f7d-4562-a461-d814bd9a46b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p"
Jul 28 22:17:10.105623 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:10.105587    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p"
Jul 28 22:17:10.237603 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:10.237576    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p"]
Jul 28 22:17:10.240157 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:17:10.240136    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf449c93a_3f7d_4562_a461_d814bd9a46b4.slice/crio-1106965438cd38f5c5797d36365f926e266a654ee7eb5f056d2bcef7f6fd71aa WatchSource:0}: Error finding container 1106965438cd38f5c5797d36365f926e266a654ee7eb5f056d2bcef7f6fd71aa: Status 404 returned error can't find the container with id 1106965438cd38f5c5797d36365f926e266a654ee7eb5f056d2bcef7f6fd71aa
Jul 28 22:17:10.360747 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:10.360658    2551 generic.go:356] "Generic (PLEG): container finished" podID="18bee0e3-5641-4281-918c-8c46ca84019d" containerID="f12cb6ed03d4757a2cf31ef5aceca51a6a68b5597b1fa99b4447700faabd5828" exitCode=0
Jul 28 22:17:10.360747 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:10.360702    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6"
Jul 28 22:17:10.360933 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:10.360747    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6" event={"ID":"18bee0e3-5641-4281-918c-8c46ca84019d","Type":"ContainerDied","Data":"f12cb6ed03d4757a2cf31ef5aceca51a6a68b5597b1fa99b4447700faabd5828"}
Jul 28 22:17:10.360933 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:10.360802    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6" event={"ID":"18bee0e3-5641-4281-918c-8c46ca84019d","Type":"ContainerDied","Data":"3a42ea9b1ceb3680a5b8fe5b288d12383d7000328a572fedde2ae69ad15449bf"}
Jul 28 22:17:10.360933 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:10.360831    2551 scope.go:117] "RemoveContainer" containerID="f12cb6ed03d4757a2cf31ef5aceca51a6a68b5597b1fa99b4447700faabd5828"
Jul 28 22:17:10.362585 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:10.362562    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p" event={"ID":"f449c93a-3f7d-4562-a461-d814bd9a46b4","Type":"ContainerStarted","Data":"cf36094c4ba79ee897a5611ab61ea4d14b417789a993a9ca4c2636ff0d97d7f3"}
Jul 28 22:17:10.362713 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:10.362604    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p" event={"ID":"f449c93a-3f7d-4562-a461-d814bd9a46b4","Type":"ContainerStarted","Data":"1106965438cd38f5c5797d36365f926e266a654ee7eb5f056d2bcef7f6fd71aa"}
Jul 28 22:17:10.362713 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:10.362642    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p"
Jul 28 22:17:10.369352 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:10.369335    2551 scope.go:117] "RemoveContainer" containerID="f12cb6ed03d4757a2cf31ef5aceca51a6a68b5597b1fa99b4447700faabd5828"
Jul 28 22:17:10.369631 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:17:10.369611    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f12cb6ed03d4757a2cf31ef5aceca51a6a68b5597b1fa99b4447700faabd5828\": container with ID starting with f12cb6ed03d4757a2cf31ef5aceca51a6a68b5597b1fa99b4447700faabd5828 not found: ID does not exist" containerID="f12cb6ed03d4757a2cf31ef5aceca51a6a68b5597b1fa99b4447700faabd5828"
Jul 28 22:17:10.369677 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:10.369638    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f12cb6ed03d4757a2cf31ef5aceca51a6a68b5597b1fa99b4447700faabd5828"} err="failed to get container status \"f12cb6ed03d4757a2cf31ef5aceca51a6a68b5597b1fa99b4447700faabd5828\": rpc error: code = NotFound desc = could not find container \"f12cb6ed03d4757a2cf31ef5aceca51a6a68b5597b1fa99b4447700faabd5828\": container with ID starting with f12cb6ed03d4757a2cf31ef5aceca51a6a68b5597b1fa99b4447700faabd5828 not found: ID does not exist"
Jul 28 22:17:10.380708 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:10.380665    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p" podStartSLOduration=1.380651364 podStartE2EDuration="1.380651364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:17:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:17:10.379755136 +0000 UTC m=+438.397720848" watchObservedRunningTime="2026-07-28 22:17:10.380651364 +0000 UTC m=+438.398617057"
Jul 28 22:17:10.398340 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:10.398313    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6"]
Jul 28 22:17:10.402493 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:10.402469    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4k5b6"]
Jul 28 22:17:10.462456 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:10.462421    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18bee0e3-5641-4281-918c-8c46ca84019d" path="/var/lib/kubelet/pods/18bee0e3-5641-4281-918c-8c46ca84019d/volumes"
Jul 28 22:17:21.370868 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:21.370836    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p"
Jul 28 22:17:29.581747 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:29.581715    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:17:29.582197 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:29.582032    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18bee0e3-5641-4281-918c-8c46ca84019d" containerName="manager"
Jul 28 22:17:29.582197 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:29.582043    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="18bee0e3-5641-4281-918c-8c46ca84019d" containerName="manager"
Jul 28 22:17:29.582197 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:29.582092    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="18bee0e3-5641-4281-918c-8c46ca84019d" containerName="manager"
Jul 28 22:17:29.585095 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:29.585072    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-rztht"
Jul 28 22:17:29.587003 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:29.586974    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 28 22:17:29.587129 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:29.587017    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-j2rx6\""
Jul 28 22:17:29.594935 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:29.594909    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:17:29.626276 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:29.626242    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:17:29.738219 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:29.738187    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/38118b91-a8ba-47a3-9fb6-bac0869b17ab-config-file\") pod \"limitador-limitador-54d6d988d5-rztht\" (UID: \"38118b91-a8ba-47a3-9fb6-bac0869b17ab\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rztht"
Jul 28 22:17:29.738410 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:29.738232    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtskv\" (UniqueName: \"kubernetes.io/projected/38118b91-a8ba-47a3-9fb6-bac0869b17ab-kube-api-access-qtskv\") pod \"limitador-limitador-54d6d988d5-rztht\" (UID: \"38118b91-a8ba-47a3-9fb6-bac0869b17ab\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rztht"
Jul 28 22:17:29.839385 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:29.839288    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/38118b91-a8ba-47a3-9fb6-bac0869b17ab-config-file\") pod \"limitador-limitador-54d6d988d5-rztht\" (UID: \"38118b91-a8ba-47a3-9fb6-bac0869b17ab\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rztht"
Jul 28 22:17:29.839385 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:29.839326    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtskv\" (UniqueName: \"kubernetes.io/projected/38118b91-a8ba-47a3-9fb6-bac0869b17ab-kube-api-access-qtskv\") pod \"limitador-limitador-54d6d988d5-rztht\" (UID: \"38118b91-a8ba-47a3-9fb6-bac0869b17ab\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rztht"
Jul 28 22:17:29.839948 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:29.839926    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/38118b91-a8ba-47a3-9fb6-bac0869b17ab-config-file\") pod \"limitador-limitador-54d6d988d5-rztht\" (UID: \"38118b91-a8ba-47a3-9fb6-bac0869b17ab\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rztht"
Jul 28 22:17:29.846786 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:29.846760    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtskv\" (UniqueName: \"kubernetes.io/projected/38118b91-a8ba-47a3-9fb6-bac0869b17ab-kube-api-access-qtskv\") pod \"limitador-limitador-54d6d988d5-rztht\" (UID: \"38118b91-a8ba-47a3-9fb6-bac0869b17ab\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-rztht"
Jul 28 22:17:29.896410 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:29.896374    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-rztht"
Jul 28 22:17:30.028759 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:30.028736    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:17:30.031441 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:17:30.031415    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38118b91_a8ba_47a3_9fb6_bac0869b17ab.slice/crio-de1ccc01e0f001b0421b6f6480065c01186e7fc9f0bffb3a6efff77756672fc6 WatchSource:0}: Error finding container de1ccc01e0f001b0421b6f6480065c01186e7fc9f0bffb3a6efff77756672fc6: Status 404 returned error can't find the container with id de1ccc01e0f001b0421b6f6480065c01186e7fc9f0bffb3a6efff77756672fc6
Jul 28 22:17:30.179631 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:30.179546    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-t982g"]
Jul 28 22:17:30.183914 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:30.183892    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t982g"
Jul 28 22:17:30.186340 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:30.186315    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6cgks\""
Jul 28 22:17:30.191130 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:30.191105    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t982g"]
Jul 28 22:17:30.243340 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:30.243312    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nslsq\" (UniqueName: \"kubernetes.io/projected/9fbf7e50-8b11-4d52-b6b8-9d6536da33a7-kube-api-access-nslsq\") pod \"authorino-75d556bd77-t982g\" (UID: \"9fbf7e50-8b11-4d52-b6b8-9d6536da33a7\") " pod="kuadrant-system/authorino-75d556bd77-t982g"
Jul 28 22:17:30.344374 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:30.344333    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nslsq\" (UniqueName: \"kubernetes.io/projected/9fbf7e50-8b11-4d52-b6b8-9d6536da33a7-kube-api-access-nslsq\") pod \"authorino-75d556bd77-t982g\" (UID: \"9fbf7e50-8b11-4d52-b6b8-9d6536da33a7\") " pod="kuadrant-system/authorino-75d556bd77-t982g"
Jul 28 22:17:30.352760 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:30.352723    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nslsq\" (UniqueName: \"kubernetes.io/projected/9fbf7e50-8b11-4d52-b6b8-9d6536da33a7-kube-api-access-nslsq\") pod \"authorino-75d556bd77-t982g\" (UID: \"9fbf7e50-8b11-4d52-b6b8-9d6536da33a7\") " pod="kuadrant-system/authorino-75d556bd77-t982g"
Jul 28 22:17:30.464744 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:30.464663    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-rztht" event={"ID":"38118b91-a8ba-47a3-9fb6-bac0869b17ab","Type":"ContainerStarted","Data":"de1ccc01e0f001b0421b6f6480065c01186e7fc9f0bffb3a6efff77756672fc6"}
Jul 28 22:17:30.494807 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:30.494769    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t982g"
Jul 28 22:17:30.700296 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:30.700252    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t982g"]
Jul 28 22:17:30.705293 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:17:30.705261    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9fbf7e50_8b11_4d52_b6b8_9d6536da33a7.slice/crio-50cd548878dadc6db97d334443838d7a503ed56e83a40fe43111ba0ab499ebdf WatchSource:0}: Error finding container 50cd548878dadc6db97d334443838d7a503ed56e83a40fe43111ba0ab499ebdf: Status 404 returned error can't find the container with id 50cd548878dadc6db97d334443838d7a503ed56e83a40fe43111ba0ab499ebdf
Jul 28 22:17:31.463412 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:31.463377    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t982g" event={"ID":"9fbf7e50-8b11-4d52-b6b8-9d6536da33a7","Type":"ContainerStarted","Data":"50cd548878dadc6db97d334443838d7a503ed56e83a40fe43111ba0ab499ebdf"}
Jul 28 22:17:33.477983 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:33.477939    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-rztht" event={"ID":"38118b91-a8ba-47a3-9fb6-bac0869b17ab","Type":"ContainerStarted","Data":"bd7fd9a24f8b5ac6ca03aec0454fa8a840624ab2eb4edadab02cea549900a9bf"}
Jul 28 22:17:33.478467 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:33.478040    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-rztht"
Jul 28 22:17:33.494063 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:33.493938    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-rztht" podStartSLOduration=1.948306292 podStartE2EDuration="4.493915149s" totalImagesPullingTime="2.545608857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:17:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:17:33.492437792 +0000 UTC m=+461.510403487" watchObservedRunningTime="2026-07-28 22:17:33.493915149 +0000 UTC m=+461.511880846"
Jul 28 22:17:35.488845 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:35.488811    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t982g" event={"ID":"9fbf7e50-8b11-4d52-b6b8-9d6536da33a7","Type":"ContainerStarted","Data":"0fda4cb8b22c88aadddc0280121fb1cfbada4082619300ed15394a5a5f1d4948"}
Jul 28 22:17:35.505081 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:35.505028    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-t982g" podStartSLOduration=1.347004592 podStartE2EDuration="5.505009924s" totalImagesPullingTime="4.158005332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:17:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:17:35.503695405 +0000 UTC m=+463.521661100" watchObservedRunningTime="2026-07-28 22:17:35.505009924 +0000 UTC m=+463.522975618"
Jul 28 22:17:44.483378 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:17:44.483346    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-rztht"
Jul 28 22:18:02.917608 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:18:02.917576    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t982g"]
Jul 28 22:18:02.918022 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:18:02.917772    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-t982g" podUID="9fbf7e50-8b11-4d52-b6b8-9d6536da33a7" containerName="authorino" containerID="cri-o://0fda4cb8b22c88aadddc0280121fb1cfbada4082619300ed15394a5a5f1d4948" gracePeriod=30
Jul 28 22:18:03.171380 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:18:03.171321    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t982g"
Jul 28 22:18:03.219429 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:18:03.219391    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nslsq\" (UniqueName: \"kubernetes.io/projected/9fbf7e50-8b11-4d52-b6b8-9d6536da33a7-kube-api-access-nslsq\") pod \"9fbf7e50-8b11-4d52-b6b8-9d6536da33a7\" (UID: \"9fbf7e50-8b11-4d52-b6b8-9d6536da33a7\") "
Jul 28 22:18:03.221437 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:18:03.221414    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fbf7e50-8b11-4d52-b6b8-9d6536da33a7-kube-api-access-nslsq" (OuterVolumeSpecName: "kube-api-access-nslsq") pod "9fbf7e50-8b11-4d52-b6b8-9d6536da33a7" (UID: "9fbf7e50-8b11-4d52-b6b8-9d6536da33a7"). InnerVolumeSpecName "kube-api-access-nslsq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:18:03.320724 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:18:03.320690    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nslsq\" (UniqueName: \"kubernetes.io/projected/9fbf7e50-8b11-4d52-b6b8-9d6536da33a7-kube-api-access-nslsq\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:18:03.610468 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:18:03.610431    2551 generic.go:356] "Generic (PLEG): container finished" podID="9fbf7e50-8b11-4d52-b6b8-9d6536da33a7" containerID="0fda4cb8b22c88aadddc0280121fb1cfbada4082619300ed15394a5a5f1d4948" exitCode=0
Jul 28 22:18:03.610655 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:18:03.610485    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-t982g"
Jul 28 22:18:03.610655 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:18:03.610507    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t982g" event={"ID":"9fbf7e50-8b11-4d52-b6b8-9d6536da33a7","Type":"ContainerDied","Data":"0fda4cb8b22c88aadddc0280121fb1cfbada4082619300ed15394a5a5f1d4948"}
Jul 28 22:18:03.610655 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:18:03.610582    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-t982g" event={"ID":"9fbf7e50-8b11-4d52-b6b8-9d6536da33a7","Type":"ContainerDied","Data":"50cd548878dadc6db97d334443838d7a503ed56e83a40fe43111ba0ab499ebdf"}
Jul 28 22:18:03.610655 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:18:03.610600    2551 scope.go:117] "RemoveContainer" containerID="0fda4cb8b22c88aadddc0280121fb1cfbada4082619300ed15394a5a5f1d4948"
Jul 28 22:18:03.619587 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:18:03.619567    2551 scope.go:117] "RemoveContainer" containerID="0fda4cb8b22c88aadddc0280121fb1cfbada4082619300ed15394a5a5f1d4948"
Jul 28 22:18:03.619880 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:18:03.619860    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fda4cb8b22c88aadddc0280121fb1cfbada4082619300ed15394a5a5f1d4948\": container with ID starting with 0fda4cb8b22c88aadddc0280121fb1cfbada4082619300ed15394a5a5f1d4948 not found: ID does not exist" containerID="0fda4cb8b22c88aadddc0280121fb1cfbada4082619300ed15394a5a5f1d4948"
Jul 28 22:18:03.619954 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:18:03.619893    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fda4cb8b22c88aadddc0280121fb1cfbada4082619300ed15394a5a5f1d4948"} err="failed to get container status \"0fda4cb8b22c88aadddc0280121fb1cfbada4082619300ed15394a5a5f1d4948\": rpc error: code = NotFound desc = could not find container \"0fda4cb8b22c88aadddc0280121fb1cfbada4082619300ed15394a5a5f1d4948\": container with ID starting with 0fda4cb8b22c88aadddc0280121fb1cfbada4082619300ed15394a5a5f1d4948 not found: ID does not exist"
Jul 28 22:18:03.632236 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:18:03.632206    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t982g"]
Jul 28 22:18:03.640549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:18:03.640502    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-t982g"]
Jul 28 22:18:04.463897 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:18:04.463864    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fbf7e50-8b11-4d52-b6b8-9d6536da33a7" path="/var/lib/kubelet/pods/9fbf7e50-8b11-4d52-b6b8-9d6536da33a7/volumes"
Jul 28 22:18:44.443983 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:18:44.443777    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:19:04.298154 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:04.298121    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:19:12.765909 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:12.765871    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:19:26.670967 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:26.670931    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:19:34.168614 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:34.168569    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:19:43.675371 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.675335    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"]
Jul 28 22:19:43.675882 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.675675    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fbf7e50-8b11-4d52-b6b8-9d6536da33a7" containerName="authorino"
Jul 28 22:19:43.675882 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.675688    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fbf7e50-8b11-4d52-b6b8-9d6536da33a7" containerName="authorino"
Jul 28 22:19:43.675882 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.675750    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fbf7e50-8b11-4d52-b6b8-9d6536da33a7" containerName="authorino"
Jul 28 22:19:43.678870 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.678847    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:43.681535 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.681500    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 22:19:43.681664 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.681554    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-72d9n\""
Jul 28 22:19:43.681664 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.681554    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 22:19:43.681664 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.681554    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 28 22:19:43.686731 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.686712    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"]
Jul 28 22:19:43.736669 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.736631    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vljzx\" (UniqueName: \"kubernetes.io/projected/16e277d1-9dd8-42a8-91fe-4f35fde4c4b2-kube-api-access-vljzx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ffjt6\" (UID: \"16e277d1-9dd8-42a8-91fe-4f35fde4c4b2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:43.736977 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.736946    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16e277d1-9dd8-42a8-91fe-4f35fde4c4b2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ffjt6\" (UID: \"16e277d1-9dd8-42a8-91fe-4f35fde4c4b2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:43.737099 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.737012    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16e277d1-9dd8-42a8-91fe-4f35fde4c4b2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ffjt6\" (UID: \"16e277d1-9dd8-42a8-91fe-4f35fde4c4b2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:43.737099 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.737062    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16e277d1-9dd8-42a8-91fe-4f35fde4c4b2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ffjt6\" (UID: \"16e277d1-9dd8-42a8-91fe-4f35fde4c4b2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:43.737182 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.737098    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16e277d1-9dd8-42a8-91fe-4f35fde4c4b2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ffjt6\" (UID: \"16e277d1-9dd8-42a8-91fe-4f35fde4c4b2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:43.737182 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.737133    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16e277d1-9dd8-42a8-91fe-4f35fde4c4b2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ffjt6\" (UID: \"16e277d1-9dd8-42a8-91fe-4f35fde4c4b2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:43.837818 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.837776    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16e277d1-9dd8-42a8-91fe-4f35fde4c4b2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ffjt6\" (UID: \"16e277d1-9dd8-42a8-91fe-4f35fde4c4b2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:43.838001 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.837829    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16e277d1-9dd8-42a8-91fe-4f35fde4c4b2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ffjt6\" (UID: \"16e277d1-9dd8-42a8-91fe-4f35fde4c4b2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:43.838001 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.837854    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16e277d1-9dd8-42a8-91fe-4f35fde4c4b2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ffjt6\" (UID: \"16e277d1-9dd8-42a8-91fe-4f35fde4c4b2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:43.838001 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.837877    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16e277d1-9dd8-42a8-91fe-4f35fde4c4b2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ffjt6\" (UID: \"16e277d1-9dd8-42a8-91fe-4f35fde4c4b2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:43.838001 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.837900    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16e277d1-9dd8-42a8-91fe-4f35fde4c4b2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ffjt6\" (UID: \"16e277d1-9dd8-42a8-91fe-4f35fde4c4b2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:43.838001 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.837932    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vljzx\" (UniqueName: \"kubernetes.io/projected/16e277d1-9dd8-42a8-91fe-4f35fde4c4b2-kube-api-access-vljzx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ffjt6\" (UID: \"16e277d1-9dd8-42a8-91fe-4f35fde4c4b2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:43.838264 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.838237    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/16e277d1-9dd8-42a8-91fe-4f35fde4c4b2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ffjt6\" (UID: \"16e277d1-9dd8-42a8-91fe-4f35fde4c4b2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:43.838324 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.838266    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16e277d1-9dd8-42a8-91fe-4f35fde4c4b2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ffjt6\" (UID: \"16e277d1-9dd8-42a8-91fe-4f35fde4c4b2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:43.838324 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.838291    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16e277d1-9dd8-42a8-91fe-4f35fde4c4b2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ffjt6\" (UID: \"16e277d1-9dd8-42a8-91fe-4f35fde4c4b2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:43.840128 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.840104    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16e277d1-9dd8-42a8-91fe-4f35fde4c4b2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ffjt6\" (UID: \"16e277d1-9dd8-42a8-91fe-4f35fde4c4b2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:43.840374 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.840353    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16e277d1-9dd8-42a8-91fe-4f35fde4c4b2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ffjt6\" (UID: \"16e277d1-9dd8-42a8-91fe-4f35fde4c4b2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:43.845715 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.845693    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vljzx\" (UniqueName: \"kubernetes.io/projected/16e277d1-9dd8-42a8-91fe-4f35fde4c4b2-kube-api-access-vljzx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ffjt6\" (UID: \"16e277d1-9dd8-42a8-91fe-4f35fde4c4b2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:43.990103 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:43.990004    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:44.122371 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:44.122343    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"]
Jul 28 22:19:44.124988 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:19:44.124958    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16e277d1_9dd8_42a8_91fe_4f35fde4c4b2.slice/crio-996d946e1c263bd92582a347f8997a7cb700001b6eac7b2a95ea20b5b66cf5bc WatchSource:0}: Error finding container 996d946e1c263bd92582a347f8997a7cb700001b6eac7b2a95ea20b5b66cf5bc: Status 404 returned error can't find the container with id 996d946e1c263bd92582a347f8997a7cb700001b6eac7b2a95ea20b5b66cf5bc
Jul 28 22:19:45.067908 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:45.067875    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6" event={"ID":"16e277d1-9dd8-42a8-91fe-4f35fde4c4b2","Type":"ContainerStarted","Data":"996d946e1c263bd92582a347f8997a7cb700001b6eac7b2a95ea20b5b66cf5bc"}
Jul 28 22:19:46.073966 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:46.073933    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6" event={"ID":"16e277d1-9dd8-42a8-91fe-4f35fde4c4b2","Type":"ContainerStarted","Data":"76912aad49a25d000841017c5a4585dc06fc671145c7a7ac27889eeab69ded6b"}
Jul 28 22:19:46.090381 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:46.090322    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6" podStartSLOduration=2.124060773 podStartE2EDuration="3.090303233s" totalImagesPullingTime="966.24246ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:19:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:19:46.088734441 +0000 UTC m=+594.106700146" watchObservedRunningTime="2026-07-28 22:19:46.090303233 +0000 UTC m=+594.108269063"
Jul 28 22:19:46.676065 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:46.676021    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:19:51.465283 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.465251    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"]
Jul 28 22:19:51.468376 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.468356    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:19:51.470422 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.470404    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 28 22:19:51.479713 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.479685    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"]
Jul 28 22:19:51.605885 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.605848    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0131959-e6ab-4db7-b8dd-8f876c8d3d88-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm\" (UID: \"d0131959-e6ab-4db7-b8dd-8f876c8d3d88\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:19:51.605885 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.605899    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0131959-e6ab-4db7-b8dd-8f876c8d3d88-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm\" (UID: \"d0131959-e6ab-4db7-b8dd-8f876c8d3d88\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:19:51.606093 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.605968    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0131959-e6ab-4db7-b8dd-8f876c8d3d88-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm\" (UID: \"d0131959-e6ab-4db7-b8dd-8f876c8d3d88\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:19:51.606093 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.606006    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg4d4\" (UniqueName: \"kubernetes.io/projected/d0131959-e6ab-4db7-b8dd-8f876c8d3d88-kube-api-access-xg4d4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm\" (UID: \"d0131959-e6ab-4db7-b8dd-8f876c8d3d88\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:19:51.606161 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.606095    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0131959-e6ab-4db7-b8dd-8f876c8d3d88-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm\" (UID: \"d0131959-e6ab-4db7-b8dd-8f876c8d3d88\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:19:51.606161 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.606119    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0131959-e6ab-4db7-b8dd-8f876c8d3d88-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm\" (UID: \"d0131959-e6ab-4db7-b8dd-8f876c8d3d88\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:19:51.706844 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.706800    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0131959-e6ab-4db7-b8dd-8f876c8d3d88-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm\" (UID: \"d0131959-e6ab-4db7-b8dd-8f876c8d3d88\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:19:51.707045 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.706864    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0131959-e6ab-4db7-b8dd-8f876c8d3d88-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm\" (UID: \"d0131959-e6ab-4db7-b8dd-8f876c8d3d88\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:19:51.707045 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.706887    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg4d4\" (UniqueName: \"kubernetes.io/projected/d0131959-e6ab-4db7-b8dd-8f876c8d3d88-kube-api-access-xg4d4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm\" (UID: \"d0131959-e6ab-4db7-b8dd-8f876c8d3d88\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:19:51.707045 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.706945    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0131959-e6ab-4db7-b8dd-8f876c8d3d88-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm\" (UID: \"d0131959-e6ab-4db7-b8dd-8f876c8d3d88\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:19:51.707045 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.706967    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0131959-e6ab-4db7-b8dd-8f876c8d3d88-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm\" (UID: \"d0131959-e6ab-4db7-b8dd-8f876c8d3d88\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:19:51.707045 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.707002    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0131959-e6ab-4db7-b8dd-8f876c8d3d88-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm\" (UID: \"d0131959-e6ab-4db7-b8dd-8f876c8d3d88\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:19:51.707354 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.707332    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0131959-e6ab-4db7-b8dd-8f876c8d3d88-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm\" (UID: \"d0131959-e6ab-4db7-b8dd-8f876c8d3d88\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:19:51.707417 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.707389    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0131959-e6ab-4db7-b8dd-8f876c8d3d88-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm\" (UID: \"d0131959-e6ab-4db7-b8dd-8f876c8d3d88\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:19:51.707475 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.707421    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0131959-e6ab-4db7-b8dd-8f876c8d3d88-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm\" (UID: \"d0131959-e6ab-4db7-b8dd-8f876c8d3d88\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:19:51.709174 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.709153    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0131959-e6ab-4db7-b8dd-8f876c8d3d88-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm\" (UID: \"d0131959-e6ab-4db7-b8dd-8f876c8d3d88\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:19:51.709336 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.709312    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0131959-e6ab-4db7-b8dd-8f876c8d3d88-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm\" (UID: \"d0131959-e6ab-4db7-b8dd-8f876c8d3d88\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:19:51.714956 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.714926    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg4d4\" (UniqueName: \"kubernetes.io/projected/d0131959-e6ab-4db7-b8dd-8f876c8d3d88-kube-api-access-xg4d4\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm\" (UID: \"d0131959-e6ab-4db7-b8dd-8f876c8d3d88\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:19:51.777815 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.777738    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:19:51.915492 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:51.915460    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"]
Jul 28 22:19:51.918744 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:19:51.918713    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0131959_e6ab_4db7_b8dd_8f876c8d3d88.slice/crio-8fd806a9daac2721002f3d491cb0e570c7b7861c6b74178cf6dd5da7f2e49e81 WatchSource:0}: Error finding container 8fd806a9daac2721002f3d491cb0e570c7b7861c6b74178cf6dd5da7f2e49e81: Status 404 returned error can't find the container with id 8fd806a9daac2721002f3d491cb0e570c7b7861c6b74178cf6dd5da7f2e49e81
Jul 28 22:19:52.103596 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:52.103569    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm" event={"ID":"d0131959-e6ab-4db7-b8dd-8f876c8d3d88","Type":"ContainerStarted","Data":"8fd806a9daac2721002f3d491cb0e570c7b7861c6b74178cf6dd5da7f2e49e81"}
Jul 28 22:19:53.109727 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:53.109695    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm" event={"ID":"d0131959-e6ab-4db7-b8dd-8f876c8d3d88","Type":"ContainerStarted","Data":"0cda13821352aff956176b6958f04bf72c9782714d65c8e0f21e40da9a6685f4"}
Jul 28 22:19:53.126086 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:53.126039    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm" podStartSLOduration=1.961169293 podStartE2EDuration="2.126022677s" totalImagesPullingTime="164.853384ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:19:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:19:53.12495269 +0000 UTC m=+601.142918384" watchObservedRunningTime="2026-07-28 22:19:53.126022677 +0000 UTC m=+601.143988370"
Jul 28 22:19:53.990701 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:53.990662    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:53.992865 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:53.990751    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:54.003615 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:54.003586    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:54.125973 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:54.125938    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ffjt6"
Jul 28 22:19:54.903105 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:54.903076    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:19:54.903272 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:19:54.903076    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:20:00.774997 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:20:00.774960    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:20:01.777996 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:20:01.777967    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:20:01.778439 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:20:01.778010    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:20:01.790004 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:20:01.789975    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:20:02.163809 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:20:02.163777    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm"
Jul 28 22:20:58.540314 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:20:58.540275    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-675879f779-qcpzc"]
Jul 28 22:20:58.543580 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:20:58.543559    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-675879f779-qcpzc"
Jul 28 22:20:58.546418 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:20:58.546399    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6cgks\""
Jul 28 22:20:58.546520 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:20:58.546405    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 28 22:20:58.550935 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:20:58.550910    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-675879f779-qcpzc"]
Jul 28 22:20:58.590164 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:20:58.590125    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4s6d\" (UniqueName: \"kubernetes.io/projected/0f53b7be-c8b1-4ee8-b101-2c8b360b287f-kube-api-access-j4s6d\") pod \"authorino-675879f779-qcpzc\" (UID: \"0f53b7be-c8b1-4ee8-b101-2c8b360b287f\") " pod="kuadrant-system/authorino-675879f779-qcpzc"
Jul 28 22:20:58.590318 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:20:58.590228    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0f53b7be-c8b1-4ee8-b101-2c8b360b287f-tls-cert\") pod \"authorino-675879f779-qcpzc\" (UID: \"0f53b7be-c8b1-4ee8-b101-2c8b360b287f\") " pod="kuadrant-system/authorino-675879f779-qcpzc"
Jul 28 22:20:58.698085 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:20:58.698044    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4s6d\" (UniqueName: \"kubernetes.io/projected/0f53b7be-c8b1-4ee8-b101-2c8b360b287f-kube-api-access-j4s6d\") pod \"authorino-675879f779-qcpzc\" (UID: \"0f53b7be-c8b1-4ee8-b101-2c8b360b287f\") " pod="kuadrant-system/authorino-675879f779-qcpzc"
Jul 28 22:20:58.698309 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:20:58.698285    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0f53b7be-c8b1-4ee8-b101-2c8b360b287f-tls-cert\") pod \"authorino-675879f779-qcpzc\" (UID: \"0f53b7be-c8b1-4ee8-b101-2c8b360b287f\") " pod="kuadrant-system/authorino-675879f779-qcpzc"
Jul 28 22:20:58.700697 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:20:58.700669    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0f53b7be-c8b1-4ee8-b101-2c8b360b287f-tls-cert\") pod \"authorino-675879f779-qcpzc\" (UID: \"0f53b7be-c8b1-4ee8-b101-2c8b360b287f\") " pod="kuadrant-system/authorino-675879f779-qcpzc"
Jul 28 22:20:58.708013 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:20:58.707988    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4s6d\" (UniqueName: \"kubernetes.io/projected/0f53b7be-c8b1-4ee8-b101-2c8b360b287f-kube-api-access-j4s6d\") pod \"authorino-675879f779-qcpzc\" (UID: \"0f53b7be-c8b1-4ee8-b101-2c8b360b287f\") " pod="kuadrant-system/authorino-675879f779-qcpzc"
Jul 28 22:20:58.853267 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:20:58.853233    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-675879f779-qcpzc"
Jul 28 22:20:58.988343 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:20:58.988291    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-675879f779-qcpzc"]
Jul 28 22:20:58.990656 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:20:58.990625    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0f53b7be_c8b1_4ee8_b101_2c8b360b287f.slice/crio-840a94286b4a7f7724b094d4fa1677d2510f4fbc84c1bf47492548009fdb6c0d WatchSource:0}: Error finding container 840a94286b4a7f7724b094d4fa1677d2510f4fbc84c1bf47492548009fdb6c0d: Status 404 returned error can't find the container with id 840a94286b4a7f7724b094d4fa1677d2510f4fbc84c1bf47492548009fdb6c0d
Jul 28 22:20:58.991934 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:20:58.991912    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 22:20:59.411149 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:20:59.411108    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-675879f779-qcpzc" event={"ID":"0f53b7be-c8b1-4ee8-b101-2c8b360b287f","Type":"ContainerStarted","Data":"840a94286b4a7f7724b094d4fa1677d2510f4fbc84c1bf47492548009fdb6c0d"}
Jul 28 22:21:00.417397 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:21:00.417344    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-675879f779-qcpzc" event={"ID":"0f53b7be-c8b1-4ee8-b101-2c8b360b287f","Type":"ContainerStarted","Data":"0bd8513d11d53773fe532efecb842498f7f4c678a4c64d8fd6b6d2e67936f216"}
Jul 28 22:21:00.438238 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:21:00.438170    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-675879f779-qcpzc" podStartSLOduration=1.743188567 podStartE2EDuration="2.438150625s" totalImagesPullingTime="694.962058ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:21:00.436801069 +0000 UTC m=+668.454766764" watchObservedRunningTime="2026-07-28 22:21:00.438150625 +0000 UTC m=+668.456116321"
Jul 28 22:21:29.491990 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:21:29.491947    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:21:39.078021 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:21:39.077990    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:21:48.275498 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:21:48.275412    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:22:52.380537 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:22:52.380493    2551 scope.go:117] "RemoveContainer" containerID="fdb72674c6b7d3a6c7d6f2195a5b53c7ce66f47d53f7527468cb572fe5434063"
Jul 28 22:22:52.388705 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:22:52.388686    2551 scope.go:117] "RemoveContainer" containerID="28198d8d8f9496d5e21254b99173f6ea32caabdd5fe637a1aede4634f905459d"
Jul 28 22:23:29.574466 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:23:29.574424    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:23:40.568128 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:23:40.568090    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:24:17.167764 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:24:17.167727    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:24:26.266718 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:24:26.266681    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:24:35.972332 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:24:35.972293    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:24:43.868025 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:24:43.867985    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:24:54.273078 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:24:54.273035    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:24:54.929736 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:24:54.929707    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:24:54.931201 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:24:54.931181    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:25:04.473848 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:25:04.473811    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:25:15.375408 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:25:15.375372    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:25:25.575224 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:25:25.575181    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:25:47.668179 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:25:47.668145    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:25:59.175127 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:25:59.175085    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:26:22.874042 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:26:22.874007    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:26:33.779396 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:26:33.779353    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:26:45.178039 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:26:45.178003    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:27:01.074323 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:27:01.074287    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:27:12.866703 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:27:12.866666    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:27:22.562941 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:27:22.562898    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:27:32.167805 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:27:32.167771    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:27:41.968638 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:27:41.968599    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:28:06.257839 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:28:06.257749    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p"]
Jul 28 22:28:06.258285 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:28:06.258011    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p" podUID="f449c93a-3f7d-4562-a461-d814bd9a46b4" containerName="manager" containerID="cri-o://cf36094c4ba79ee897a5611ab61ea4d14b417789a993a9ca4c2636ff0d97d7f3" gracePeriod=10
Jul 28 22:28:08.813068 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:28:08.813036    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p"
Jul 28 22:28:08.953409 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:28:08.953321    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f449c93a-3f7d-4562-a461-d814bd9a46b4-extensions-socket-volume\") pod \"f449c93a-3f7d-4562-a461-d814bd9a46b4\" (UID: \"f449c93a-3f7d-4562-a461-d814bd9a46b4\") "
Jul 28 22:28:08.953575 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:28:08.953417    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6bhx\" (UniqueName: \"kubernetes.io/projected/f449c93a-3f7d-4562-a461-d814bd9a46b4-kube-api-access-l6bhx\") pod \"f449c93a-3f7d-4562-a461-d814bd9a46b4\" (UID: \"f449c93a-3f7d-4562-a461-d814bd9a46b4\") "
Jul 28 22:28:08.953728 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:28:08.953703    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f449c93a-3f7d-4562-a461-d814bd9a46b4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f449c93a-3f7d-4562-a461-d814bd9a46b4" (UID: "f449c93a-3f7d-4562-a461-d814bd9a46b4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:28:08.955401 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:28:08.955367    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f449c93a-3f7d-4562-a461-d814bd9a46b4-kube-api-access-l6bhx" (OuterVolumeSpecName: "kube-api-access-l6bhx") pod "f449c93a-3f7d-4562-a461-d814bd9a46b4" (UID: "f449c93a-3f7d-4562-a461-d814bd9a46b4"). InnerVolumeSpecName "kube-api-access-l6bhx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:28:09.054230 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:28:09.054192    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6bhx\" (UniqueName: \"kubernetes.io/projected/f449c93a-3f7d-4562-a461-d814bd9a46b4-kube-api-access-l6bhx\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:28:09.054230 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:28:09.054225    2551 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f449c93a-3f7d-4562-a461-d814bd9a46b4-extensions-socket-volume\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:28:09.405254 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:28:09.405218    2551 generic.go:356] "Generic (PLEG): container finished" podID="f449c93a-3f7d-4562-a461-d814bd9a46b4" containerID="cf36094c4ba79ee897a5611ab61ea4d14b417789a993a9ca4c2636ff0d97d7f3" exitCode=0
Jul 28 22:28:09.405429 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:28:09.405262    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p"
Jul 28 22:28:09.405429 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:28:09.405292    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p" event={"ID":"f449c93a-3f7d-4562-a461-d814bd9a46b4","Type":"ContainerDied","Data":"cf36094c4ba79ee897a5611ab61ea4d14b417789a993a9ca4c2636ff0d97d7f3"}
Jul 28 22:28:09.405429 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:28:09.405334    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p" event={"ID":"f449c93a-3f7d-4562-a461-d814bd9a46b4","Type":"ContainerDied","Data":"1106965438cd38f5c5797d36365f926e266a654ee7eb5f056d2bcef7f6fd71aa"}
Jul 28 22:28:09.405429 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:28:09.405362    2551 scope.go:117] "RemoveContainer" containerID="cf36094c4ba79ee897a5611ab61ea4d14b417789a993a9ca4c2636ff0d97d7f3"
Jul 28 22:28:09.414554 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:28:09.414537    2551 scope.go:117] "RemoveContainer" containerID="cf36094c4ba79ee897a5611ab61ea4d14b417789a993a9ca4c2636ff0d97d7f3"
Jul 28 22:28:09.414821 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:28:09.414800    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf36094c4ba79ee897a5611ab61ea4d14b417789a993a9ca4c2636ff0d97d7f3\": container with ID starting with cf36094c4ba79ee897a5611ab61ea4d14b417789a993a9ca4c2636ff0d97d7f3 not found: ID does not exist" containerID="cf36094c4ba79ee897a5611ab61ea4d14b417789a993a9ca4c2636ff0d97d7f3"
Jul 28 22:28:09.414863 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:28:09.414833    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf36094c4ba79ee897a5611ab61ea4d14b417789a993a9ca4c2636ff0d97d7f3"} err="failed to get container status \"cf36094c4ba79ee897a5611ab61ea4d14b417789a993a9ca4c2636ff0d97d7f3\": rpc error: code = NotFound desc = could not find container \"cf36094c4ba79ee897a5611ab61ea4d14b417789a993a9ca4c2636ff0d97d7f3\": container with ID starting with cf36094c4ba79ee897a5611ab61ea4d14b417789a993a9ca4c2636ff0d97d7f3 not found: ID does not exist"
Jul 28 22:28:09.425343 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:28:09.425303    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p"]
Jul 28 22:28:09.427232 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:28:09.427212    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-djw5p"]
Jul 28 22:28:10.461810 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:28:10.461775    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f449c93a-3f7d-4562-a461-d814bd9a46b4" path="/var/lib/kubelet/pods/f449c93a-3f7d-4562-a461-d814bd9a46b4/volumes"
Jul 28 22:29:07.396356 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:07.396320    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qxkqm"]
Jul 28 22:29:07.396789 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:07.396686    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f449c93a-3f7d-4562-a461-d814bd9a46b4" containerName="manager"
Jul 28 22:29:07.396789 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:07.396699    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="f449c93a-3f7d-4562-a461-d814bd9a46b4" containerName="manager"
Jul 28 22:29:07.396789 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:07.396744    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="f449c93a-3f7d-4562-a461-d814bd9a46b4" containerName="manager"
Jul 28 22:29:07.400654 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:07.400636    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qxkqm"
Jul 28 22:29:07.404812 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:07.404792    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-smk6t\""
Jul 28 22:29:07.412604 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:07.412581    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qxkqm"]
Jul 28 22:29:07.537863 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:07.537829    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/068f572e-5833-41e8-9bc7-0bdfaaecd84b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qxkqm\" (UID: \"068f572e-5833-41e8-9bc7-0bdfaaecd84b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qxkqm"
Jul 28 22:29:07.538045 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:07.537988    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6w8c\" (UniqueName: \"kubernetes.io/projected/068f572e-5833-41e8-9bc7-0bdfaaecd84b-kube-api-access-j6w8c\") pod \"kuadrant-operator-controller-manager-86878c769f-qxkqm\" (UID: \"068f572e-5833-41e8-9bc7-0bdfaaecd84b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qxkqm"
Jul 28 22:29:07.638632 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:07.638577    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6w8c\" (UniqueName: \"kubernetes.io/projected/068f572e-5833-41e8-9bc7-0bdfaaecd84b-kube-api-access-j6w8c\") pod \"kuadrant-operator-controller-manager-86878c769f-qxkqm\" (UID: \"068f572e-5833-41e8-9bc7-0bdfaaecd84b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qxkqm"
Jul 28 22:29:07.638810 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:07.638668    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/068f572e-5833-41e8-9bc7-0bdfaaecd84b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qxkqm\" (UID: \"068f572e-5833-41e8-9bc7-0bdfaaecd84b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qxkqm"
Jul 28 22:29:07.639027 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:07.639011    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/068f572e-5833-41e8-9bc7-0bdfaaecd84b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qxkqm\" (UID: \"068f572e-5833-41e8-9bc7-0bdfaaecd84b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qxkqm"
Jul 28 22:29:07.647289 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:07.647225    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6w8c\" (UniqueName: \"kubernetes.io/projected/068f572e-5833-41e8-9bc7-0bdfaaecd84b-kube-api-access-j6w8c\") pod \"kuadrant-operator-controller-manager-86878c769f-qxkqm\" (UID: \"068f572e-5833-41e8-9bc7-0bdfaaecd84b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qxkqm"
Jul 28 22:29:07.711029 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:07.710994    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qxkqm"
Jul 28 22:29:07.849238 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:07.849206    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qxkqm"]
Jul 28 22:29:07.852938 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:29:07.852907    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod068f572e_5833_41e8_9bc7_0bdfaaecd84b.slice/crio-434b19c0c7180f72224f51d6dbc735575777b9e2fe428917a1f40de804e3ee73 WatchSource:0}: Error finding container 434b19c0c7180f72224f51d6dbc735575777b9e2fe428917a1f40de804e3ee73: Status 404 returned error can't find the container with id 434b19c0c7180f72224f51d6dbc735575777b9e2fe428917a1f40de804e3ee73
Jul 28 22:29:08.681887 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:08.681850    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qxkqm" event={"ID":"068f572e-5833-41e8-9bc7-0bdfaaecd84b","Type":"ContainerStarted","Data":"c093ac6128ea01c89a1147569e06a11cf51a19ea9df0843b8127a545556429e4"}
Jul 28 22:29:08.682260 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:08.681927    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qxkqm" event={"ID":"068f572e-5833-41e8-9bc7-0bdfaaecd84b","Type":"ContainerStarted","Data":"434b19c0c7180f72224f51d6dbc735575777b9e2fe428917a1f40de804e3ee73"}
Jul 28 22:29:08.682260 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:08.681983    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qxkqm"
Jul 28 22:29:08.703846 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:08.703798    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qxkqm" podStartSLOduration=1.703783612 podStartE2EDuration="1.703783612s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:29:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:29:08.701519891 +0000 UTC m=+1156.719485579" watchObservedRunningTime="2026-07-28 22:29:08.703783612 +0000 UTC m=+1156.721749338"
Jul 28 22:29:19.690082 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:19.690041    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qxkqm"
Jul 28 22:29:29.272718 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:29.272680    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:29:40.173807 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:40.173770    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:29:54.955699 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:54.955671    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:29:54.957981 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:54.957440    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:29:55.274210 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:29:55.274130    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:30:07.564469 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:30:07.564431    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:30:17.089501 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:30:17.089468    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:30:25.975610 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:30:25.975574    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:30:37.267318 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:30:37.267284    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:30:47.768246 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:30:47.768154    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:30:56.574537 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:30:56.574497    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:31:07.763110 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:31:07.763072    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:31:19.668647 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:31:19.668610    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:31:27.275100 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:31:27.275065    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:31:44.474393 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:31:44.474361    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:31:53.676148 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:31:53.676108    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:32:07.873205 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:32:07.873167    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:32:19.867025 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:32:19.866976    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:32:35.372360 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:32:35.372317    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:32:49.763845 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:32:49.763807    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:32:56.477726 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:32:56.477687    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:33:12.567075 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:33:12.567037    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:33:22.278980 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:33:22.278937    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:33:31.666652 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:33:31.666615    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:33:39.674428 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:33:39.674388    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:33:49.375042 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:33:49.374962    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:33:59.067785 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:33:59.067747    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:34:11.674147 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:34:11.674106    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:34:20.872160 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:34:20.872124    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:34:30.465339 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:34:30.465304    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:34:42.074638 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:34:42.074594    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:34:54.981802 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:34:54.981702    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:34:54.985372 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:34:54.983293    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:34:55.067503 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:34:55.067469    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:35:03.274830 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:35:03.274792    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:35:14.469400 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:35:14.469312    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:35:25.977676 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:35:25.977632    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:35:38.468794 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:35:38.468762    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:35:52.971265 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:35:52.971227    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:36:05.983764 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:36:05.983724    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:36:19.565352 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:36:19.565283    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:36:48.569960 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:36:48.569871    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:36:49.102420 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:36:49.102381    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7"]
Jul 28 22:36:49.105841 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:36:49.105825    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7"
Jul 28 22:36:49.108944 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:36:49.108926    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-6ad9588a-dockercfg-7hgvr\""
Jul 28 22:36:49.109025 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:36:49.108950    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-6ad9588a\""
Jul 28 22:36:49.117326 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:36:49.117302    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7"]
Jul 28 22:36:49.227598 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:36:49.227566    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j9m9\" (UniqueName: \"kubernetes.io/projected/c0e07398-034c-4bd4-9b7b-08f9a3181fad-kube-api-access-5j9m9\") pod \"payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7\" (UID: \"c0e07398-034c-4bd4-9b7b-08f9a3181fad\") " pod="openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7"
Jul 28 22:36:49.227598 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:36:49.227607    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0e07398-034c-4bd4-9b7b-08f9a3181fad-plugins-config-volume\") pod \"payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7\" (UID: \"c0e07398-034c-4bd4-9b7b-08f9a3181fad\") " pod="openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7"
Jul 28 22:36:49.328953 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:36:49.328918    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0e07398-034c-4bd4-9b7b-08f9a3181fad-plugins-config-volume\") pod \"payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7\" (UID: \"c0e07398-034c-4bd4-9b7b-08f9a3181fad\") " pod="openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7"
Jul 28 22:36:49.329116 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:36:49.329013    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j9m9\" (UniqueName: \"kubernetes.io/projected/c0e07398-034c-4bd4-9b7b-08f9a3181fad-kube-api-access-5j9m9\") pod \"payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7\" (UID: \"c0e07398-034c-4bd4-9b7b-08f9a3181fad\") " pod="openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7"
Jul 28 22:36:49.329696 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:36:49.329678    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0e07398-034c-4bd4-9b7b-08f9a3181fad-plugins-config-volume\") pod \"payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7\" (UID: \"c0e07398-034c-4bd4-9b7b-08f9a3181fad\") " pod="openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7"
Jul 28 22:36:49.336187 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:36:49.336159    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j9m9\" (UniqueName: \"kubernetes.io/projected/c0e07398-034c-4bd4-9b7b-08f9a3181fad-kube-api-access-5j9m9\") pod \"payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7\" (UID: \"c0e07398-034c-4bd4-9b7b-08f9a3181fad\") " pod="openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7"
Jul 28 22:36:49.418136 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:36:49.418039    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7"
Jul 28 22:36:49.549869 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:36:49.549839    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7"]
Jul 28 22:36:49.550833 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:36:49.550795    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0e07398_034c_4bd4_9b7b_08f9a3181fad.slice/crio-e0486b817db4b818de8e0313cf7398ae4af72d86a9c152b2fc261ba70c07ea8e WatchSource:0}: Error finding container e0486b817db4b818de8e0313cf7398ae4af72d86a9c152b2fc261ba70c07ea8e: Status 404 returned error can't find the container with id e0486b817db4b818de8e0313cf7398ae4af72d86a9c152b2fc261ba70c07ea8e
Jul 28 22:36:49.552789 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:36:49.552773    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 22:36:49.775721 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:36:49.775634    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7" event={"ID":"c0e07398-034c-4bd4-9b7b-08f9a3181fad","Type":"ContainerStarted","Data":"e0486b817db4b818de8e0313cf7398ae4af72d86a9c152b2fc261ba70c07ea8e"}
Jul 28 22:36:52.791943 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:36:52.791902    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7" event={"ID":"c0e07398-034c-4bd4-9b7b-08f9a3181fad","Type":"ContainerStarted","Data":"5b8d8a1fe013439e84fc3b7004bb151f771f443ef8ef54cfb5ecd0a7537243ef"}
Jul 28 22:36:52.792304 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:36:52.791983    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7"
Jul 28 22:36:52.807563 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:36:52.807502    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7" podStartSLOduration=1.451142048 podStartE2EDuration="3.807487197s" totalImagesPullingTime="2.356345149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:36:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:36:52.806965384 +0000 UTC m=+1620.824931081" watchObservedRunningTime="2026-07-28 22:36:52.807487197 +0000 UTC m=+1620.825452890"
Jul 28 22:36:59.773742 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:36:59.773702    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:37:03.799673 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:03.799646    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7"
Jul 28 22:37:28.910742 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:28.910703    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7"]
Jul 28 22:37:28.911323 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:28.911292    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7" podUID="c0e07398-034c-4bd4-9b7b-08f9a3181fad" containerName="payload-processing" containerID="cri-o://5b8d8a1fe013439e84fc3b7004bb151f771f443ef8ef54cfb5ecd0a7537243ef" gracePeriod=30
Jul 28 22:37:29.170920 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:29.170857    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7"
Jul 28 22:37:29.250515 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:29.250486    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5j9m9\" (UniqueName: \"kubernetes.io/projected/c0e07398-034c-4bd4-9b7b-08f9a3181fad-kube-api-access-5j9m9\") pod \"c0e07398-034c-4bd4-9b7b-08f9a3181fad\" (UID: \"c0e07398-034c-4bd4-9b7b-08f9a3181fad\") "
Jul 28 22:37:29.250719 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:29.250558    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0e07398-034c-4bd4-9b7b-08f9a3181fad-plugins-config-volume\") pod \"c0e07398-034c-4bd4-9b7b-08f9a3181fad\" (UID: \"c0e07398-034c-4bd4-9b7b-08f9a3181fad\") "
Jul 28 22:37:29.250917 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:29.250891    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0e07398-034c-4bd4-9b7b-08f9a3181fad-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c0e07398-034c-4bd4-9b7b-08f9a3181fad" (UID: "c0e07398-034c-4bd4-9b7b-08f9a3181fad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:37:29.252492 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:29.252461    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0e07398-034c-4bd4-9b7b-08f9a3181fad-kube-api-access-5j9m9" (OuterVolumeSpecName: "kube-api-access-5j9m9") pod "c0e07398-034c-4bd4-9b7b-08f9a3181fad" (UID: "c0e07398-034c-4bd4-9b7b-08f9a3181fad"). InnerVolumeSpecName "kube-api-access-5j9m9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:37:29.351976 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:29.351938    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5j9m9\" (UniqueName: \"kubernetes.io/projected/c0e07398-034c-4bd4-9b7b-08f9a3181fad-kube-api-access-5j9m9\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:37:29.351976 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:29.351969    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c0e07398-034c-4bd4-9b7b-08f9a3181fad-plugins-config-volume\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:37:29.966791 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:29.966754    2551 generic.go:356] "Generic (PLEG): container finished" podID="c0e07398-034c-4bd4-9b7b-08f9a3181fad" containerID="5b8d8a1fe013439e84fc3b7004bb151f771f443ef8ef54cfb5ecd0a7537243ef" exitCode=0
Jul 28 22:37:29.967303 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:29.966809    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7"
Jul 28 22:37:29.967303 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:29.966853    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7" event={"ID":"c0e07398-034c-4bd4-9b7b-08f9a3181fad","Type":"ContainerDied","Data":"5b8d8a1fe013439e84fc3b7004bb151f771f443ef8ef54cfb5ecd0a7537243ef"}
Jul 28 22:37:29.967303 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:29.966901    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7" event={"ID":"c0e07398-034c-4bd4-9b7b-08f9a3181fad","Type":"ContainerDied","Data":"e0486b817db4b818de8e0313cf7398ae4af72d86a9c152b2fc261ba70c07ea8e"}
Jul 28 22:37:29.967303 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:29.966927    2551 scope.go:117] "RemoveContainer" containerID="5b8d8a1fe013439e84fc3b7004bb151f771f443ef8ef54cfb5ecd0a7537243ef"
Jul 28 22:37:29.978001 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:29.977947    2551 scope.go:117] "RemoveContainer" containerID="5b8d8a1fe013439e84fc3b7004bb151f771f443ef8ef54cfb5ecd0a7537243ef"
Jul 28 22:37:29.978295 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:37:29.978258    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b8d8a1fe013439e84fc3b7004bb151f771f443ef8ef54cfb5ecd0a7537243ef\": container with ID starting with 5b8d8a1fe013439e84fc3b7004bb151f771f443ef8ef54cfb5ecd0a7537243ef not found: ID does not exist" containerID="5b8d8a1fe013439e84fc3b7004bb151f771f443ef8ef54cfb5ecd0a7537243ef"
Jul 28 22:37:29.978370 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:29.978306    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b8d8a1fe013439e84fc3b7004bb151f771f443ef8ef54cfb5ecd0a7537243ef"} err="failed to get container status \"5b8d8a1fe013439e84fc3b7004bb151f771f443ef8ef54cfb5ecd0a7537243ef\": rpc error: code = NotFound desc = could not find container \"5b8d8a1fe013439e84fc3b7004bb151f771f443ef8ef54cfb5ecd0a7537243ef\": container with ID starting with 5b8d8a1fe013439e84fc3b7004bb151f771f443ef8ef54cfb5ecd0a7537243ef not found: ID does not exist"
Jul 28 22:37:29.997876 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:29.997844    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7"]
Jul 28 22:37:30.006488 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:30.006459    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6ad9588a-859669b9c7-56xr7"]
Jul 28 22:37:30.462493 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:30.462460    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0e07398-034c-4bd4-9b7b-08f9a3181fad" path="/var/lib/kubelet/pods/c0e07398-034c-4bd4-9b7b-08f9a3181fad/volumes"
Jul 28 22:37:33.025180 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.025135    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86"]
Jul 28 22:37:33.025575 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.025560    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0e07398-034c-4bd4-9b7b-08f9a3181fad" containerName="payload-processing"
Jul 28 22:37:33.025575 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.025576    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0e07398-034c-4bd4-9b7b-08f9a3181fad" containerName="payload-processing"
Jul 28 22:37:33.025660 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.025646    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0e07398-034c-4bd4-9b7b-08f9a3181fad" containerName="payload-processing"
Jul 28 22:37:33.029939 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.029921    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86"
Jul 28 22:37:33.032370 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.032339    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-ae3227ae\""
Jul 28 22:37:33.032370 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.032357    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-ae3227ae-dockercfg-rlklf\""
Jul 28 22:37:33.039214 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.039189    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86"]
Jul 28 22:37:33.069466 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.069430    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv"]
Jul 28 22:37:33.072696 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.072680    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv"
Jul 28 22:37:33.083325 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.083300    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv"]
Jul 28 22:37:33.085506 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.085479    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5v7s\" (UniqueName: \"kubernetes.io/projected/e3f215be-7ca9-4e89-b9a9-32e5b5184136-kube-api-access-g5v7s\") pod \"payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86\" (UID: \"e3f215be-7ca9-4e89-b9a9-32e5b5184136\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86"
Jul 28 22:37:33.085617 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.085519    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv79w\" (UniqueName: \"kubernetes.io/projected/189439b9-d13a-44a4-886e-9fde49a1dda8-kube-api-access-jv79w\") pod \"payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv\" (UID: \"189439b9-d13a-44a4-886e-9fde49a1dda8\") " pod="openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv"
Jul 28 22:37:33.085617 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.085603    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3f215be-7ca9-4e89-b9a9-32e5b5184136-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86\" (UID: \"e3f215be-7ca9-4e89-b9a9-32e5b5184136\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86"
Jul 28 22:37:33.085694 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.085678    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/189439b9-d13a-44a4-886e-9fde49a1dda8-plugins-config-volume\") pod \"payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv\" (UID: \"189439b9-d13a-44a4-886e-9fde49a1dda8\") " pod="openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv"
Jul 28 22:37:33.187106 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.187072    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5v7s\" (UniqueName: \"kubernetes.io/projected/e3f215be-7ca9-4e89-b9a9-32e5b5184136-kube-api-access-g5v7s\") pod \"payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86\" (UID: \"e3f215be-7ca9-4e89-b9a9-32e5b5184136\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86"
Jul 28 22:37:33.187106 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.187106    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv79w\" (UniqueName: \"kubernetes.io/projected/189439b9-d13a-44a4-886e-9fde49a1dda8-kube-api-access-jv79w\") pod \"payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv\" (UID: \"189439b9-d13a-44a4-886e-9fde49a1dda8\") " pod="openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv"
Jul 28 22:37:33.187334 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.187132    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3f215be-7ca9-4e89-b9a9-32e5b5184136-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86\" (UID: \"e3f215be-7ca9-4e89-b9a9-32e5b5184136\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86"
Jul 28 22:37:33.187334 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.187182    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/189439b9-d13a-44a4-886e-9fde49a1dda8-plugins-config-volume\") pod \"payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv\" (UID: \"189439b9-d13a-44a4-886e-9fde49a1dda8\") " pod="openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv"
Jul 28 22:37:33.187931 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.187906    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/189439b9-d13a-44a4-886e-9fde49a1dda8-plugins-config-volume\") pod \"payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv\" (UID: \"189439b9-d13a-44a4-886e-9fde49a1dda8\") " pod="openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv"
Jul 28 22:37:33.188031 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.187943    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3f215be-7ca9-4e89-b9a9-32e5b5184136-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86\" (UID: \"e3f215be-7ca9-4e89-b9a9-32e5b5184136\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86"
Jul 28 22:37:33.195677 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.195650    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5v7s\" (UniqueName: \"kubernetes.io/projected/e3f215be-7ca9-4e89-b9a9-32e5b5184136-kube-api-access-g5v7s\") pod \"payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86\" (UID: \"e3f215be-7ca9-4e89-b9a9-32e5b5184136\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86"
Jul 28 22:37:33.195767 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.195652    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv79w\" (UniqueName: \"kubernetes.io/projected/189439b9-d13a-44a4-886e-9fde49a1dda8-kube-api-access-jv79w\") pod \"payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv\" (UID: \"189439b9-d13a-44a4-886e-9fde49a1dda8\") " pod="openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv"
Jul 28 22:37:33.212334 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.212308    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:37:33.340968 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.340936    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86"
Jul 28 22:37:33.382863 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.382831    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv"
Jul 28 22:37:33.501939 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.501903    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86"]
Jul 28 22:37:33.504569 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:37:33.504520    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3f215be_7ca9_4e89_b9a9_32e5b5184136.slice/crio-3aab9f2138f600527200ef9c48d0bb9a8c212da614223483b6eae7c81b15a85d WatchSource:0}: Error finding container 3aab9f2138f600527200ef9c48d0bb9a8c212da614223483b6eae7c81b15a85d: Status 404 returned error can't find the container with id 3aab9f2138f600527200ef9c48d0bb9a8c212da614223483b6eae7c81b15a85d
Jul 28 22:37:33.736810 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.736784    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv"]
Jul 28 22:37:33.739047 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:37:33.739016    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod189439b9_d13a_44a4_886e_9fde49a1dda8.slice/crio-6715b6ccdcd8c9fa8630689dd29f2ad7bee2d993f916d5dfa8cbd702332fae0e WatchSource:0}: Error finding container 6715b6ccdcd8c9fa8630689dd29f2ad7bee2d993f916d5dfa8cbd702332fae0e: Status 404 returned error can't find the container with id 6715b6ccdcd8c9fa8630689dd29f2ad7bee2d993f916d5dfa8cbd702332fae0e
Jul 28 22:37:33.987247 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.987145    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv" event={"ID":"189439b9-d13a-44a4-886e-9fde49a1dda8","Type":"ContainerStarted","Data":"3a59dd85989bde0685f924d940e6723ccb0aa53b29da0fe64320ada14f285c39"}
Jul 28 22:37:33.987247 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.987207    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv" event={"ID":"189439b9-d13a-44a4-886e-9fde49a1dda8","Type":"ContainerStarted","Data":"6715b6ccdcd8c9fa8630689dd29f2ad7bee2d993f916d5dfa8cbd702332fae0e"}
Jul 28 22:37:33.987472 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.987248    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv"
Jul 28 22:37:33.988770 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.988740    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86" event={"ID":"e3f215be-7ca9-4e89-b9a9-32e5b5184136","Type":"ContainerStarted","Data":"82203b638f1fdc9afe1deab2bd0f5213f715328f5b66b0e86a6773b20c157b06"}
Jul 28 22:37:33.988911 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.988783    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86" event={"ID":"e3f215be-7ca9-4e89-b9a9-32e5b5184136","Type":"ContainerStarted","Data":"3aab9f2138f600527200ef9c48d0bb9a8c212da614223483b6eae7c81b15a85d"}
Jul 28 22:37:33.988911 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:33.988822    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86"
Jul 28 22:37:34.004400 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:34.004319    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv" podStartSLOduration=1.004302957 podStartE2EDuration="1.004302957s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:37:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:37:34.004194608 +0000 UTC m=+1662.022160303" watchObservedRunningTime="2026-07-28 22:37:34.004302957 +0000 UTC m=+1662.022268652"
Jul 28 22:37:34.023549 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:34.023475    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86" podStartSLOduration=1.023455883 podStartE2EDuration="1.023455883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:37:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:37:34.023025325 +0000 UTC m=+1662.040991023" watchObservedRunningTime="2026-07-28 22:37:34.023455883 +0000 UTC m=+1662.041421578"
Jul 28 22:37:44.165483 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:44.165441    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:37:44.995049 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:44.995012    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86"
Jul 28 22:37:44.995245 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:37:44.995092    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv"
Jul 28 22:38:03.042326 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:03.042288    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv"]
Jul 28 22:38:03.042851 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:03.042564    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv" podUID="189439b9-d13a-44a4-886e-9fde49a1dda8" containerName="payload-processing" containerID="cri-o://3a59dd85989bde0685f924d940e6723ccb0aa53b29da0fe64320ada14f285c39" gracePeriod=30
Jul 28 22:38:03.050145 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:03.050114    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86"]
Jul 28 22:38:03.051280 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:03.051222    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86" podUID="e3f215be-7ca9-4e89-b9a9-32e5b5184136" containerName="payload-pre-processing" containerID="cri-o://82203b638f1fdc9afe1deab2bd0f5213f715328f5b66b0e86a6773b20c157b06" gracePeriod=30
Jul 28 22:38:03.334515 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:03.334488    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv"
Jul 28 22:38:03.337208 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:03.337189    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86"
Jul 28 22:38:03.454328 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:03.454285    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jv79w\" (UniqueName: \"kubernetes.io/projected/189439b9-d13a-44a4-886e-9fde49a1dda8-kube-api-access-jv79w\") pod \"189439b9-d13a-44a4-886e-9fde49a1dda8\" (UID: \"189439b9-d13a-44a4-886e-9fde49a1dda8\") "
Jul 28 22:38:03.454505 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:03.454349    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/189439b9-d13a-44a4-886e-9fde49a1dda8-plugins-config-volume\") pod \"189439b9-d13a-44a4-886e-9fde49a1dda8\" (UID: \"189439b9-d13a-44a4-886e-9fde49a1dda8\") "
Jul 28 22:38:03.454505 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:03.454382    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3f215be-7ca9-4e89-b9a9-32e5b5184136-plugins-config-volume\") pod \"e3f215be-7ca9-4e89-b9a9-32e5b5184136\" (UID: \"e3f215be-7ca9-4e89-b9a9-32e5b5184136\") "
Jul 28 22:38:03.454505 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:03.454399    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5v7s\" (UniqueName: \"kubernetes.io/projected/e3f215be-7ca9-4e89-b9a9-32e5b5184136-kube-api-access-g5v7s\") pod \"e3f215be-7ca9-4e89-b9a9-32e5b5184136\" (UID: \"e3f215be-7ca9-4e89-b9a9-32e5b5184136\") "
Jul 28 22:38:03.454792 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:03.454769    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/189439b9-d13a-44a4-886e-9fde49a1dda8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "189439b9-d13a-44a4-886e-9fde49a1dda8" (UID: "189439b9-d13a-44a4-886e-9fde49a1dda8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:38:03.454828 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:03.454808    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3f215be-7ca9-4e89-b9a9-32e5b5184136-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e3f215be-7ca9-4e89-b9a9-32e5b5184136" (UID: "e3f215be-7ca9-4e89-b9a9-32e5b5184136"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:38:03.456457 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:03.456423    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/189439b9-d13a-44a4-886e-9fde49a1dda8-kube-api-access-jv79w" (OuterVolumeSpecName: "kube-api-access-jv79w") pod "189439b9-d13a-44a4-886e-9fde49a1dda8" (UID: "189439b9-d13a-44a4-886e-9fde49a1dda8"). InnerVolumeSpecName "kube-api-access-jv79w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:38:03.456642 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:03.456460    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3f215be-7ca9-4e89-b9a9-32e5b5184136-kube-api-access-g5v7s" (OuterVolumeSpecName: "kube-api-access-g5v7s") pod "e3f215be-7ca9-4e89-b9a9-32e5b5184136" (UID: "e3f215be-7ca9-4e89-b9a9-32e5b5184136"). InnerVolumeSpecName "kube-api-access-g5v7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:38:03.555637 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:03.555591    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/189439b9-d13a-44a4-886e-9fde49a1dda8-plugins-config-volume\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:38:03.555637 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:03.555630    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e3f215be-7ca9-4e89-b9a9-32e5b5184136-plugins-config-volume\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:38:03.555836 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:03.555647    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5v7s\" (UniqueName: \"kubernetes.io/projected/e3f215be-7ca9-4e89-b9a9-32e5b5184136-kube-api-access-g5v7s\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:38:03.555836 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:03.555662    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jv79w\" (UniqueName: \"kubernetes.io/projected/189439b9-d13a-44a4-886e-9fde49a1dda8-kube-api-access-jv79w\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:38:04.141733 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:04.141700    2551 generic.go:356] "Generic (PLEG): container finished" podID="189439b9-d13a-44a4-886e-9fde49a1dda8" containerID="3a59dd85989bde0685f924d940e6723ccb0aa53b29da0fe64320ada14f285c39" exitCode=0
Jul 28 22:38:04.142180 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:04.141744    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv"
Jul 28 22:38:04.142180 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:04.141780    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv" event={"ID":"189439b9-d13a-44a4-886e-9fde49a1dda8","Type":"ContainerDied","Data":"3a59dd85989bde0685f924d940e6723ccb0aa53b29da0fe64320ada14f285c39"}
Jul 28 22:38:04.142180 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:04.141837    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv" event={"ID":"189439b9-d13a-44a4-886e-9fde49a1dda8","Type":"ContainerDied","Data":"6715b6ccdcd8c9fa8630689dd29f2ad7bee2d993f916d5dfa8cbd702332fae0e"}
Jul 28 22:38:04.142180 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:04.141865    2551 scope.go:117] "RemoveContainer" containerID="3a59dd85989bde0685f924d940e6723ccb0aa53b29da0fe64320ada14f285c39"
Jul 28 22:38:04.143200 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:04.143109    2551 generic.go:356] "Generic (PLEG): container finished" podID="e3f215be-7ca9-4e89-b9a9-32e5b5184136" containerID="82203b638f1fdc9afe1deab2bd0f5213f715328f5b66b0e86a6773b20c157b06" exitCode=0
Jul 28 22:38:04.143200 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:04.143144    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86" event={"ID":"e3f215be-7ca9-4e89-b9a9-32e5b5184136","Type":"ContainerDied","Data":"82203b638f1fdc9afe1deab2bd0f5213f715328f5b66b0e86a6773b20c157b06"}
Jul 28 22:38:04.143200 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:04.143178    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86"
Jul 28 22:38:04.143333 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:04.143197    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86" event={"ID":"e3f215be-7ca9-4e89-b9a9-32e5b5184136","Type":"ContainerDied","Data":"3aab9f2138f600527200ef9c48d0bb9a8c212da614223483b6eae7c81b15a85d"}
Jul 28 22:38:04.151671 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:04.151647    2551 scope.go:117] "RemoveContainer" containerID="3a59dd85989bde0685f924d940e6723ccb0aa53b29da0fe64320ada14f285c39"
Jul 28 22:38:04.151956 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:38:04.151937    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a59dd85989bde0685f924d940e6723ccb0aa53b29da0fe64320ada14f285c39\": container with ID starting with 3a59dd85989bde0685f924d940e6723ccb0aa53b29da0fe64320ada14f285c39 not found: ID does not exist" containerID="3a59dd85989bde0685f924d940e6723ccb0aa53b29da0fe64320ada14f285c39"
Jul 28 22:38:04.152023 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:04.151969    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a59dd85989bde0685f924d940e6723ccb0aa53b29da0fe64320ada14f285c39"} err="failed to get container status \"3a59dd85989bde0685f924d940e6723ccb0aa53b29da0fe64320ada14f285c39\": rpc error: code = NotFound desc = could not find container \"3a59dd85989bde0685f924d940e6723ccb0aa53b29da0fe64320ada14f285c39\": container with ID starting with 3a59dd85989bde0685f924d940e6723ccb0aa53b29da0fe64320ada14f285c39 not found: ID does not exist"
Jul 28 22:38:04.152023 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:04.151997    2551 scope.go:117] "RemoveContainer" containerID="82203b638f1fdc9afe1deab2bd0f5213f715328f5b66b0e86a6773b20c157b06"
Jul 28 22:38:04.159940 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:04.159920    2551 scope.go:117] "RemoveContainer" containerID="82203b638f1fdc9afe1deab2bd0f5213f715328f5b66b0e86a6773b20c157b06"
Jul 28 22:38:04.160243 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:38:04.160221    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82203b638f1fdc9afe1deab2bd0f5213f715328f5b66b0e86a6773b20c157b06\": container with ID starting with 82203b638f1fdc9afe1deab2bd0f5213f715328f5b66b0e86a6773b20c157b06 not found: ID does not exist" containerID="82203b638f1fdc9afe1deab2bd0f5213f715328f5b66b0e86a6773b20c157b06"
Jul 28 22:38:04.160312 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:04.160253    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82203b638f1fdc9afe1deab2bd0f5213f715328f5b66b0e86a6773b20c157b06"} err="failed to get container status \"82203b638f1fdc9afe1deab2bd0f5213f715328f5b66b0e86a6773b20c157b06\": rpc error: code = NotFound desc = could not find container \"82203b638f1fdc9afe1deab2bd0f5213f715328f5b66b0e86a6773b20c157b06\": container with ID starting with 82203b638f1fdc9afe1deab2bd0f5213f715328f5b66b0e86a6773b20c157b06 not found: ID does not exist"
Jul 28 22:38:04.162961 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:04.162939    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv"]
Jul 28 22:38:04.166642 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:04.166621    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-ae3227ae-57f57759b-hqkwv"]
Jul 28 22:38:04.179190 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:04.179161    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86"]
Jul 28 22:38:04.180861 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:04.180839    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-ae3227ae-c5bc59db8-qqn86"]
Jul 28 22:38:04.463124 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:04.463093    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="189439b9-d13a-44a4-886e-9fde49a1dda8" path="/var/lib/kubelet/pods/189439b9-d13a-44a4-886e-9fde49a1dda8/volumes"
Jul 28 22:38:04.463432 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:04.463419    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3f215be-7ca9-4e89-b9a9-32e5b5184136" path="/var/lib/kubelet/pods/e3f215be-7ca9-4e89-b9a9-32e5b5184136/volumes"
Jul 28 22:38:07.164050 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:07.164017    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:38:08.156959 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:08.156925    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl"]
Jul 28 22:38:08.157274 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:08.157262    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3f215be-7ca9-4e89-b9a9-32e5b5184136" containerName="payload-pre-processing"
Jul 28 22:38:08.157314 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:08.157276    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3f215be-7ca9-4e89-b9a9-32e5b5184136" containerName="payload-pre-processing"
Jul 28 22:38:08.157314 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:08.157283    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="189439b9-d13a-44a4-886e-9fde49a1dda8" containerName="payload-processing"
Jul 28 22:38:08.157314 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:08.157289    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="189439b9-d13a-44a4-886e-9fde49a1dda8" containerName="payload-processing"
Jul 28 22:38:08.157425 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:08.157343    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3f215be-7ca9-4e89-b9a9-32e5b5184136" containerName="payload-pre-processing"
Jul 28 22:38:08.157425 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:08.157354    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="189439b9-d13a-44a4-886e-9fde49a1dda8" containerName="payload-processing"
Jul 28 22:38:08.161717 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:08.161694    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl"
Jul 28 22:38:08.163888 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:08.163869    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-b9e632fc\""
Jul 28 22:38:08.163972 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:08.163899    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-b9e632fc-dockercfg-mzqmg\""
Jul 28 22:38:08.168496 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:08.168470    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl"]
Jul 28 22:38:08.298644 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:08.298609    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/989b3c31-2558-41c2-a5b6-c7b4114c3ed8-plugins-config-volume\") pod \"payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl\" (UID: \"989b3c31-2558-41c2-a5b6-c7b4114c3ed8\") " pod="openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl"
Jul 28 22:38:08.298808 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:08.298687    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lsc5\" (UniqueName: \"kubernetes.io/projected/989b3c31-2558-41c2-a5b6-c7b4114c3ed8-kube-api-access-5lsc5\") pod \"payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl\" (UID: \"989b3c31-2558-41c2-a5b6-c7b4114c3ed8\") " pod="openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl"
Jul 28 22:38:08.399916 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:08.399882    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/989b3c31-2558-41c2-a5b6-c7b4114c3ed8-plugins-config-volume\") pod \"payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl\" (UID: \"989b3c31-2558-41c2-a5b6-c7b4114c3ed8\") " pod="openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl"
Jul 28 22:38:08.400094 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:08.399943    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lsc5\" (UniqueName: \"kubernetes.io/projected/989b3c31-2558-41c2-a5b6-c7b4114c3ed8-kube-api-access-5lsc5\") pod \"payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl\" (UID: \"989b3c31-2558-41c2-a5b6-c7b4114c3ed8\") " pod="openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl"
Jul 28 22:38:08.400629 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:08.400609    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/989b3c31-2558-41c2-a5b6-c7b4114c3ed8-plugins-config-volume\") pod \"payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl\" (UID: \"989b3c31-2558-41c2-a5b6-c7b4114c3ed8\") " pod="openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl"
Jul 28 22:38:08.407928 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:08.407872    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lsc5\" (UniqueName: \"kubernetes.io/projected/989b3c31-2558-41c2-a5b6-c7b4114c3ed8-kube-api-access-5lsc5\") pod \"payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl\" (UID: \"989b3c31-2558-41c2-a5b6-c7b4114c3ed8\") " pod="openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl"
Jul 28 22:38:08.473953 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:08.473914    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl"
Jul 28 22:38:08.608242 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:08.608216    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl"]
Jul 28 22:38:08.610154 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:38:08.610128    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod989b3c31_2558_41c2_a5b6_c7b4114c3ed8.slice/crio-d317b4abca340ae7a8f248da2112a9288f298f39dec9e7255b77c23cd4a896ea WatchSource:0}: Error finding container d317b4abca340ae7a8f248da2112a9288f298f39dec9e7255b77c23cd4a896ea: Status 404 returned error can't find the container with id d317b4abca340ae7a8f248da2112a9288f298f39dec9e7255b77c23cd4a896ea
Jul 28 22:38:09.170145 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:09.170112    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl" event={"ID":"989b3c31-2558-41c2-a5b6-c7b4114c3ed8","Type":"ContainerStarted","Data":"a9110f19990b2cd7d50b09ff4bc9385ae4453423e218398a5c8762f9503c4e48"}
Jul 28 22:38:09.170556 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:09.170171    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl" event={"ID":"989b3c31-2558-41c2-a5b6-c7b4114c3ed8","Type":"ContainerStarted","Data":"d317b4abca340ae7a8f248da2112a9288f298f39dec9e7255b77c23cd4a896ea"}
Jul 28 22:38:09.170556 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:09.170210    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl"
Jul 28 22:38:09.184921 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:09.184872    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl" podStartSLOduration=1.184856033 podStartE2EDuration="1.184856033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:38:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:38:09.184048592 +0000 UTC m=+1697.202014286" watchObservedRunningTime="2026-07-28 22:38:09.184856033 +0000 UTC m=+1697.202821726"
Jul 28 22:38:17.268516 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:17.268438    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:38:20.176728 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:20.176693    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl"
Jul 28 22:38:37.497743 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:37.497704    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl"]
Jul 28 22:38:37.498742 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:37.497984    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl" podUID="989b3c31-2558-41c2-a5b6-c7b4114c3ed8" containerName="payload-processing" containerID="cri-o://a9110f19990b2cd7d50b09ff4bc9385ae4453423e218398a5c8762f9503c4e48" gracePeriod=30
Jul 28 22:38:37.746484 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:37.746459    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl"
Jul 28 22:38:37.832921 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:37.832892    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lsc5\" (UniqueName: \"kubernetes.io/projected/989b3c31-2558-41c2-a5b6-c7b4114c3ed8-kube-api-access-5lsc5\") pod \"989b3c31-2558-41c2-a5b6-c7b4114c3ed8\" (UID: \"989b3c31-2558-41c2-a5b6-c7b4114c3ed8\") "
Jul 28 22:38:37.833097 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:37.832967    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/989b3c31-2558-41c2-a5b6-c7b4114c3ed8-plugins-config-volume\") pod \"989b3c31-2558-41c2-a5b6-c7b4114c3ed8\" (UID: \"989b3c31-2558-41c2-a5b6-c7b4114c3ed8\") "
Jul 28 22:38:37.833359 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:37.833327    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/989b3c31-2558-41c2-a5b6-c7b4114c3ed8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "989b3c31-2558-41c2-a5b6-c7b4114c3ed8" (UID: "989b3c31-2558-41c2-a5b6-c7b4114c3ed8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:38:37.834999 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:37.834975    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/989b3c31-2558-41c2-a5b6-c7b4114c3ed8-kube-api-access-5lsc5" (OuterVolumeSpecName: "kube-api-access-5lsc5") pod "989b3c31-2558-41c2-a5b6-c7b4114c3ed8" (UID: "989b3c31-2558-41c2-a5b6-c7b4114c3ed8"). InnerVolumeSpecName "kube-api-access-5lsc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:38:37.933859 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:37.933820    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/989b3c31-2558-41c2-a5b6-c7b4114c3ed8-plugins-config-volume\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:38:37.933859 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:37.933854    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lsc5\" (UniqueName: \"kubernetes.io/projected/989b3c31-2558-41c2-a5b6-c7b4114c3ed8-kube-api-access-5lsc5\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:38:38.303852 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:38.303771    2551 generic.go:356] "Generic (PLEG): container finished" podID="989b3c31-2558-41c2-a5b6-c7b4114c3ed8" containerID="a9110f19990b2cd7d50b09ff4bc9385ae4453423e218398a5c8762f9503c4e48" exitCode=0
Jul 28 22:38:38.303852 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:38.303817    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl"
Jul 28 22:38:38.304031 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:38.303859    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl" event={"ID":"989b3c31-2558-41c2-a5b6-c7b4114c3ed8","Type":"ContainerDied","Data":"a9110f19990b2cd7d50b09ff4bc9385ae4453423e218398a5c8762f9503c4e48"}
Jul 28 22:38:38.304031 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:38.303914    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl" event={"ID":"989b3c31-2558-41c2-a5b6-c7b4114c3ed8","Type":"ContainerDied","Data":"d317b4abca340ae7a8f248da2112a9288f298f39dec9e7255b77c23cd4a896ea"}
Jul 28 22:38:38.304031 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:38.303937    2551 scope.go:117] "RemoveContainer" containerID="a9110f19990b2cd7d50b09ff4bc9385ae4453423e218398a5c8762f9503c4e48"
Jul 28 22:38:38.313142 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:38.313121    2551 scope.go:117] "RemoveContainer" containerID="a9110f19990b2cd7d50b09ff4bc9385ae4453423e218398a5c8762f9503c4e48"
Jul 28 22:38:38.313488 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:38:38.313465    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9110f19990b2cd7d50b09ff4bc9385ae4453423e218398a5c8762f9503c4e48\": container with ID starting with a9110f19990b2cd7d50b09ff4bc9385ae4453423e218398a5c8762f9503c4e48 not found: ID does not exist" containerID="a9110f19990b2cd7d50b09ff4bc9385ae4453423e218398a5c8762f9503c4e48"
Jul 28 22:38:38.313608 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:38.313499    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9110f19990b2cd7d50b09ff4bc9385ae4453423e218398a5c8762f9503c4e48"} err="failed to get container status \"a9110f19990b2cd7d50b09ff4bc9385ae4453423e218398a5c8762f9503c4e48\": rpc error: code = NotFound desc = could not find container \"a9110f19990b2cd7d50b09ff4bc9385ae4453423e218398a5c8762f9503c4e48\": container with ID starting with a9110f19990b2cd7d50b09ff4bc9385ae4453423e218398a5c8762f9503c4e48 not found: ID does not exist"
Jul 28 22:38:38.323010 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:38.322983    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl"]
Jul 28 22:38:38.326580 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:38.326558    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-b9e632fc-5fdc5d5bc4-f9lkl"]
Jul 28 22:38:38.462466 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:38.462427    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="989b3c31-2558-41c2-a5b6-c7b4114c3ed8" path="/var/lib/kubelet/pods/989b3c31-2558-41c2-a5b6-c7b4114c3ed8/volumes"
Jul 28 22:38:41.168984 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:38:41.168937    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:39:55.009269 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:39:55.009152    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:39:55.013283 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:39:55.010102    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:40:02.167939 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:02.167897    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:40:14.168927 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:14.168893    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:40:23.967837 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:23.967794    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:40:28.798669 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:28.798627    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g"]
Jul 28 22:40:28.799135 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:28.799107    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="989b3c31-2558-41c2-a5b6-c7b4114c3ed8" containerName="payload-processing"
Jul 28 22:40:28.799135 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:28.799121    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="989b3c31-2558-41c2-a5b6-c7b4114c3ed8" containerName="payload-processing"
Jul 28 22:40:28.799246 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:28.799233    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="989b3c31-2558-41c2-a5b6-c7b4114c3ed8" containerName="payload-processing"
Jul 28 22:40:28.802623 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:28.802601    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g"
Jul 28 22:40:28.804634 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:28.804612    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-b45d9e9d\""
Jul 28 22:40:28.804726 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:28.804631    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-b45d9e9d-dockercfg-r4qjh\""
Jul 28 22:40:28.813267 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:28.813198    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g"]
Jul 28 22:40:28.874652 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:28.874615    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bc7691b-714d-40fc-95fc-1493aeafcbf1-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g\" (UID: \"0bc7691b-714d-40fc-95fc-1493aeafcbf1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g"
Jul 28 22:40:28.874825 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:28.874715    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcxp2\" (UniqueName: \"kubernetes.io/projected/0bc7691b-714d-40fc-95fc-1493aeafcbf1-kube-api-access-kcxp2\") pod \"payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g\" (UID: \"0bc7691b-714d-40fc-95fc-1493aeafcbf1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g"
Jul 28 22:40:28.907029 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:28.906995    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7"]
Jul 28 22:40:28.914355 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:28.912891    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7"
Jul 28 22:40:28.921344 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:28.921308    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7"]
Jul 28 22:40:28.975358 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:28.975322    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgt6h\" (UniqueName: \"kubernetes.io/projected/445bca24-8466-45d4-985a-437f500aebb0-kube-api-access-vgt6h\") pod \"payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7\" (UID: \"445bca24-8466-45d4-985a-437f500aebb0\") " pod="openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7"
Jul 28 22:40:28.975569 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:28.975373    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bc7691b-714d-40fc-95fc-1493aeafcbf1-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g\" (UID: \"0bc7691b-714d-40fc-95fc-1493aeafcbf1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g"
Jul 28 22:40:28.975569 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:28.975397    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/445bca24-8466-45d4-985a-437f500aebb0-plugins-config-volume\") pod \"payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7\" (UID: \"445bca24-8466-45d4-985a-437f500aebb0\") " pod="openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7"
Jul 28 22:40:28.975569 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:28.975451    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcxp2\" (UniqueName: \"kubernetes.io/projected/0bc7691b-714d-40fc-95fc-1493aeafcbf1-kube-api-access-kcxp2\") pod \"payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g\" (UID: \"0bc7691b-714d-40fc-95fc-1493aeafcbf1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g"
Jul 28 22:40:28.976048 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:28.976027    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bc7691b-714d-40fc-95fc-1493aeafcbf1-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g\" (UID: \"0bc7691b-714d-40fc-95fc-1493aeafcbf1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g"
Jul 28 22:40:28.984581 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:28.984556    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcxp2\" (UniqueName: \"kubernetes.io/projected/0bc7691b-714d-40fc-95fc-1493aeafcbf1-kube-api-access-kcxp2\") pod \"payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g\" (UID: \"0bc7691b-714d-40fc-95fc-1493aeafcbf1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g"
Jul 28 22:40:29.075888 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:29.075851    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgt6h\" (UniqueName: \"kubernetes.io/projected/445bca24-8466-45d4-985a-437f500aebb0-kube-api-access-vgt6h\") pod \"payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7\" (UID: \"445bca24-8466-45d4-985a-437f500aebb0\") " pod="openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7"
Jul 28 22:40:29.076093 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:29.075906    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/445bca24-8466-45d4-985a-437f500aebb0-plugins-config-volume\") pod \"payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7\" (UID: \"445bca24-8466-45d4-985a-437f500aebb0\") " pod="openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7"
Jul 28 22:40:29.076584 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:29.076563    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/445bca24-8466-45d4-985a-437f500aebb0-plugins-config-volume\") pod \"payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7\" (UID: \"445bca24-8466-45d4-985a-437f500aebb0\") " pod="openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7"
Jul 28 22:40:29.083587 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:29.083554    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgt6h\" (UniqueName: \"kubernetes.io/projected/445bca24-8466-45d4-985a-437f500aebb0-kube-api-access-vgt6h\") pod \"payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7\" (UID: \"445bca24-8466-45d4-985a-437f500aebb0\") " pod="openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7"
Jul 28 22:40:29.117363 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:29.117328    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g"
Jul 28 22:40:29.227961 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:29.227880    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7"
Jul 28 22:40:29.407093 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:29.407068    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7"]
Jul 28 22:40:29.409015 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:40:29.408977    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod445bca24_8466_45d4_985a_437f500aebb0.slice/crio-7f05ccca2cd28b46266251e943ec809de6f545cfc98ed1e9cc91b82242283903 WatchSource:0}: Error finding container 7f05ccca2cd28b46266251e943ec809de6f545cfc98ed1e9cc91b82242283903: Status 404 returned error can't find the container with id 7f05ccca2cd28b46266251e943ec809de6f545cfc98ed1e9cc91b82242283903
Jul 28 22:40:29.466778 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:29.466751    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g"]
Jul 28 22:40:29.470979 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:40:29.470944    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bc7691b_714d_40fc_95fc_1493aeafcbf1.slice/crio-90bc7b09063beab8e1854cdcf8ead35ecdb125c1b19a85be7ea1497c6f36e41d WatchSource:0}: Error finding container 90bc7b09063beab8e1854cdcf8ead35ecdb125c1b19a85be7ea1497c6f36e41d: Status 404 returned error can't find the container with id 90bc7b09063beab8e1854cdcf8ead35ecdb125c1b19a85be7ea1497c6f36e41d
Jul 28 22:40:29.828935 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:29.828893    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g" event={"ID":"0bc7691b-714d-40fc-95fc-1493aeafcbf1","Type":"ContainerStarted","Data":"a258465e6e2e0a583711908e2591d5d2c07a68b4f4b382290cc8b31e31b627ec"}
Jul 28 22:40:29.829405 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:29.828964    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g" event={"ID":"0bc7691b-714d-40fc-95fc-1493aeafcbf1","Type":"ContainerStarted","Data":"90bc7b09063beab8e1854cdcf8ead35ecdb125c1b19a85be7ea1497c6f36e41d"}
Jul 28 22:40:29.829405 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:29.828999    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g"
Jul 28 22:40:29.830564 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:29.830513    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7" event={"ID":"445bca24-8466-45d4-985a-437f500aebb0","Type":"ContainerStarted","Data":"65a377107534dfac266ee4f3c6cc98aa0eac4e47d4f72f351db0ec4b6597e0cf"}
Jul 28 22:40:29.830680 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:29.830588    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7" event={"ID":"445bca24-8466-45d4-985a-437f500aebb0","Type":"ContainerStarted","Data":"7f05ccca2cd28b46266251e943ec809de6f545cfc98ed1e9cc91b82242283903"}
Jul 28 22:40:29.830680 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:29.830622    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7"
Jul 28 22:40:29.850140 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:29.850086    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g" podStartSLOduration=1.850065895 podStartE2EDuration="1.850065895s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:40:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:40:29.849691947 +0000 UTC m=+1837.867657644" watchObservedRunningTime="2026-07-28 22:40:29.850065895 +0000 UTC m=+1837.868031587"
Jul 28 22:40:29.868333 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:29.868283    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7" podStartSLOduration=1.868264297 podStartE2EDuration="1.868264297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:40:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:40:29.86685955 +0000 UTC m=+1837.884825244" watchObservedRunningTime="2026-07-28 22:40:29.868264297 +0000 UTC m=+1837.886229993"
Jul 28 22:40:34.473400 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:34.473359    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:40:40.838310 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:40.838281    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g"
Jul 28 22:40:40.838772 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:40:40.838363    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7"
Jul 28 22:41:10.523504 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:10.523465    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7"]
Jul 28 22:41:10.524096 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:10.523717    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7" podUID="445bca24-8466-45d4-985a-437f500aebb0" containerName="payload-processing" containerID="cri-o://65a377107534dfac266ee4f3c6cc98aa0eac4e47d4f72f351db0ec4b6597e0cf" gracePeriod=30
Jul 28 22:41:10.531754 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:10.531720    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g"]
Jul 28 22:41:10.532012 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:10.531971    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g" podUID="0bc7691b-714d-40fc-95fc-1493aeafcbf1" containerName="payload-pre-processing" containerID="cri-o://a258465e6e2e0a583711908e2591d5d2c07a68b4f4b382290cc8b31e31b627ec" gracePeriod=30
Jul 28 22:41:10.778830 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:10.778762    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7"
Jul 28 22:41:10.808206 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:10.808184    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g"
Jul 28 22:41:10.839600 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:10.839569    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcxp2\" (UniqueName: \"kubernetes.io/projected/0bc7691b-714d-40fc-95fc-1493aeafcbf1-kube-api-access-kcxp2\") pod \"0bc7691b-714d-40fc-95fc-1493aeafcbf1\" (UID: \"0bc7691b-714d-40fc-95fc-1493aeafcbf1\") "
Jul 28 22:41:10.839775 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:10.839636    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgt6h\" (UniqueName: \"kubernetes.io/projected/445bca24-8466-45d4-985a-437f500aebb0-kube-api-access-vgt6h\") pod \"445bca24-8466-45d4-985a-437f500aebb0\" (UID: \"445bca24-8466-45d4-985a-437f500aebb0\") "
Jul 28 22:41:10.839775 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:10.839668    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/445bca24-8466-45d4-985a-437f500aebb0-plugins-config-volume\") pod \"445bca24-8466-45d4-985a-437f500aebb0\" (UID: \"445bca24-8466-45d4-985a-437f500aebb0\") "
Jul 28 22:41:10.839775 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:10.839723    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bc7691b-714d-40fc-95fc-1493aeafcbf1-plugins-config-volume\") pod \"0bc7691b-714d-40fc-95fc-1493aeafcbf1\" (UID: \"0bc7691b-714d-40fc-95fc-1493aeafcbf1\") "
Jul 28 22:41:10.840136 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:10.840102    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/445bca24-8466-45d4-985a-437f500aebb0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "445bca24-8466-45d4-985a-437f500aebb0" (UID: "445bca24-8466-45d4-985a-437f500aebb0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:41:10.840259 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:10.840157    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bc7691b-714d-40fc-95fc-1493aeafcbf1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0bc7691b-714d-40fc-95fc-1493aeafcbf1" (UID: "0bc7691b-714d-40fc-95fc-1493aeafcbf1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:41:10.841707 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:10.841685    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/445bca24-8466-45d4-985a-437f500aebb0-kube-api-access-vgt6h" (OuterVolumeSpecName: "kube-api-access-vgt6h") pod "445bca24-8466-45d4-985a-437f500aebb0" (UID: "445bca24-8466-45d4-985a-437f500aebb0"). InnerVolumeSpecName "kube-api-access-vgt6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:41:10.842046 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:10.842032    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bc7691b-714d-40fc-95fc-1493aeafcbf1-kube-api-access-kcxp2" (OuterVolumeSpecName: "kube-api-access-kcxp2") pod "0bc7691b-714d-40fc-95fc-1493aeafcbf1" (UID: "0bc7691b-714d-40fc-95fc-1493aeafcbf1"). InnerVolumeSpecName "kube-api-access-kcxp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:41:10.940892 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:10.940860    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0bc7691b-714d-40fc-95fc-1493aeafcbf1-plugins-config-volume\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:41:10.940892 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:10.940889    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcxp2\" (UniqueName: \"kubernetes.io/projected/0bc7691b-714d-40fc-95fc-1493aeafcbf1-kube-api-access-kcxp2\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:41:10.940892 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:10.940898    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgt6h\" (UniqueName: \"kubernetes.io/projected/445bca24-8466-45d4-985a-437f500aebb0-kube-api-access-vgt6h\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:41:10.941109 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:10.940907    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/445bca24-8466-45d4-985a-437f500aebb0-plugins-config-volume\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:41:11.029913 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:11.029831    2551 generic.go:356] "Generic (PLEG): container finished" podID="445bca24-8466-45d4-985a-437f500aebb0" containerID="65a377107534dfac266ee4f3c6cc98aa0eac4e47d4f72f351db0ec4b6597e0cf" exitCode=0
Jul 28 22:41:11.029913 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:11.029881    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7"
Jul 28 22:41:11.030137 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:11.029911    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7" event={"ID":"445bca24-8466-45d4-985a-437f500aebb0","Type":"ContainerDied","Data":"65a377107534dfac266ee4f3c6cc98aa0eac4e47d4f72f351db0ec4b6597e0cf"}
Jul 28 22:41:11.030137 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:11.029957    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7" event={"ID":"445bca24-8466-45d4-985a-437f500aebb0","Type":"ContainerDied","Data":"7f05ccca2cd28b46266251e943ec809de6f545cfc98ed1e9cc91b82242283903"}
Jul 28 22:41:11.030137 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:11.029985    2551 scope.go:117] "RemoveContainer" containerID="65a377107534dfac266ee4f3c6cc98aa0eac4e47d4f72f351db0ec4b6597e0cf"
Jul 28 22:41:11.031562 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:11.031521    2551 generic.go:356] "Generic (PLEG): container finished" podID="0bc7691b-714d-40fc-95fc-1493aeafcbf1" containerID="a258465e6e2e0a583711908e2591d5d2c07a68b4f4b382290cc8b31e31b627ec" exitCode=0
Jul 28 22:41:11.031679 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:11.031572    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g"
Jul 28 22:41:11.031679 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:11.031617    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g" event={"ID":"0bc7691b-714d-40fc-95fc-1493aeafcbf1","Type":"ContainerDied","Data":"a258465e6e2e0a583711908e2591d5d2c07a68b4f4b382290cc8b31e31b627ec"}
Jul 28 22:41:11.031679 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:11.031664    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g" event={"ID":"0bc7691b-714d-40fc-95fc-1493aeafcbf1","Type":"ContainerDied","Data":"90bc7b09063beab8e1854cdcf8ead35ecdb125c1b19a85be7ea1497c6f36e41d"}
Jul 28 22:41:11.038866 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:11.038848    2551 scope.go:117] "RemoveContainer" containerID="65a377107534dfac266ee4f3c6cc98aa0eac4e47d4f72f351db0ec4b6597e0cf"
Jul 28 22:41:11.039184 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:41:11.039124    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65a377107534dfac266ee4f3c6cc98aa0eac4e47d4f72f351db0ec4b6597e0cf\": container with ID starting with 65a377107534dfac266ee4f3c6cc98aa0eac4e47d4f72f351db0ec4b6597e0cf not found: ID does not exist" containerID="65a377107534dfac266ee4f3c6cc98aa0eac4e47d4f72f351db0ec4b6597e0cf"
Jul 28 22:41:11.039184 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:11.039154    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65a377107534dfac266ee4f3c6cc98aa0eac4e47d4f72f351db0ec4b6597e0cf"} err="failed to get container status \"65a377107534dfac266ee4f3c6cc98aa0eac4e47d4f72f351db0ec4b6597e0cf\": rpc error: code = NotFound desc = could not find container \"65a377107534dfac266ee4f3c6cc98aa0eac4e47d4f72f351db0ec4b6597e0cf\": container with ID starting with 65a377107534dfac266ee4f3c6cc98aa0eac4e47d4f72f351db0ec4b6597e0cf not found: ID does not exist"
Jul 28 22:41:11.039184 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:11.039176    2551 scope.go:117] "RemoveContainer" containerID="a258465e6e2e0a583711908e2591d5d2c07a68b4f4b382290cc8b31e31b627ec"
Jul 28 22:41:11.049854 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:11.049836    2551 scope.go:117] "RemoveContainer" containerID="a258465e6e2e0a583711908e2591d5d2c07a68b4f4b382290cc8b31e31b627ec"
Jul 28 22:41:11.050101 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:41:11.050080    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a258465e6e2e0a583711908e2591d5d2c07a68b4f4b382290cc8b31e31b627ec\": container with ID starting with a258465e6e2e0a583711908e2591d5d2c07a68b4f4b382290cc8b31e31b627ec not found: ID does not exist" containerID="a258465e6e2e0a583711908e2591d5d2c07a68b4f4b382290cc8b31e31b627ec"
Jul 28 22:41:11.050188 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:11.050106    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a258465e6e2e0a583711908e2591d5d2c07a68b4f4b382290cc8b31e31b627ec"} err="failed to get container status \"a258465e6e2e0a583711908e2591d5d2c07a68b4f4b382290cc8b31e31b627ec\": rpc error: code = NotFound desc = could not find container \"a258465e6e2e0a583711908e2591d5d2c07a68b4f4b382290cc8b31e31b627ec\": container with ID starting with a258465e6e2e0a583711908e2591d5d2c07a68b4f4b382290cc8b31e31b627ec not found: ID does not exist"
Jul 28 22:41:11.053859 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:11.053835    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g"]
Jul 28 22:41:11.057298 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:11.057272    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-b45d9e9d-88694fb7b-vh72g"]
Jul 28 22:41:11.066568 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:11.066516    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7"]
Jul 28 22:41:11.070518 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:11.070494    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-b45d9e9d-697d99889d-b9rx7"]
Jul 28 22:41:12.461988 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:12.461953    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bc7691b-714d-40fc-95fc-1493aeafcbf1" path="/var/lib/kubelet/pods/0bc7691b-714d-40fc-95fc-1493aeafcbf1/volumes"
Jul 28 22:41:12.462329 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:12.462279    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="445bca24-8466-45d4-985a-437f500aebb0" path="/var/lib/kubelet/pods/445bca24-8466-45d4-985a-437f500aebb0/volumes"
Jul 28 22:41:13.469895 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:13.469859    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:41:41.171466 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:41.171432    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz"]
Jul 28 22:41:41.171972 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:41.171795    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="445bca24-8466-45d4-985a-437f500aebb0" containerName="payload-processing"
Jul 28 22:41:41.171972 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:41.171809    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="445bca24-8466-45d4-985a-437f500aebb0" containerName="payload-processing"
Jul 28 22:41:41.171972 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:41.171833    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bc7691b-714d-40fc-95fc-1493aeafcbf1" containerName="payload-pre-processing"
Jul 28 22:41:41.171972 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:41.171839    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bc7691b-714d-40fc-95fc-1493aeafcbf1" containerName="payload-pre-processing"
Jul 28 22:41:41.171972 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:41.171903    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="445bca24-8466-45d4-985a-437f500aebb0" containerName="payload-processing"
Jul 28 22:41:41.171972 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:41.171917    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bc7691b-714d-40fc-95fc-1493aeafcbf1" containerName="payload-pre-processing"
Jul 28 22:41:41.175696 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:41.175678    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz"
Jul 28 22:41:41.177898 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:41.177874    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-8f322d7b\""
Jul 28 22:41:41.178010 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:41.177875    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-8f322d7b-dockercfg-846qq\""
Jul 28 22:41:41.183479 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:41.183419    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz"]
Jul 28 22:41:41.301024 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:41.300991    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62c20ce9-e0cb-4020-9efe-fce7872b5861-plugins-config-volume\") pod \"payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz\" (UID: \"62c20ce9-e0cb-4020-9efe-fce7872b5861\") " pod="openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz"
Jul 28 22:41:41.301273 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:41.301117    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxlr2\" (UniqueName: \"kubernetes.io/projected/62c20ce9-e0cb-4020-9efe-fce7872b5861-kube-api-access-zxlr2\") pod \"payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz\" (UID: \"62c20ce9-e0cb-4020-9efe-fce7872b5861\") " pod="openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz"
Jul 28 22:41:41.401820 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:41.401766    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62c20ce9-e0cb-4020-9efe-fce7872b5861-plugins-config-volume\") pod \"payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz\" (UID: \"62c20ce9-e0cb-4020-9efe-fce7872b5861\") " pod="openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz"
Jul 28 22:41:41.402005 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:41.401894    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxlr2\" (UniqueName: \"kubernetes.io/projected/62c20ce9-e0cb-4020-9efe-fce7872b5861-kube-api-access-zxlr2\") pod \"payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz\" (UID: \"62c20ce9-e0cb-4020-9efe-fce7872b5861\") " pod="openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz"
Jul 28 22:41:41.402485 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:41.402465    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62c20ce9-e0cb-4020-9efe-fce7872b5861-plugins-config-volume\") pod \"payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz\" (UID: \"62c20ce9-e0cb-4020-9efe-fce7872b5861\") " pod="openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz"
Jul 28 22:41:41.410096 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:41.410077    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxlr2\" (UniqueName: \"kubernetes.io/projected/62c20ce9-e0cb-4020-9efe-fce7872b5861-kube-api-access-zxlr2\") pod \"payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz\" (UID: \"62c20ce9-e0cb-4020-9efe-fce7872b5861\") " pod="openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz"
Jul 28 22:41:41.488642 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:41.488554    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz"
Jul 28 22:41:41.621505 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:41.621475    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz"]
Jul 28 22:41:41.623958 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:41:41.623916    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62c20ce9_e0cb_4020_9efe_fce7872b5861.slice/crio-d0b5a30b5cb5fb541f5a3f26c0d96ac78ba49992cde24412cba91b2315984f50 WatchSource:0}: Error finding container d0b5a30b5cb5fb541f5a3f26c0d96ac78ba49992cde24412cba91b2315984f50: Status 404 returned error can't find the container with id d0b5a30b5cb5fb541f5a3f26c0d96ac78ba49992cde24412cba91b2315984f50
Jul 28 22:41:42.178141 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:42.178105    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz" event={"ID":"62c20ce9-e0cb-4020-9efe-fce7872b5861","Type":"ContainerStarted","Data":"d92a5d53e8ddbc1b9bbe9c6780880fb04e639936299d6410e902dacddb7d52ac"}
Jul 28 22:41:42.178560 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:42.178155    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz" event={"ID":"62c20ce9-e0cb-4020-9efe-fce7872b5861","Type":"ContainerStarted","Data":"d0b5a30b5cb5fb541f5a3f26c0d96ac78ba49992cde24412cba91b2315984f50"}
Jul 28 22:41:42.178560 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:42.178196    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz"
Jul 28 22:41:42.192621 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:42.192573    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz" podStartSLOduration=1.192557909 podStartE2EDuration="1.192557909s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:41:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:41:42.192203046 +0000 UTC m=+1910.210168740" watchObservedRunningTime="2026-07-28 22:41:42.192557909 +0000 UTC m=+1910.210523597"
Jul 28 22:41:46.270521 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:46.270484    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:41:53.186243 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:41:53.186206    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz"
Jul 28 22:42:19.865428 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:42:19.865387    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:42:45.074712 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:42:45.074623    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:43:04.066690 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:43:04.066654    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:43:05.438053 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:43:05.438015    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz"]
Jul 28 22:43:05.438556 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:43:05.438259    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz" podUID="62c20ce9-e0cb-4020-9efe-fce7872b5861" containerName="payload-processing" containerID="cri-o://d92a5d53e8ddbc1b9bbe9c6780880fb04e639936299d6410e902dacddb7d52ac" gracePeriod=30
Jul 28 22:43:05.560278 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:43:05.560251    2551 generic.go:356] "Generic (PLEG): container finished" podID="62c20ce9-e0cb-4020-9efe-fce7872b5861" containerID="d92a5d53e8ddbc1b9bbe9c6780880fb04e639936299d6410e902dacddb7d52ac" exitCode=0
Jul 28 22:43:05.560409 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:43:05.560322    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz" event={"ID":"62c20ce9-e0cb-4020-9efe-fce7872b5861","Type":"ContainerDied","Data":"d92a5d53e8ddbc1b9bbe9c6780880fb04e639936299d6410e902dacddb7d52ac"}
Jul 28 22:43:05.691275 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:43:05.691215    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz"
Jul 28 22:43:05.726986 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:43:05.726950    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxlr2\" (UniqueName: \"kubernetes.io/projected/62c20ce9-e0cb-4020-9efe-fce7872b5861-kube-api-access-zxlr2\") pod \"62c20ce9-e0cb-4020-9efe-fce7872b5861\" (UID: \"62c20ce9-e0cb-4020-9efe-fce7872b5861\") "
Jul 28 22:43:05.727166 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:43:05.727013    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62c20ce9-e0cb-4020-9efe-fce7872b5861-plugins-config-volume\") pod \"62c20ce9-e0cb-4020-9efe-fce7872b5861\" (UID: \"62c20ce9-e0cb-4020-9efe-fce7872b5861\") "
Jul 28 22:43:05.727448 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:43:05.727420    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62c20ce9-e0cb-4020-9efe-fce7872b5861-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "62c20ce9-e0cb-4020-9efe-fce7872b5861" (UID: "62c20ce9-e0cb-4020-9efe-fce7872b5861"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:43:05.729002 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:43:05.728976    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62c20ce9-e0cb-4020-9efe-fce7872b5861-kube-api-access-zxlr2" (OuterVolumeSpecName: "kube-api-access-zxlr2") pod "62c20ce9-e0cb-4020-9efe-fce7872b5861" (UID: "62c20ce9-e0cb-4020-9efe-fce7872b5861"). InnerVolumeSpecName "kube-api-access-zxlr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:43:05.828673 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:43:05.828638    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62c20ce9-e0cb-4020-9efe-fce7872b5861-plugins-config-volume\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:43:05.828673 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:43:05.828665    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxlr2\" (UniqueName: \"kubernetes.io/projected/62c20ce9-e0cb-4020-9efe-fce7872b5861-kube-api-access-zxlr2\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:43:06.565697 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:43:06.565661    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz"
Jul 28 22:43:06.566162 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:43:06.565701    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz" event={"ID":"62c20ce9-e0cb-4020-9efe-fce7872b5861","Type":"ContainerDied","Data":"d0b5a30b5cb5fb541f5a3f26c0d96ac78ba49992cde24412cba91b2315984f50"}
Jul 28 22:43:06.566162 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:43:06.565771    2551 scope.go:117] "RemoveContainer" containerID="d92a5d53e8ddbc1b9bbe9c6780880fb04e639936299d6410e902dacddb7d52ac"
Jul 28 22:43:06.592486 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:43:06.592461    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz"]
Jul 28 22:43:06.600246 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:43:06.600222    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-8f322d7b-5cb7cddf78-l8fwz"]
Jul 28 22:43:08.464069 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:43:08.464031    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62c20ce9-e0cb-4020-9efe-fce7872b5861" path="/var/lib/kubelet/pods/62c20ce9-e0cb-4020-9efe-fce7872b5861/volumes"
Jul 28 22:43:12.873588 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:43:12.873554    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:44:00.428914 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:00.428869    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:44:13.080905 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:13.080867    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:44:55.037414 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:55.037288    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:44:55.040197 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:55.040177    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:44:56.229653 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:56.229620    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d"]
Jul 28 22:44:56.230033 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:56.229949    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62c20ce9-e0cb-4020-9efe-fce7872b5861" containerName="payload-processing"
Jul 28 22:44:56.230033 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:56.229960    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="62c20ce9-e0cb-4020-9efe-fce7872b5861" containerName="payload-processing"
Jul 28 22:44:56.230033 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:56.230018    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="62c20ce9-e0cb-4020-9efe-fce7872b5861" containerName="payload-processing"
Jul 28 22:44:56.232781 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:56.232764    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d"
Jul 28 22:44:56.235038 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:56.235011    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-104d62\""
Jul 28 22:44:56.235175 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:56.235101    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-104d62-dockercfg-g4fr2\""
Jul 28 22:44:56.241477 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:56.241450    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d"]
Jul 28 22:44:56.270119 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:56.270088    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlp6k\" (UniqueName: \"kubernetes.io/projected/d891edf6-559a-49aa-b484-3a94c81d6fcf-kube-api-access-rlp6k\") pod \"payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d\" (UID: \"d891edf6-559a-49aa-b484-3a94c81d6fcf\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d"
Jul 28 22:44:56.270277 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:56.270185    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d891edf6-559a-49aa-b484-3a94c81d6fcf-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d\" (UID: \"d891edf6-559a-49aa-b484-3a94c81d6fcf\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d"
Jul 28 22:44:56.371442 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:56.371405    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d891edf6-559a-49aa-b484-3a94c81d6fcf-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d\" (UID: \"d891edf6-559a-49aa-b484-3a94c81d6fcf\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d"
Jul 28 22:44:56.371631 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:56.371455    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlp6k\" (UniqueName: \"kubernetes.io/projected/d891edf6-559a-49aa-b484-3a94c81d6fcf-kube-api-access-rlp6k\") pod \"payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d\" (UID: \"d891edf6-559a-49aa-b484-3a94c81d6fcf\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d"
Jul 28 22:44:56.372149 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:56.372127    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d891edf6-559a-49aa-b484-3a94c81d6fcf-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d\" (UID: \"d891edf6-559a-49aa-b484-3a94c81d6fcf\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d"
Jul 28 22:44:56.378993 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:56.378970    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlp6k\" (UniqueName: \"kubernetes.io/projected/d891edf6-559a-49aa-b484-3a94c81d6fcf-kube-api-access-rlp6k\") pod \"payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d\" (UID: \"d891edf6-559a-49aa-b484-3a94c81d6fcf\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d"
Jul 28 22:44:56.543150 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:56.543064    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d"
Jul 28 22:44:56.671731 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:56.671574    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d"]
Jul 28 22:44:56.673921 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:44:56.673890    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd891edf6_559a_49aa_b484_3a94c81d6fcf.slice/crio-c10358e74b2ff8f5b06fd2c6705dd0acf4d4effa267654763bbbabd655cc3ede WatchSource:0}: Error finding container c10358e74b2ff8f5b06fd2c6705dd0acf4d4effa267654763bbbabd655cc3ede: Status 404 returned error can't find the container with id c10358e74b2ff8f5b06fd2c6705dd0acf4d4effa267654763bbbabd655cc3ede
Jul 28 22:44:57.071233 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:57.071190    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d" event={"ID":"d891edf6-559a-49aa-b484-3a94c81d6fcf","Type":"ContainerStarted","Data":"9cbc506af953ab97f4c5a6114bc440f36c19984bc17ee62dde1ca2cbde127cb9"}
Jul 28 22:44:57.071405 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:57.071244    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d" event={"ID":"d891edf6-559a-49aa-b484-3a94c81d6fcf","Type":"ContainerStarted","Data":"c10358e74b2ff8f5b06fd2c6705dd0acf4d4effa267654763bbbabd655cc3ede"}
Jul 28 22:44:57.071405 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:57.071332    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d"
Jul 28 22:44:57.086029 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:44:57.085977    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d" podStartSLOduration=1.085958102 podStartE2EDuration="1.085958102s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:44:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:44:57.085357343 +0000 UTC m=+2105.103323043" watchObservedRunningTime="2026-07-28 22:44:57.085958102 +0000 UTC m=+2105.103923796"
Jul 28 22:45:00.968688 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:00.968650    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:45:08.078278 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:08.078245    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d"
Jul 28 22:45:25.450480 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:25.450438    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k"]
Jul 28 22:45:25.454029 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:25.454008    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k"
Jul 28 22:45:25.456327 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:25.456306    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-72fe47\""
Jul 28 22:45:25.463412 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:25.463392    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k"]
Jul 28 22:45:25.497415 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:25.497384    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg"]
Jul 28 22:45:25.500700 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:25.500684    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg"
Jul 28 22:45:25.502656 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:25.502633    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-72fe47-dockercfg-snzrm\""
Jul 28 22:45:25.511683 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:25.511656    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg"]
Jul 28 22:45:25.519618 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:25.519592    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k\" (UID: \"42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k"
Jul 28 22:45:25.519756 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:25.519649    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wrj2\" (UniqueName: \"kubernetes.io/projected/42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4-kube-api-access-4wrj2\") pod \"payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k\" (UID: \"42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k"
Jul 28 22:45:25.620256 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:25.620214    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sfmc\" (UniqueName: \"kubernetes.io/projected/eadd9aec-7f0a-4d79-99d0-ec07477f0288-kube-api-access-6sfmc\") pod \"payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg\" (UID: \"eadd9aec-7f0a-4d79-99d0-ec07477f0288\") " pod="openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg"
Jul 28 22:45:25.620410 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:25.620277    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k\" (UID: \"42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k"
Jul 28 22:45:25.620410 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:25.620303    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eadd9aec-7f0a-4d79-99d0-ec07477f0288-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg\" (UID: \"eadd9aec-7f0a-4d79-99d0-ec07477f0288\") " pod="openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg"
Jul 28 22:45:25.620410 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:25.620338    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wrj2\" (UniqueName: \"kubernetes.io/projected/42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4-kube-api-access-4wrj2\") pod \"payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k\" (UID: \"42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k"
Jul 28 22:45:25.621098 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:25.621075    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k\" (UID: \"42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k"
Jul 28 22:45:25.627891 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:25.627865    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wrj2\" (UniqueName: \"kubernetes.io/projected/42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4-kube-api-access-4wrj2\") pod \"payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k\" (UID: \"42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k"
Jul 28 22:45:25.720927 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:25.720840    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eadd9aec-7f0a-4d79-99d0-ec07477f0288-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg\" (UID: \"eadd9aec-7f0a-4d79-99d0-ec07477f0288\") " pod="openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg"
Jul 28 22:45:25.720927 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:25.720927    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sfmc\" (UniqueName: \"kubernetes.io/projected/eadd9aec-7f0a-4d79-99d0-ec07477f0288-kube-api-access-6sfmc\") pod \"payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg\" (UID: \"eadd9aec-7f0a-4d79-99d0-ec07477f0288\") " pod="openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg"
Jul 28 22:45:25.721579 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:25.721561    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eadd9aec-7f0a-4d79-99d0-ec07477f0288-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg\" (UID: \"eadd9aec-7f0a-4d79-99d0-ec07477f0288\") " pod="openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg"
Jul 28 22:45:25.728102 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:25.728080    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sfmc\" (UniqueName: \"kubernetes.io/projected/eadd9aec-7f0a-4d79-99d0-ec07477f0288-kube-api-access-6sfmc\") pod \"payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg\" (UID: \"eadd9aec-7f0a-4d79-99d0-ec07477f0288\") " pod="openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg"
Jul 28 22:45:25.765115 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:25.765072    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k"
Jul 28 22:45:25.813607 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:25.813068    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg"
Jul 28 22:45:25.959635 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:25.959612    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg"]
Jul 28 22:45:25.961970 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:45:25.961943    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeadd9aec_7f0a_4d79_99d0_ec07477f0288.slice/crio-42d8a7c9f8c963b095e722cc1b93e3b5b34e52e4e3aa73bf393b2fc19adffe5a WatchSource:0}: Error finding container 42d8a7c9f8c963b095e722cc1b93e3b5b34e52e4e3aa73bf393b2fc19adffe5a: Status 404 returned error can't find the container with id 42d8a7c9f8c963b095e722cc1b93e3b5b34e52e4e3aa73bf393b2fc19adffe5a
Jul 28 22:45:26.116053 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:26.115794    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k"]
Jul 28 22:45:26.118728 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:45:26.118698    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42fb39a8_5e72_4eec_9a4e_395d9ecbbfb4.slice/crio-d78ff21848c8d90c564c90c0b24fc326911530bd510437434f164d6e16a2b2a1 WatchSource:0}: Error finding container d78ff21848c8d90c564c90c0b24fc326911530bd510437434f164d6e16a2b2a1: Status 404 returned error can't find the container with id d78ff21848c8d90c564c90c0b24fc326911530bd510437434f164d6e16a2b2a1
Jul 28 22:45:26.205410 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:26.205376    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg" event={"ID":"eadd9aec-7f0a-4d79-99d0-ec07477f0288","Type":"ContainerStarted","Data":"18acb48519ab96d159213983c2ae0e6a4423e2ed6296d138a7479c36a0e07ed3"}
Jul 28 22:45:26.205606 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:26.205440    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg"
Jul 28 22:45:26.205606 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:26.205462    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg" event={"ID":"eadd9aec-7f0a-4d79-99d0-ec07477f0288","Type":"ContainerStarted","Data":"42d8a7c9f8c963b095e722cc1b93e3b5b34e52e4e3aa73bf393b2fc19adffe5a"}
Jul 28 22:45:26.206870 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:26.206846    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k" event={"ID":"42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4","Type":"ContainerStarted","Data":"9b9ae5a1b80b3a954d80627fe6fb8a97d043fc11584e0afdbe1030ec4b24bd98"}
Jul 28 22:45:26.206965 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:26.206886    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k" event={"ID":"42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4","Type":"ContainerStarted","Data":"d78ff21848c8d90c564c90c0b24fc326911530bd510437434f164d6e16a2b2a1"}
Jul 28 22:45:26.206965 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:26.206918    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k"
Jul 28 22:45:26.221038 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:26.220994    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg" podStartSLOduration=1.220980058 podStartE2EDuration="1.220980058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:45:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:45:26.220243767 +0000 UTC m=+2134.238209463" watchObservedRunningTime="2026-07-28 22:45:26.220980058 +0000 UTC m=+2134.238945751"
Jul 28 22:45:26.234915 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:26.234835    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k" podStartSLOduration=1.234821326 podStartE2EDuration="1.234821326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:45:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:45:26.234004202 +0000 UTC m=+2134.251969920" watchObservedRunningTime="2026-07-28 22:45:26.234821326 +0000 UTC m=+2134.252787020"
Jul 28 22:45:32.468972 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:32.468944    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:45:37.213857 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:37.213803    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k" podUID="42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.133.0.46:9004: connect: connection refused"
Jul 28 22:45:37.214334 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:37.214307    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg"
Jul 28 22:45:47.212383 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:47.212310    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k"
Jul 28 22:45:55.767019 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:45:55.766980    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:46:06.777880 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:46:06.777843    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:46:23.064229 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:46:23.064194    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:46:36.564097 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:46:36.564061    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:47:03.968117 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:03.968074    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:47:10.687764 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:10.687726    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d"]
Jul 28 22:47:10.688311 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:10.687976    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d" podUID="d891edf6-559a-49aa-b484-3a94c81d6fcf" containerName="payload-pre-processing" containerID="cri-o://9cbc506af953ab97f4c5a6114bc440f36c19984bc17ee62dde1ca2cbde127cb9" gracePeriod=30
Jul 28 22:47:10.960910 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:10.960839    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d"
Jul 28 22:47:11.028111 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:11.028073    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d891edf6-559a-49aa-b484-3a94c81d6fcf-plugins-config-volume\") pod \"d891edf6-559a-49aa-b484-3a94c81d6fcf\" (UID: \"d891edf6-559a-49aa-b484-3a94c81d6fcf\") "
Jul 28 22:47:11.028278 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:11.028160    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlp6k\" (UniqueName: \"kubernetes.io/projected/d891edf6-559a-49aa-b484-3a94c81d6fcf-kube-api-access-rlp6k\") pod \"d891edf6-559a-49aa-b484-3a94c81d6fcf\" (UID: \"d891edf6-559a-49aa-b484-3a94c81d6fcf\") "
Jul 28 22:47:11.028454 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:11.028429    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d891edf6-559a-49aa-b484-3a94c81d6fcf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d891edf6-559a-49aa-b484-3a94c81d6fcf" (UID: "d891edf6-559a-49aa-b484-3a94c81d6fcf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:47:11.030188 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:11.030162    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d891edf6-559a-49aa-b484-3a94c81d6fcf-kube-api-access-rlp6k" (OuterVolumeSpecName: "kube-api-access-rlp6k") pod "d891edf6-559a-49aa-b484-3a94c81d6fcf" (UID: "d891edf6-559a-49aa-b484-3a94c81d6fcf"). InnerVolumeSpecName "kube-api-access-rlp6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:47:11.129546 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:11.129498    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlp6k\" (UniqueName: \"kubernetes.io/projected/d891edf6-559a-49aa-b484-3a94c81d6fcf-kube-api-access-rlp6k\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:47:11.129546 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:11.129547    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d891edf6-559a-49aa-b484-3a94c81d6fcf-plugins-config-volume\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:47:11.689625 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:11.689585    2551 generic.go:356] "Generic (PLEG): container finished" podID="d891edf6-559a-49aa-b484-3a94c81d6fcf" containerID="9cbc506af953ab97f4c5a6114bc440f36c19984bc17ee62dde1ca2cbde127cb9" exitCode=0
Jul 28 22:47:11.690029 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:11.689643    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d"
Jul 28 22:47:11.690029 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:11.689676    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d" event={"ID":"d891edf6-559a-49aa-b484-3a94c81d6fcf","Type":"ContainerDied","Data":"9cbc506af953ab97f4c5a6114bc440f36c19984bc17ee62dde1ca2cbde127cb9"}
Jul 28 22:47:11.690029 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:11.689724    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d" event={"ID":"d891edf6-559a-49aa-b484-3a94c81d6fcf","Type":"ContainerDied","Data":"c10358e74b2ff8f5b06fd2c6705dd0acf4d4effa267654763bbbabd655cc3ede"}
Jul 28 22:47:11.690029 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:11.689745    2551 scope.go:117] "RemoveContainer" containerID="9cbc506af953ab97f4c5a6114bc440f36c19984bc17ee62dde1ca2cbde127cb9"
Jul 28 22:47:11.698793 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:11.698777    2551 scope.go:117] "RemoveContainer" containerID="9cbc506af953ab97f4c5a6114bc440f36c19984bc17ee62dde1ca2cbde127cb9"
Jul 28 22:47:11.699068 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:47:11.699045    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cbc506af953ab97f4c5a6114bc440f36c19984bc17ee62dde1ca2cbde127cb9\": container with ID starting with 9cbc506af953ab97f4c5a6114bc440f36c19984bc17ee62dde1ca2cbde127cb9 not found: ID does not exist" containerID="9cbc506af953ab97f4c5a6114bc440f36c19984bc17ee62dde1ca2cbde127cb9"
Jul 28 22:47:11.699139 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:11.699077    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cbc506af953ab97f4c5a6114bc440f36c19984bc17ee62dde1ca2cbde127cb9"} err="failed to get container status \"9cbc506af953ab97f4c5a6114bc440f36c19984bc17ee62dde1ca2cbde127cb9\": rpc error: code = NotFound desc = could not find container \"9cbc506af953ab97f4c5a6114bc440f36c19984bc17ee62dde1ca2cbde127cb9\": container with ID starting with 9cbc506af953ab97f4c5a6114bc440f36c19984bc17ee62dde1ca2cbde127cb9 not found: ID does not exist"
Jul 28 22:47:11.710108 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:11.710084    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d"]
Jul 28 22:47:11.713871 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:11.713850    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-104d62-74f79577c6-7fk7d"]
Jul 28 22:47:12.462476 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:12.462445    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d891edf6-559a-49aa-b484-3a94c81d6fcf" path="/var/lib/kubelet/pods/d891edf6-559a-49aa-b484-3a94c81d6fcf/volumes"
Jul 28 22:47:19.671148 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:19.671108    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:47:28.160064 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.160029    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg"]
Jul 28 22:47:28.160556 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.160283    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg" podUID="eadd9aec-7f0a-4d79-99d0-ec07477f0288" containerName="payload-processing" containerID="cri-o://18acb48519ab96d159213983c2ae0e6a4423e2ed6296d138a7479c36a0e07ed3" gracePeriod=30
Jul 28 22:47:28.166110 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.166083    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k"]
Jul 28 22:47:28.166334 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.166289    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k" podUID="42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4" containerName="payload-pre-processing" containerID="cri-o://9b9ae5a1b80b3a954d80627fe6fb8a97d043fc11584e0afdbe1030ec4b24bd98" gracePeriod=30
Jul 28 22:47:28.454645 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.454622    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg"
Jul 28 22:47:28.457662 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.457641    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k"
Jul 28 22:47:28.581813 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.581771    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wrj2\" (UniqueName: \"kubernetes.io/projected/42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4-kube-api-access-4wrj2\") pod \"42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4\" (UID: \"42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4\") "
Jul 28 22:47:28.582005 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.581838    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eadd9aec-7f0a-4d79-99d0-ec07477f0288-plugins-config-volume\") pod \"eadd9aec-7f0a-4d79-99d0-ec07477f0288\" (UID: \"eadd9aec-7f0a-4d79-99d0-ec07477f0288\") "
Jul 28 22:47:28.582005 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.581892    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6sfmc\" (UniqueName: \"kubernetes.io/projected/eadd9aec-7f0a-4d79-99d0-ec07477f0288-kube-api-access-6sfmc\") pod \"eadd9aec-7f0a-4d79-99d0-ec07477f0288\" (UID: \"eadd9aec-7f0a-4d79-99d0-ec07477f0288\") "
Jul 28 22:47:28.582005 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.581930    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4-plugins-config-volume\") pod \"42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4\" (UID: \"42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4\") "
Jul 28 22:47:28.582573 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.582187    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eadd9aec-7f0a-4d79-99d0-ec07477f0288-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eadd9aec-7f0a-4d79-99d0-ec07477f0288" (UID: "eadd9aec-7f0a-4d79-99d0-ec07477f0288"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:47:28.582573 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.582472    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4" (UID: "42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:47:28.582765 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.582603    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eadd9aec-7f0a-4d79-99d0-ec07477f0288-plugins-config-volume\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:47:28.584103 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.584076    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eadd9aec-7f0a-4d79-99d0-ec07477f0288-kube-api-access-6sfmc" (OuterVolumeSpecName: "kube-api-access-6sfmc") pod "eadd9aec-7f0a-4d79-99d0-ec07477f0288" (UID: "eadd9aec-7f0a-4d79-99d0-ec07477f0288"). InnerVolumeSpecName "kube-api-access-6sfmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:47:28.584206 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.584104    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4-kube-api-access-4wrj2" (OuterVolumeSpecName: "kube-api-access-4wrj2") pod "42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4" (UID: "42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4"). InnerVolumeSpecName "kube-api-access-4wrj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:47:28.683712 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.683635    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wrj2\" (UniqueName: \"kubernetes.io/projected/42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4-kube-api-access-4wrj2\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:47:28.683712 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.683662    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sfmc\" (UniqueName: \"kubernetes.io/projected/eadd9aec-7f0a-4d79-99d0-ec07477f0288-kube-api-access-6sfmc\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:47:28.683712 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.683673    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4-plugins-config-volume\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:47:28.775861 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.775827    2551 generic.go:356] "Generic (PLEG): container finished" podID="42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4" containerID="9b9ae5a1b80b3a954d80627fe6fb8a97d043fc11584e0afdbe1030ec4b24bd98" exitCode=0
Jul 28 22:47:28.776045 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.775887    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k"
Jul 28 22:47:28.776045 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.775917    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k" event={"ID":"42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4","Type":"ContainerDied","Data":"9b9ae5a1b80b3a954d80627fe6fb8a97d043fc11584e0afdbe1030ec4b24bd98"}
Jul 28 22:47:28.776045 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.775978    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k" event={"ID":"42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4","Type":"ContainerDied","Data":"d78ff21848c8d90c564c90c0b24fc326911530bd510437434f164d6e16a2b2a1"}
Jul 28 22:47:28.776045 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.776011    2551 scope.go:117] "RemoveContainer" containerID="9b9ae5a1b80b3a954d80627fe6fb8a97d043fc11584e0afdbe1030ec4b24bd98"
Jul 28 22:47:28.777242 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.777219    2551 generic.go:356] "Generic (PLEG): container finished" podID="eadd9aec-7f0a-4d79-99d0-ec07477f0288" containerID="18acb48519ab96d159213983c2ae0e6a4423e2ed6296d138a7479c36a0e07ed3" exitCode=0
Jul 28 22:47:28.777330 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.777258    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg" event={"ID":"eadd9aec-7f0a-4d79-99d0-ec07477f0288","Type":"ContainerDied","Data":"18acb48519ab96d159213983c2ae0e6a4423e2ed6296d138a7479c36a0e07ed3"}
Jul 28 22:47:28.777330 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.777271    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg"
Jul 28 22:47:28.777330 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.777300    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg" event={"ID":"eadd9aec-7f0a-4d79-99d0-ec07477f0288","Type":"ContainerDied","Data":"42d8a7c9f8c963b095e722cc1b93e3b5b34e52e4e3aa73bf393b2fc19adffe5a"}
Jul 28 22:47:28.788669 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.786868    2551 scope.go:117] "RemoveContainer" containerID="9b9ae5a1b80b3a954d80627fe6fb8a97d043fc11584e0afdbe1030ec4b24bd98"
Jul 28 22:47:28.788958 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:47:28.788921    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b9ae5a1b80b3a954d80627fe6fb8a97d043fc11584e0afdbe1030ec4b24bd98\": container with ID starting with 9b9ae5a1b80b3a954d80627fe6fb8a97d043fc11584e0afdbe1030ec4b24bd98 not found: ID does not exist" containerID="9b9ae5a1b80b3a954d80627fe6fb8a97d043fc11584e0afdbe1030ec4b24bd98"
Jul 28 22:47:28.789043 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.788971    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b9ae5a1b80b3a954d80627fe6fb8a97d043fc11584e0afdbe1030ec4b24bd98"} err="failed to get container status \"9b9ae5a1b80b3a954d80627fe6fb8a97d043fc11584e0afdbe1030ec4b24bd98\": rpc error: code = NotFound desc = could not find container \"9b9ae5a1b80b3a954d80627fe6fb8a97d043fc11584e0afdbe1030ec4b24bd98\": container with ID starting with 9b9ae5a1b80b3a954d80627fe6fb8a97d043fc11584e0afdbe1030ec4b24bd98 not found: ID does not exist"
Jul 28 22:47:28.789043 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.789013    2551 scope.go:117] "RemoveContainer" containerID="18acb48519ab96d159213983c2ae0e6a4423e2ed6296d138a7479c36a0e07ed3"
Jul 28 22:47:28.797503 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.797486    2551 scope.go:117] "RemoveContainer" containerID="18acb48519ab96d159213983c2ae0e6a4423e2ed6296d138a7479c36a0e07ed3"
Jul 28 22:47:28.797770 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:47:28.797750    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18acb48519ab96d159213983c2ae0e6a4423e2ed6296d138a7479c36a0e07ed3\": container with ID starting with 18acb48519ab96d159213983c2ae0e6a4423e2ed6296d138a7479c36a0e07ed3 not found: ID does not exist" containerID="18acb48519ab96d159213983c2ae0e6a4423e2ed6296d138a7479c36a0e07ed3"
Jul 28 22:47:28.797828 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.797777    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18acb48519ab96d159213983c2ae0e6a4423e2ed6296d138a7479c36a0e07ed3"} err="failed to get container status \"18acb48519ab96d159213983c2ae0e6a4423e2ed6296d138a7479c36a0e07ed3\": rpc error: code = NotFound desc = could not find container \"18acb48519ab96d159213983c2ae0e6a4423e2ed6296d138a7479c36a0e07ed3\": container with ID starting with 18acb48519ab96d159213983c2ae0e6a4423e2ed6296d138a7479c36a0e07ed3 not found: ID does not exist"
Jul 28 22:47:28.805221 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.805200    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k"]
Jul 28 22:47:28.808395 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.808375    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-72fe47-75ff898788-bx58k"]
Jul 28 22:47:28.817915 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.817894    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg"]
Jul 28 22:47:28.822173 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:28.822155    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-72fe47-7f77fcdff4-vp5xg"]
Jul 28 22:47:30.462910 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:30.462877    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4" path="/var/lib/kubelet/pods/42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4/volumes"
Jul 28 22:47:30.463281 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:30.463228    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eadd9aec-7f0a-4d79-99d0-ec07477f0288" path="/var/lib/kubelet/pods/eadd9aec-7f0a-4d79-99d0-ec07477f0288/volumes"
Jul 28 22:47:31.368537 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:31.368501    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:47:39.117358 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.117314    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh"]
Jul 28 22:47:39.117793 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.117685    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d891edf6-559a-49aa-b484-3a94c81d6fcf" containerName="payload-pre-processing"
Jul 28 22:47:39.117793 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.117698    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="d891edf6-559a-49aa-b484-3a94c81d6fcf" containerName="payload-pre-processing"
Jul 28 22:47:39.117793 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.117709    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eadd9aec-7f0a-4d79-99d0-ec07477f0288" containerName="payload-processing"
Jul 28 22:47:39.117793 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.117716    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="eadd9aec-7f0a-4d79-99d0-ec07477f0288" containerName="payload-processing"
Jul 28 22:47:39.117793 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.117728    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4" containerName="payload-pre-processing"
Jul 28 22:47:39.117793 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.117736    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4" containerName="payload-pre-processing"
Jul 28 22:47:39.117793 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.117786    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="d891edf6-559a-49aa-b484-3a94c81d6fcf" containerName="payload-pre-processing"
Jul 28 22:47:39.117793 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.117794    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="eadd9aec-7f0a-4d79-99d0-ec07477f0288" containerName="payload-processing"
Jul 28 22:47:39.118234 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.117802    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="42fb39a8-5e72-4eec-9a4e-395d9ecbbfb4" containerName="payload-pre-processing"
Jul 28 22:47:39.122324 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.122300    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh"
Jul 28 22:47:39.124734 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.124709    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-63f77a\""
Jul 28 22:47:39.124876 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.124831    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-63f77a-dockercfg-r749p\""
Jul 28 22:47:39.129995 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.129688    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh"]
Jul 28 22:47:39.143513 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.143489    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6"]
Jul 28 22:47:39.146719 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.146705    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6"
Jul 28 22:47:39.157121 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.157102    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6"]
Jul 28 22:47:39.283106 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.283067    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58cb9eb4-0c57-4e35-930b-e6fa2ad8007f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh\" (UID: \"58cb9eb4-0c57-4e35-930b-e6fa2ad8007f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh"
Jul 28 22:47:39.283106 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.283107    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ed0c421-3107-4a5f-a440-242ba2eff366-plugins-config-volume\") pod \"payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6\" (UID: \"1ed0c421-3107-4a5f-a440-242ba2eff366\") " pod="openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6"
Jul 28 22:47:39.283346 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.283189    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcqg4\" (UniqueName: \"kubernetes.io/projected/58cb9eb4-0c57-4e35-930b-e6fa2ad8007f-kube-api-access-hcqg4\") pod \"payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh\" (UID: \"58cb9eb4-0c57-4e35-930b-e6fa2ad8007f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh"
Jul 28 22:47:39.283346 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.283205    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffrbw\" (UniqueName: \"kubernetes.io/projected/1ed0c421-3107-4a5f-a440-242ba2eff366-kube-api-access-ffrbw\") pod \"payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6\" (UID: \"1ed0c421-3107-4a5f-a440-242ba2eff366\") " pod="openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6"
Jul 28 22:47:39.384597 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.384496    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58cb9eb4-0c57-4e35-930b-e6fa2ad8007f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh\" (UID: \"58cb9eb4-0c57-4e35-930b-e6fa2ad8007f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh"
Jul 28 22:47:39.384597 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.384556    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ed0c421-3107-4a5f-a440-242ba2eff366-plugins-config-volume\") pod \"payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6\" (UID: \"1ed0c421-3107-4a5f-a440-242ba2eff366\") " pod="openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6"
Jul 28 22:47:39.384829 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.384669    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcqg4\" (UniqueName: \"kubernetes.io/projected/58cb9eb4-0c57-4e35-930b-e6fa2ad8007f-kube-api-access-hcqg4\") pod \"payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh\" (UID: \"58cb9eb4-0c57-4e35-930b-e6fa2ad8007f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh"
Jul 28 22:47:39.384829 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.384691    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffrbw\" (UniqueName: \"kubernetes.io/projected/1ed0c421-3107-4a5f-a440-242ba2eff366-kube-api-access-ffrbw\") pod \"payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6\" (UID: \"1ed0c421-3107-4a5f-a440-242ba2eff366\") " pod="openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6"
Jul 28 22:47:39.385198 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.385179    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58cb9eb4-0c57-4e35-930b-e6fa2ad8007f-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh\" (UID: \"58cb9eb4-0c57-4e35-930b-e6fa2ad8007f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh"
Jul 28 22:47:39.385356 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.385326    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ed0c421-3107-4a5f-a440-242ba2eff366-plugins-config-volume\") pod \"payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6\" (UID: \"1ed0c421-3107-4a5f-a440-242ba2eff366\") " pod="openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6"
Jul 28 22:47:39.392068 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.392041    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcqg4\" (UniqueName: \"kubernetes.io/projected/58cb9eb4-0c57-4e35-930b-e6fa2ad8007f-kube-api-access-hcqg4\") pod \"payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh\" (UID: \"58cb9eb4-0c57-4e35-930b-e6fa2ad8007f\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh"
Jul 28 22:47:39.392365 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.392349    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffrbw\" (UniqueName: \"kubernetes.io/projected/1ed0c421-3107-4a5f-a440-242ba2eff366-kube-api-access-ffrbw\") pod \"payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6\" (UID: \"1ed0c421-3107-4a5f-a440-242ba2eff366\") " pod="openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6"
Jul 28 22:47:39.433800 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.433768    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh"
Jul 28 22:47:39.456884 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.456852    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6"
Jul 28 22:47:39.578323 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.578272    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh"]
Jul 28 22:47:39.582101 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:47:39.582066    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58cb9eb4_0c57_4e35_930b_e6fa2ad8007f.slice/crio-d1a8f80aa6bd61aaa2ae6a72e155c1f590ad4c542b356da3bc2c8cb2e4137766 WatchSource:0}: Error finding container d1a8f80aa6bd61aaa2ae6a72e155c1f590ad4c542b356da3bc2c8cb2e4137766: Status 404 returned error can't find the container with id d1a8f80aa6bd61aaa2ae6a72e155c1f590ad4c542b356da3bc2c8cb2e4137766
Jul 28 22:47:39.604431 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.604409    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6"]
Jul 28 22:47:39.613910 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:47:39.613884    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ed0c421_3107_4a5f_a440_242ba2eff366.slice/crio-0f50a6b5d0e05d0a481b72c294b414180daca144685f7014eef858cae448a760 WatchSource:0}: Error finding container 0f50a6b5d0e05d0a481b72c294b414180daca144685f7014eef858cae448a760: Status 404 returned error can't find the container with id 0f50a6b5d0e05d0a481b72c294b414180daca144685f7014eef858cae448a760
Jul 28 22:47:39.830827 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.830789    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6" event={"ID":"1ed0c421-3107-4a5f-a440-242ba2eff366","Type":"ContainerStarted","Data":"902a516afc1838350a41599d6a8ad46695e105f71b3f0a83b7e6b7208012a680"}
Jul 28 22:47:39.831010 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.830855    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6" event={"ID":"1ed0c421-3107-4a5f-a440-242ba2eff366","Type":"ContainerStarted","Data":"0f50a6b5d0e05d0a481b72c294b414180daca144685f7014eef858cae448a760"}
Jul 28 22:47:39.831010 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.830965    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6"
Jul 28 22:47:39.832376 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.832353    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh" event={"ID":"58cb9eb4-0c57-4e35-930b-e6fa2ad8007f","Type":"ContainerStarted","Data":"c1aa23f250041ba1ab461879a92293805601fb4e71bb3dfaa499fff62aab3fd8"}
Jul 28 22:47:39.832477 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.832389    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh" event={"ID":"58cb9eb4-0c57-4e35-930b-e6fa2ad8007f","Type":"ContainerStarted","Data":"d1a8f80aa6bd61aaa2ae6a72e155c1f590ad4c542b356da3bc2c8cb2e4137766"}
Jul 28 22:47:39.832477 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.832409    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh"
Jul 28 22:47:39.847499 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.847457    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6" podStartSLOduration=0.847435565 podStartE2EDuration="847.435565ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:47:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:47:39.846640523 +0000 UTC m=+2267.864606217" watchObservedRunningTime="2026-07-28 22:47:39.847435565 +0000 UTC m=+2267.865401259"
Jul 28 22:47:39.861302 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:39.861259    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh" podStartSLOduration=0.861244981 podStartE2EDuration="861.244981ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:47:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:47:39.860661812 +0000 UTC m=+2267.878627497" watchObservedRunningTime="2026-07-28 22:47:39.861244981 +0000 UTC m=+2267.879210699"
Jul 28 22:47:43.671379 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:43.671341    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:47:50.839229 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:50.839201    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh"
Jul 28 22:47:50.839616 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:47:50.839268    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6"
Jul 28 22:48:08.319190 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:08.319147    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7"]
Jul 28 22:48:08.322755 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:08.322732    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7"
Jul 28 22:48:08.324935 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:08.324913    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-472fa3-dockercfg-rhgmk\""
Jul 28 22:48:08.325067 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:08.324914    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-472fa3\""
Jul 28 22:48:08.334783 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:08.334761    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7"]
Jul 28 22:48:08.433719 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:08.433684    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/784a5bf8-a4d2-4520-a981-29a7c8181e58-plugins-config-volume\") pod \"payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7\" (UID: \"784a5bf8-a4d2-4520-a981-29a7c8181e58\") " pod="openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7"
Jul 28 22:48:08.433903 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:08.433741    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx9lr\" (UniqueName: \"kubernetes.io/projected/784a5bf8-a4d2-4520-a981-29a7c8181e58-kube-api-access-rx9lr\") pod \"payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7\" (UID: \"784a5bf8-a4d2-4520-a981-29a7c8181e58\") " pod="openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7"
Jul 28 22:48:08.534620 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:08.534585    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/784a5bf8-a4d2-4520-a981-29a7c8181e58-plugins-config-volume\") pod \"payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7\" (UID: \"784a5bf8-a4d2-4520-a981-29a7c8181e58\") " pod="openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7"
Jul 28 22:48:08.534808 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:08.534649    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx9lr\" (UniqueName: \"kubernetes.io/projected/784a5bf8-a4d2-4520-a981-29a7c8181e58-kube-api-access-rx9lr\") pod \"payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7\" (UID: \"784a5bf8-a4d2-4520-a981-29a7c8181e58\") " pod="openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7"
Jul 28 22:48:08.535404 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:08.535385    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/784a5bf8-a4d2-4520-a981-29a7c8181e58-plugins-config-volume\") pod \"payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7\" (UID: \"784a5bf8-a4d2-4520-a981-29a7c8181e58\") " pod="openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7"
Jul 28 22:48:08.541750 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:08.541727    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx9lr\" (UniqueName: \"kubernetes.io/projected/784a5bf8-a4d2-4520-a981-29a7c8181e58-kube-api-access-rx9lr\") pod \"payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7\" (UID: \"784a5bf8-a4d2-4520-a981-29a7c8181e58\") " pod="openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7"
Jul 28 22:48:08.633154 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:08.633079    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7"
Jul 28 22:48:08.998606 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:08.998581    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7"]
Jul 28 22:48:09.000695 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:48:09.000666    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod784a5bf8_a4d2_4520_a981_29a7c8181e58.slice/crio-9673b81bb8ee65c9a75bf091806d34d2578b8460bb953b0e2c3822f7510a2c07 WatchSource:0}: Error finding container 9673b81bb8ee65c9a75bf091806d34d2578b8460bb953b0e2c3822f7510a2c07: Status 404 returned error can't find the container with id 9673b81bb8ee65c9a75bf091806d34d2578b8460bb953b0e2c3822f7510a2c07
Jul 28 22:48:09.978742 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:09.978707    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7" event={"ID":"784a5bf8-a4d2-4520-a981-29a7c8181e58","Type":"ContainerStarted","Data":"3db976fd7bc570ace5f306155f6aca78abcb507fda3ab8de9dfa486d6b787b78"}
Jul 28 22:48:09.979143 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:09.978757    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7" event={"ID":"784a5bf8-a4d2-4520-a981-29a7c8181e58","Type":"ContainerStarted","Data":"9673b81bb8ee65c9a75bf091806d34d2578b8460bb953b0e2c3822f7510a2c07"}
Jul 28 22:48:09.979143 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:09.978790    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7"
Jul 28 22:48:09.993832 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:09.993786    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7" podStartSLOduration=1.993769538 podStartE2EDuration="1.993769538s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:48:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:48:09.992425944 +0000 UTC m=+2298.010391639" watchObservedRunningTime="2026-07-28 22:48:09.993769538 +0000 UTC m=+2298.011735259"
Jul 28 22:48:14.470865 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:14.470832    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:48:20.985662 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:20.985625    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7"
Jul 28 22:48:46.082714 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:46.082628    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6"]
Jul 28 22:48:46.083236 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:46.082945    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6" podUID="1ed0c421-3107-4a5f-a440-242ba2eff366" containerName="payload-processing" containerID="cri-o://902a516afc1838350a41599d6a8ad46695e105f71b3f0a83b7e6b7208012a680" gracePeriod=30
Jul 28 22:48:46.089579 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:46.089512    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh"]
Jul 28 22:48:46.089898 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:46.089783    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh" podUID="58cb9eb4-0c57-4e35-930b-e6fa2ad8007f" containerName="payload-pre-processing" containerID="cri-o://c1aa23f250041ba1ab461879a92293805601fb4e71bb3dfaa499fff62aab3fd8" gracePeriod=30
Jul 28 22:48:46.335613 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:46.335556    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6"
Jul 28 22:48:46.368885 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:46.368864    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh"
Jul 28 22:48:46.371430 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:46.371407    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ed0c421-3107-4a5f-a440-242ba2eff366-plugins-config-volume\") pod \"1ed0c421-3107-4a5f-a440-242ba2eff366\" (UID: \"1ed0c421-3107-4a5f-a440-242ba2eff366\") "
Jul 28 22:48:46.371558 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:46.371543    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ffrbw\" (UniqueName: \"kubernetes.io/projected/1ed0c421-3107-4a5f-a440-242ba2eff366-kube-api-access-ffrbw\") pod \"1ed0c421-3107-4a5f-a440-242ba2eff366\" (UID: \"1ed0c421-3107-4a5f-a440-242ba2eff366\") "
Jul 28 22:48:46.371817 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:46.371796    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ed0c421-3107-4a5f-a440-242ba2eff366-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1ed0c421-3107-4a5f-a440-242ba2eff366" (UID: "1ed0c421-3107-4a5f-a440-242ba2eff366"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:48:46.373452 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:46.373431    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ed0c421-3107-4a5f-a440-242ba2eff366-kube-api-access-ffrbw" (OuterVolumeSpecName: "kube-api-access-ffrbw") pod "1ed0c421-3107-4a5f-a440-242ba2eff366" (UID: "1ed0c421-3107-4a5f-a440-242ba2eff366"). InnerVolumeSpecName "kube-api-access-ffrbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:48:46.472246 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:46.472060    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58cb9eb4-0c57-4e35-930b-e6fa2ad8007f-plugins-config-volume\") pod \"58cb9eb4-0c57-4e35-930b-e6fa2ad8007f\" (UID: \"58cb9eb4-0c57-4e35-930b-e6fa2ad8007f\") "
Jul 28 22:48:46.472246 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:46.472099    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hcqg4\" (UniqueName: \"kubernetes.io/projected/58cb9eb4-0c57-4e35-930b-e6fa2ad8007f-kube-api-access-hcqg4\") pod \"58cb9eb4-0c57-4e35-930b-e6fa2ad8007f\" (UID: \"58cb9eb4-0c57-4e35-930b-e6fa2ad8007f\") "
Jul 28 22:48:46.472476 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:46.472453    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffrbw\" (UniqueName: \"kubernetes.io/projected/1ed0c421-3107-4a5f-a440-242ba2eff366-kube-api-access-ffrbw\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:48:46.472627 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:46.472504    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1ed0c421-3107-4a5f-a440-242ba2eff366-plugins-config-volume\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:48:46.472627 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:46.472454    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58cb9eb4-0c57-4e35-930b-e6fa2ad8007f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "58cb9eb4-0c57-4e35-930b-e6fa2ad8007f" (UID: "58cb9eb4-0c57-4e35-930b-e6fa2ad8007f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:48:46.474226 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:46.474201    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58cb9eb4-0c57-4e35-930b-e6fa2ad8007f-kube-api-access-hcqg4" (OuterVolumeSpecName: "kube-api-access-hcqg4") pod "58cb9eb4-0c57-4e35-930b-e6fa2ad8007f" (UID: "58cb9eb4-0c57-4e35-930b-e6fa2ad8007f"). InnerVolumeSpecName "kube-api-access-hcqg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:48:46.573455 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:46.573417    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58cb9eb4-0c57-4e35-930b-e6fa2ad8007f-plugins-config-volume\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:48:46.573455 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:46.573446    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcqg4\" (UniqueName: \"kubernetes.io/projected/58cb9eb4-0c57-4e35-930b-e6fa2ad8007f-kube-api-access-hcqg4\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:48:47.153940 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:47.153904    2551 generic.go:356] "Generic (PLEG): container finished" podID="58cb9eb4-0c57-4e35-930b-e6fa2ad8007f" containerID="c1aa23f250041ba1ab461879a92293805601fb4e71bb3dfaa499fff62aab3fd8" exitCode=0
Jul 28 22:48:47.154398 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:47.153956    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh"
Jul 28 22:48:47.154398 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:47.153984    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh" event={"ID":"58cb9eb4-0c57-4e35-930b-e6fa2ad8007f","Type":"ContainerDied","Data":"c1aa23f250041ba1ab461879a92293805601fb4e71bb3dfaa499fff62aab3fd8"}
Jul 28 22:48:47.154398 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:47.154042    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh" event={"ID":"58cb9eb4-0c57-4e35-930b-e6fa2ad8007f","Type":"ContainerDied","Data":"d1a8f80aa6bd61aaa2ae6a72e155c1f590ad4c542b356da3bc2c8cb2e4137766"}
Jul 28 22:48:47.154398 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:47.154065    2551 scope.go:117] "RemoveContainer" containerID="c1aa23f250041ba1ab461879a92293805601fb4e71bb3dfaa499fff62aab3fd8"
Jul 28 22:48:47.155403 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:47.155289    2551 generic.go:356] "Generic (PLEG): container finished" podID="1ed0c421-3107-4a5f-a440-242ba2eff366" containerID="902a516afc1838350a41599d6a8ad46695e105f71b3f0a83b7e6b7208012a680" exitCode=0
Jul 28 22:48:47.155403 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:47.155337    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6" event={"ID":"1ed0c421-3107-4a5f-a440-242ba2eff366","Type":"ContainerDied","Data":"902a516afc1838350a41599d6a8ad46695e105f71b3f0a83b7e6b7208012a680"}
Jul 28 22:48:47.155403 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:47.155344    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6"
Jul 28 22:48:47.155403 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:47.155368    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6" event={"ID":"1ed0c421-3107-4a5f-a440-242ba2eff366","Type":"ContainerDied","Data":"0f50a6b5d0e05d0a481b72c294b414180daca144685f7014eef858cae448a760"}
Jul 28 22:48:47.164538 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:47.164510    2551 scope.go:117] "RemoveContainer" containerID="c1aa23f250041ba1ab461879a92293805601fb4e71bb3dfaa499fff62aab3fd8"
Jul 28 22:48:47.164802 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:48:47.164785    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1aa23f250041ba1ab461879a92293805601fb4e71bb3dfaa499fff62aab3fd8\": container with ID starting with c1aa23f250041ba1ab461879a92293805601fb4e71bb3dfaa499fff62aab3fd8 not found: ID does not exist" containerID="c1aa23f250041ba1ab461879a92293805601fb4e71bb3dfaa499fff62aab3fd8"
Jul 28 22:48:47.164854 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:47.164811    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1aa23f250041ba1ab461879a92293805601fb4e71bb3dfaa499fff62aab3fd8"} err="failed to get container status \"c1aa23f250041ba1ab461879a92293805601fb4e71bb3dfaa499fff62aab3fd8\": rpc error: code = NotFound desc = could not find container \"c1aa23f250041ba1ab461879a92293805601fb4e71bb3dfaa499fff62aab3fd8\": container with ID starting with c1aa23f250041ba1ab461879a92293805601fb4e71bb3dfaa499fff62aab3fd8 not found: ID does not exist"
Jul 28 22:48:47.164854 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:47.164827    2551 scope.go:117] "RemoveContainer" containerID="902a516afc1838350a41599d6a8ad46695e105f71b3f0a83b7e6b7208012a680"
Jul 28 22:48:47.172401 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:47.172386    2551 scope.go:117] "RemoveContainer" containerID="902a516afc1838350a41599d6a8ad46695e105f71b3f0a83b7e6b7208012a680"
Jul 28 22:48:47.172676 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:48:47.172658    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"902a516afc1838350a41599d6a8ad46695e105f71b3f0a83b7e6b7208012a680\": container with ID starting with 902a516afc1838350a41599d6a8ad46695e105f71b3f0a83b7e6b7208012a680 not found: ID does not exist" containerID="902a516afc1838350a41599d6a8ad46695e105f71b3f0a83b7e6b7208012a680"
Jul 28 22:48:47.172733 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:47.172681    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"902a516afc1838350a41599d6a8ad46695e105f71b3f0a83b7e6b7208012a680"} err="failed to get container status \"902a516afc1838350a41599d6a8ad46695e105f71b3f0a83b7e6b7208012a680\": rpc error: code = NotFound desc = could not find container \"902a516afc1838350a41599d6a8ad46695e105f71b3f0a83b7e6b7208012a680\": container with ID starting with 902a516afc1838350a41599d6a8ad46695e105f71b3f0a83b7e6b7208012a680 not found: ID does not exist"
Jul 28 22:48:47.173001 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:47.172985    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6"]
Jul 28 22:48:47.177316 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:47.177294    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-63f77a-59b4f49496-pbmn6"]
Jul 28 22:48:47.187483 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:47.187459    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh"]
Jul 28 22:48:47.195362 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:47.195340    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-63f77a-6ff94fcb6d-697wh"]
Jul 28 22:48:48.463135 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:48.463102    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ed0c421-3107-4a5f-a440-242ba2eff366" path="/var/lib/kubelet/pods/1ed0c421-3107-4a5f-a440-242ba2eff366/volumes"
Jul 28 22:48:48.463485 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:48.463453    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58cb9eb4-0c57-4e35-930b-e6fa2ad8007f" path="/var/lib/kubelet/pods/58cb9eb4-0c57-4e35-930b-e6fa2ad8007f/volumes"
Jul 28 22:48:49.072397 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:48:49.072364    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:49:04.328128 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:04.328098    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7"]
Jul 28 22:49:04.328564 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:04.328343    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7" podUID="784a5bf8-a4d2-4520-a981-29a7c8181e58" containerName="payload-processing" containerID="cri-o://3db976fd7bc570ace5f306155f6aca78abcb507fda3ab8de9dfa486d6b787b78" gracePeriod=30
Jul 28 22:49:04.580632 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:04.580573    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7"
Jul 28 22:49:04.729271 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:04.729239    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/784a5bf8-a4d2-4520-a981-29a7c8181e58-plugins-config-volume\") pod \"784a5bf8-a4d2-4520-a981-29a7c8181e58\" (UID: \"784a5bf8-a4d2-4520-a981-29a7c8181e58\") "
Jul 28 22:49:04.729455 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:04.729332    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rx9lr\" (UniqueName: \"kubernetes.io/projected/784a5bf8-a4d2-4520-a981-29a7c8181e58-kube-api-access-rx9lr\") pod \"784a5bf8-a4d2-4520-a981-29a7c8181e58\" (UID: \"784a5bf8-a4d2-4520-a981-29a7c8181e58\") "
Jul 28 22:49:04.729673 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:04.729645    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/784a5bf8-a4d2-4520-a981-29a7c8181e58-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "784a5bf8-a4d2-4520-a981-29a7c8181e58" (UID: "784a5bf8-a4d2-4520-a981-29a7c8181e58"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 22:49:04.731398 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:04.731372    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/784a5bf8-a4d2-4520-a981-29a7c8181e58-kube-api-access-rx9lr" (OuterVolumeSpecName: "kube-api-access-rx9lr") pod "784a5bf8-a4d2-4520-a981-29a7c8181e58" (UID: "784a5bf8-a4d2-4520-a981-29a7c8181e58"). InnerVolumeSpecName "kube-api-access-rx9lr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:49:04.830225 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:04.830186    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/784a5bf8-a4d2-4520-a981-29a7c8181e58-plugins-config-volume\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:49:04.830225 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:04.830218    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rx9lr\" (UniqueName: \"kubernetes.io/projected/784a5bf8-a4d2-4520-a981-29a7c8181e58-kube-api-access-rx9lr\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:49:05.239672 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:05.239636    2551 generic.go:356] "Generic (PLEG): container finished" podID="784a5bf8-a4d2-4520-a981-29a7c8181e58" containerID="3db976fd7bc570ace5f306155f6aca78abcb507fda3ab8de9dfa486d6b787b78" exitCode=0
Jul 28 22:49:05.239819 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:05.239685    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7"
Jul 28 22:49:05.239819 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:05.239729    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7" event={"ID":"784a5bf8-a4d2-4520-a981-29a7c8181e58","Type":"ContainerDied","Data":"3db976fd7bc570ace5f306155f6aca78abcb507fda3ab8de9dfa486d6b787b78"}
Jul 28 22:49:05.239819 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:05.239786    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7" event={"ID":"784a5bf8-a4d2-4520-a981-29a7c8181e58","Type":"ContainerDied","Data":"9673b81bb8ee65c9a75bf091806d34d2578b8460bb953b0e2c3822f7510a2c07"}
Jul 28 22:49:05.239819 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:05.239813    2551 scope.go:117] "RemoveContainer" containerID="3db976fd7bc570ace5f306155f6aca78abcb507fda3ab8de9dfa486d6b787b78"
Jul 28 22:49:05.248231 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:05.248212    2551 scope.go:117] "RemoveContainer" containerID="3db976fd7bc570ace5f306155f6aca78abcb507fda3ab8de9dfa486d6b787b78"
Jul 28 22:49:05.248504 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:49:05.248485    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3db976fd7bc570ace5f306155f6aca78abcb507fda3ab8de9dfa486d6b787b78\": container with ID starting with 3db976fd7bc570ace5f306155f6aca78abcb507fda3ab8de9dfa486d6b787b78 not found: ID does not exist" containerID="3db976fd7bc570ace5f306155f6aca78abcb507fda3ab8de9dfa486d6b787b78"
Jul 28 22:49:05.248587 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:05.248513    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3db976fd7bc570ace5f306155f6aca78abcb507fda3ab8de9dfa486d6b787b78"} err="failed to get container status \"3db976fd7bc570ace5f306155f6aca78abcb507fda3ab8de9dfa486d6b787b78\": rpc error: code = NotFound desc = could not find container \"3db976fd7bc570ace5f306155f6aca78abcb507fda3ab8de9dfa486d6b787b78\": container with ID starting with 3db976fd7bc570ace5f306155f6aca78abcb507fda3ab8de9dfa486d6b787b78 not found: ID does not exist"
Jul 28 22:49:05.259628 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:05.259602    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7"]
Jul 28 22:49:05.264882 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:05.264858    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-472fa3-54bb48b94d-dxbp7"]
Jul 28 22:49:06.367958 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:06.367926    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:49:06.462386 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:06.462353    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="784a5bf8-a4d2-4520-a981-29a7c8181e58" path="/var/lib/kubelet/pods/784a5bf8-a4d2-4520-a981-29a7c8181e58/volumes"
Jul 28 22:49:15.720407 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:15.720372    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm"]
Jul 28 22:49:15.720910 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:15.720723    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58cb9eb4-0c57-4e35-930b-e6fa2ad8007f" containerName="payload-pre-processing"
Jul 28 22:49:15.720910 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:15.720735    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="58cb9eb4-0c57-4e35-930b-e6fa2ad8007f" containerName="payload-pre-processing"
Jul 28 22:49:15.720910 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:15.720753    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="784a5bf8-a4d2-4520-a981-29a7c8181e58" containerName="payload-processing"
Jul 28 22:49:15.720910 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:15.720760    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="784a5bf8-a4d2-4520-a981-29a7c8181e58" containerName="payload-processing"
Jul 28 22:49:15.720910 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:15.720773    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ed0c421-3107-4a5f-a440-242ba2eff366" containerName="payload-processing"
Jul 28 22:49:15.720910 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:15.720778    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ed0c421-3107-4a5f-a440-242ba2eff366" containerName="payload-processing"
Jul 28 22:49:15.720910 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:15.720827    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="58cb9eb4-0c57-4e35-930b-e6fa2ad8007f" containerName="payload-pre-processing"
Jul 28 22:49:15.720910 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:15.720834    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ed0c421-3107-4a5f-a440-242ba2eff366" containerName="payload-processing"
Jul 28 22:49:15.720910 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:15.720842    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="784a5bf8-a4d2-4520-a981-29a7c8181e58" containerName="payload-processing"
Jul 28 22:49:15.724975 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:15.724954    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm"
Jul 28 22:49:15.726934 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:15.726912    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-a134df-dockercfg-l8vgj\""
Jul 28 22:49:15.727040 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:15.726976    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-a134df\""
Jul 28 22:49:15.730908 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:15.730887    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm"]
Jul 28 22:49:15.826647 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:15.826612    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3113fbc7-9efd-414f-9539-e603455f0de6-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm\" (UID: \"3113fbc7-9efd-414f-9539-e603455f0de6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm"
Jul 28 22:49:15.826810 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:15.826676    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzdn2\" (UniqueName: \"kubernetes.io/projected/3113fbc7-9efd-414f-9539-e603455f0de6-kube-api-access-vzdn2\") pod \"payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm\" (UID: \"3113fbc7-9efd-414f-9539-e603455f0de6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm"
Jul 28 22:49:15.928124 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:15.928087    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3113fbc7-9efd-414f-9539-e603455f0de6-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm\" (UID: \"3113fbc7-9efd-414f-9539-e603455f0de6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm"
Jul 28 22:49:15.928322 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:15.928159    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzdn2\" (UniqueName: \"kubernetes.io/projected/3113fbc7-9efd-414f-9539-e603455f0de6-kube-api-access-vzdn2\") pod \"payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm\" (UID: \"3113fbc7-9efd-414f-9539-e603455f0de6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm"
Jul 28 22:49:15.928844 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:15.928815    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3113fbc7-9efd-414f-9539-e603455f0de6-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm\" (UID: \"3113fbc7-9efd-414f-9539-e603455f0de6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm"
Jul 28 22:49:15.935353 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:15.935322    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzdn2\" (UniqueName: \"kubernetes.io/projected/3113fbc7-9efd-414f-9539-e603455f0de6-kube-api-access-vzdn2\") pod \"payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm\" (UID: \"3113fbc7-9efd-414f-9539-e603455f0de6\") " pod="openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm"
Jul 28 22:49:16.036578 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:16.036473    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm"
Jul 28 22:49:16.176594 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:16.176566    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm"]
Jul 28 22:49:16.179887 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:49:16.179858    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3113fbc7_9efd_414f_9539_e603455f0de6.slice/crio-800351bc553e6661f86dc52fadd645d187cdaf3b488f4e16c06b21d3d4f84b75 WatchSource:0}: Error finding container 800351bc553e6661f86dc52fadd645d187cdaf3b488f4e16c06b21d3d4f84b75: Status 404 returned error can't find the container with id 800351bc553e6661f86dc52fadd645d187cdaf3b488f4e16c06b21d3d4f84b75
Jul 28 22:49:16.300322 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:16.300224    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm" event={"ID":"3113fbc7-9efd-414f-9539-e603455f0de6","Type":"ContainerStarted","Data":"97d95211640278a5c9733b68475ce1f2656a4c84cce27e0f628fd6df2e57d268"}
Jul 28 22:49:16.300322 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:16.300316    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm" event={"ID":"3113fbc7-9efd-414f-9539-e603455f0de6","Type":"ContainerStarted","Data":"800351bc553e6661f86dc52fadd645d187cdaf3b488f4e16c06b21d3d4f84b75"}
Jul 28 22:49:16.300492 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:16.300342    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm"
Jul 28 22:49:16.315875 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:16.315824    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm" podStartSLOduration=1.315808064 podStartE2EDuration="1.315808064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:49:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:49:16.314340649 +0000 UTC m=+2364.332306335" watchObservedRunningTime="2026-07-28 22:49:16.315808064 +0000 UTC m=+2364.333773757"
Jul 28 22:49:19.373474 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:19.373439    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:49:27.308505 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:27.308472    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm"
Jul 28 22:49:43.834574 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:43.834536    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26"]
Jul 28 22:49:43.838205 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:43.838182    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26"
Jul 28 22:49:43.840087 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:43.840055    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-faf483\""
Jul 28 22:49:43.851848 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:43.851817    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26"]
Jul 28 22:49:43.860835 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:43.860809    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88"]
Jul 28 22:49:43.864345 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:43.864328    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88"
Jul 28 22:49:43.866161 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:43.866139    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-faf483-dockercfg-jk5gl\""
Jul 28 22:49:43.873769 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:43.873728    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88"]
Jul 28 22:49:43.951070 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:43.951027    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/799a6bb7-2cc6-45b5-b019-f0a273f26460-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88\" (UID: \"799a6bb7-2cc6-45b5-b019-f0a273f26460\") " pod="openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88"
Jul 28 22:49:43.951070 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:43.951072    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmvld\" (UniqueName: \"kubernetes.io/projected/799a6bb7-2cc6-45b5-b019-f0a273f26460-kube-api-access-zmvld\") pod \"payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88\" (UID: \"799a6bb7-2cc6-45b5-b019-f0a273f26460\") " pod="openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88"
Jul 28 22:49:43.951311 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:43.951164    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxxj6\" (UniqueName: \"kubernetes.io/projected/6b110cee-435d-470f-bc40-134d389b9b6b-kube-api-access-kxxj6\") pod \"payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26\" (UID: \"6b110cee-435d-470f-bc40-134d389b9b6b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26"
Jul 28 22:49:43.951311 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:43.951214    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b110cee-435d-470f-bc40-134d389b9b6b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26\" (UID: \"6b110cee-435d-470f-bc40-134d389b9b6b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26"
Jul 28 22:49:44.052458 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:44.052426    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/799a6bb7-2cc6-45b5-b019-f0a273f26460-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88\" (UID: \"799a6bb7-2cc6-45b5-b019-f0a273f26460\") " pod="openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88"
Jul 28 22:49:44.052458 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:44.052462    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmvld\" (UniqueName: \"kubernetes.io/projected/799a6bb7-2cc6-45b5-b019-f0a273f26460-kube-api-access-zmvld\") pod \"payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88\" (UID: \"799a6bb7-2cc6-45b5-b019-f0a273f26460\") " pod="openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88"
Jul 28 22:49:44.052751 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:44.052482    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxxj6\" (UniqueName: \"kubernetes.io/projected/6b110cee-435d-470f-bc40-134d389b9b6b-kube-api-access-kxxj6\") pod \"payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26\" (UID: \"6b110cee-435d-470f-bc40-134d389b9b6b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26"
Jul 28 22:49:44.052751 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:44.052499    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b110cee-435d-470f-bc40-134d389b9b6b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26\" (UID: \"6b110cee-435d-470f-bc40-134d389b9b6b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26"
Jul 28 22:49:44.053206 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:44.053181    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b110cee-435d-470f-bc40-134d389b9b6b-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26\" (UID: \"6b110cee-435d-470f-bc40-134d389b9b6b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26"
Jul 28 22:49:44.053295 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:44.053219    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/799a6bb7-2cc6-45b5-b019-f0a273f26460-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88\" (UID: \"799a6bb7-2cc6-45b5-b019-f0a273f26460\") " pod="openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88"
Jul 28 22:49:44.059803 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:44.059781    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxxj6\" (UniqueName: \"kubernetes.io/projected/6b110cee-435d-470f-bc40-134d389b9b6b-kube-api-access-kxxj6\") pod \"payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26\" (UID: \"6b110cee-435d-470f-bc40-134d389b9b6b\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26"
Jul 28 22:49:44.059949 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:44.059929    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmvld\" (UniqueName: \"kubernetes.io/projected/799a6bb7-2cc6-45b5-b019-f0a273f26460-kube-api-access-zmvld\") pod \"payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88\" (UID: \"799a6bb7-2cc6-45b5-b019-f0a273f26460\") " pod="openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88"
Jul 28 22:49:44.153849 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:44.153759    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26"
Jul 28 22:49:44.174989 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:44.174959    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88"
Jul 28 22:49:44.502297 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:44.502271    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26"]
Jul 28 22:49:44.504583 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:49:44.504553    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b110cee_435d_470f_bc40_134d389b9b6b.slice/crio-797dab7d1415455e751fe36196d31d7afb0e29e71f54767fb7b750d6fa583a85 WatchSource:0}: Error finding container 797dab7d1415455e751fe36196d31d7afb0e29e71f54767fb7b750d6fa583a85: Status 404 returned error can't find the container with id 797dab7d1415455e751fe36196d31d7afb0e29e71f54767fb7b750d6fa583a85
Jul 28 22:49:44.529929 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:44.529907    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88"]
Jul 28 22:49:44.535675 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:49:44.535649    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod799a6bb7_2cc6_45b5_b019_f0a273f26460.slice/crio-20071d41a6fad82bdf38ae52493ebf3e9585fbf655d6eaabf9a63d1e9bb42912 WatchSource:0}: Error finding container 20071d41a6fad82bdf38ae52493ebf3e9585fbf655d6eaabf9a63d1e9bb42912: Status 404 returned error can't find the container with id 20071d41a6fad82bdf38ae52493ebf3e9585fbf655d6eaabf9a63d1e9bb42912
Jul 28 22:49:45.440280 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:45.440245    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88" event={"ID":"799a6bb7-2cc6-45b5-b019-f0a273f26460","Type":"ContainerStarted","Data":"27689bdcd7abb2cdafaca5c534579560ac201993de3e2ff5ab235ea0ddbd8204"}
Jul 28 22:49:45.440770 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:45.440423    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88"
Jul 28 22:49:45.440770 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:45.440478    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88" event={"ID":"799a6bb7-2cc6-45b5-b019-f0a273f26460","Type":"ContainerStarted","Data":"20071d41a6fad82bdf38ae52493ebf3e9585fbf655d6eaabf9a63d1e9bb42912"}
Jul 28 22:49:45.441778 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:45.441757    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26" event={"ID":"6b110cee-435d-470f-bc40-134d389b9b6b","Type":"ContainerStarted","Data":"55c0916d7aefd56fefc0c686519fa5556cdf48064ddc52649110482652206d26"}
Jul 28 22:49:45.441899 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:45.441800    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26" event={"ID":"6b110cee-435d-470f-bc40-134d389b9b6b","Type":"ContainerStarted","Data":"797dab7d1415455e751fe36196d31d7afb0e29e71f54767fb7b750d6fa583a85"}
Jul 28 22:49:45.441899 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:45.441827    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26"
Jul 28 22:49:45.454079 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:45.454021    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88" podStartSLOduration=2.454003615 podStartE2EDuration="2.454003615s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:49:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:49:45.45385865 +0000 UTC m=+2393.471824345" watchObservedRunningTime="2026-07-28 22:49:45.454003615 +0000 UTC m=+2393.471969321"
Jul 28 22:49:45.467899 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:45.467856    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26" podStartSLOduration=2.467841374 podStartE2EDuration="2.467841374s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:49:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:49:45.466172937 +0000 UTC m=+2393.484138642" watchObservedRunningTime="2026-07-28 22:49:45.467841374 +0000 UTC m=+2393.485807068"
Jul 28 22:49:50.768079 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:50.768039    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:49:55.062832 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:55.062735    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:49:55.069099 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:55.066125    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:49:56.448005 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:56.447975    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26"
Jul 28 22:49:56.448445 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:49:56.448052    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88"
Jul 28 22:50:25.965138 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:50:25.965095    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:50:49.766015 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:50:49.765981    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:51:14.662723 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:51:14.662690    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:51:35.365251 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:51:35.365210    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:51:53.559598 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:51:53.559502    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:52:15.596359 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:52:15.596320    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:52:32.072513 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:52:32.072477    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:52:54.067502 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:52:54.067468    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:53:11.964701 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:53:11.964665    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:53:33.860183 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:53:33.860146    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:53:52.762542 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:53:52.762498    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:54:14.565884 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:54:14.565849    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:54:32.370655 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:54:32.370619    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:54:55.087942 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:54:55.087913    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:54:55.093806 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:54:55.093785    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:54:59.960734 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:54:59.960698    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:55:14.662851 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.662817    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"]
Jul 28 22:55:14.666806 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.666785    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:14.668768 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.668738    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a134df\"/\"kube-root-ca.crt\""
Jul 28 22:55:14.669543 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.669500    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a134df\"/\"openshift-service-ca.crt\""
Jul 28 22:55:14.669644 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.669509    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a134df\"/\"default-dockercfg-7v695\""
Jul 28 22:55:14.669707 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.669652    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a134df\"/\"sub-test-model-a134df-kserve-self-signed-certs\""
Jul 28 22:55:14.678393 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.678006    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"]
Jul 28 22:55:14.751764 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.751726    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-dshm\") pod \"sub-test-model-a134df-kserve-5d576cdf8f-4jgvk\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") " pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:14.751939 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.751777    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-tmp-dir\") pod \"sub-test-model-a134df-kserve-5d576cdf8f-4jgvk\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") " pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:14.751939 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.751802    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-model-cache\") pod \"sub-test-model-a134df-kserve-5d576cdf8f-4jgvk\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") " pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:14.751939 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.751831    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65b85c2c-de0a-4e23-b849-6b5a8f420cde-tls-certs\") pod \"sub-test-model-a134df-kserve-5d576cdf8f-4jgvk\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") " pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:14.751939 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.751861    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-home\") pod \"sub-test-model-a134df-kserve-5d576cdf8f-4jgvk\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") " pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:14.751939 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.751878    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w87rz\" (UniqueName: \"kubernetes.io/projected/65b85c2c-de0a-4e23-b849-6b5a8f420cde-kube-api-access-w87rz\") pod \"sub-test-model-a134df-kserve-5d576cdf8f-4jgvk\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") " pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:14.852828 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.852788    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-dshm\") pod \"sub-test-model-a134df-kserve-5d576cdf8f-4jgvk\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") " pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:14.853002 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.852840    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-tmp-dir\") pod \"sub-test-model-a134df-kserve-5d576cdf8f-4jgvk\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") " pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:14.853002 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.852966    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-model-cache\") pod \"sub-test-model-a134df-kserve-5d576cdf8f-4jgvk\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") " pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:14.853115 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.853017    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65b85c2c-de0a-4e23-b849-6b5a8f420cde-tls-certs\") pod \"sub-test-model-a134df-kserve-5d576cdf8f-4jgvk\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") " pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:14.853174 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.853129    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-home\") pod \"sub-test-model-a134df-kserve-5d576cdf8f-4jgvk\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") " pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:14.853174 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.853142    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-tmp-dir\") pod \"sub-test-model-a134df-kserve-5d576cdf8f-4jgvk\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") " pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:14.853289 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.853176    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w87rz\" (UniqueName: \"kubernetes.io/projected/65b85c2c-de0a-4e23-b849-6b5a8f420cde-kube-api-access-w87rz\") pod \"sub-test-model-a134df-kserve-5d576cdf8f-4jgvk\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") " pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:14.853383 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.853356    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-model-cache\") pod \"sub-test-model-a134df-kserve-5d576cdf8f-4jgvk\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") " pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:14.853454 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.853435    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-home\") pod \"sub-test-model-a134df-kserve-5d576cdf8f-4jgvk\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") " pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:14.855145 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.855115    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-dshm\") pod \"sub-test-model-a134df-kserve-5d576cdf8f-4jgvk\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") " pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:14.855364 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.855346    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65b85c2c-de0a-4e23-b849-6b5a8f420cde-tls-certs\") pod \"sub-test-model-a134df-kserve-5d576cdf8f-4jgvk\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") " pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:14.860863 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.860840    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w87rz\" (UniqueName: \"kubernetes.io/projected/65b85c2c-de0a-4e23-b849-6b5a8f420cde-kube-api-access-w87rz\") pod \"sub-test-model-a134df-kserve-5d576cdf8f-4jgvk\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") " pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:14.982316 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:14.982223    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:15.117269 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:15.117241    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"]
Jul 28 22:55:15.120815 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:55:15.120783    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod65b85c2c_de0a_4e23_b849_6b5a8f420cde.slice/crio-6d2538d390be372b02888836de3209801f4b7f20bfa1069f40b24cb9b8670525 WatchSource:0}: Error finding container 6d2538d390be372b02888836de3209801f4b7f20bfa1069f40b24cb9b8670525: Status 404 returned error can't find the container with id 6d2538d390be372b02888836de3209801f4b7f20bfa1069f40b24cb9b8670525
Jul 28 22:55:15.768508 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:15.768475    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:55:16.002482 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:16.002442    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk" event={"ID":"65b85c2c-de0a-4e23-b849-6b5a8f420cde","Type":"ContainerStarted","Data":"b0a80a75a3a9c220099a62605f46762498eff6dd4a975aa0aa83080526428bfd"}
Jul 28 22:55:16.002482 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:16.002485    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk" event={"ID":"65b85c2c-de0a-4e23-b849-6b5a8f420cde","Type":"ContainerStarted","Data":"6d2538d390be372b02888836de3209801f4b7f20bfa1069f40b24cb9b8670525"}
Jul 28 22:55:16.017766 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:16.017665    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk" podStartSLOduration=2.017646741 podStartE2EDuration="2.017646741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:55:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:55:16.017300836 +0000 UTC m=+2724.035266531" watchObservedRunningTime="2026-07-28 22:55:16.017646741 +0000 UTC m=+2724.035612438"
Jul 28 22:55:24.983316 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:24.983277    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:24.983731 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:24.983361    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:24.995726 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:24.995699    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:25.056855 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:25.056821    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:55:25.663830 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:55:25.663789    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:56:00.970877 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:56:00.970840    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:56:32.868801 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:56:32.868767    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:56:51.771399 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:56:51.771366    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:56:57.750000 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:56:57.749956    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"]
Jul 28 22:57:09.386812 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:09.386772    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:57:12.770254 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:12.770208    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk" podUID="65b85c2c-de0a-4e23-b849-6b5a8f420cde" containerName="main" containerID="cri-o://b0a80a75a3a9c220099a62605f46762498eff6dd4a975aa0aa83080526428bfd" gracePeriod=45
Jul 28 22:57:13.038062 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.038032    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:57:13.187829 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.187791    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w87rz\" (UniqueName: \"kubernetes.io/projected/65b85c2c-de0a-4e23-b849-6b5a8f420cde-kube-api-access-w87rz\") pod \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") "
Jul 28 22:57:13.187994 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.187864    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-home\") pod \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") "
Jul 28 22:57:13.187994 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.187885    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65b85c2c-de0a-4e23-b849-6b5a8f420cde-tls-certs\") pod \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") "
Jul 28 22:57:13.187994 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.187930    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-dshm\") pod \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") "
Jul 28 22:57:13.187994 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.187957    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-tmp-dir\") pod \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") "
Jul 28 22:57:13.188214 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.187997    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-model-cache\") pod \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\" (UID: \"65b85c2c-de0a-4e23-b849-6b5a8f420cde\") "
Jul 28 22:57:13.188214 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.188139    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-home" (OuterVolumeSpecName: "home") pod "65b85c2c-de0a-4e23-b849-6b5a8f420cde" (UID: "65b85c2c-de0a-4e23-b849-6b5a8f420cde"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:57:13.188320 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.188279    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "65b85c2c-de0a-4e23-b849-6b5a8f420cde" (UID: "65b85c2c-de0a-4e23-b849-6b5a8f420cde"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:57:13.188427 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.188403    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-model-cache" (OuterVolumeSpecName: "model-cache") pod "65b85c2c-de0a-4e23-b849-6b5a8f420cde" (UID: "65b85c2c-de0a-4e23-b849-6b5a8f420cde"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:57:13.190192 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.190164    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-dshm" (OuterVolumeSpecName: "dshm") pod "65b85c2c-de0a-4e23-b849-6b5a8f420cde" (UID: "65b85c2c-de0a-4e23-b849-6b5a8f420cde"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:57:13.190192 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.190163    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65b85c2c-de0a-4e23-b849-6b5a8f420cde-kube-api-access-w87rz" (OuterVolumeSpecName: "kube-api-access-w87rz") pod "65b85c2c-de0a-4e23-b849-6b5a8f420cde" (UID: "65b85c2c-de0a-4e23-b849-6b5a8f420cde"). InnerVolumeSpecName "kube-api-access-w87rz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:57:13.190344 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.190224    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65b85c2c-de0a-4e23-b849-6b5a8f420cde-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "65b85c2c-de0a-4e23-b849-6b5a8f420cde" (UID: "65b85c2c-de0a-4e23-b849-6b5a8f420cde"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:57:13.292441 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.289609    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-model-cache\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:57:13.292441 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.289659    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w87rz\" (UniqueName: \"kubernetes.io/projected/65b85c2c-de0a-4e23-b849-6b5a8f420cde-kube-api-access-w87rz\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:57:13.292441 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.289673    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-home\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:57:13.292441 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.289687    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65b85c2c-de0a-4e23-b849-6b5a8f420cde-tls-certs\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:57:13.292441 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.289701    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-dshm\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:57:13.292441 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.289718    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65b85c2c-de0a-4e23-b849-6b5a8f420cde-tmp-dir\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:57:13.561077 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.560981    2551 generic.go:356] "Generic (PLEG): container finished" podID="65b85c2c-de0a-4e23-b849-6b5a8f420cde" containerID="b0a80a75a3a9c220099a62605f46762498eff6dd4a975aa0aa83080526428bfd" exitCode=0
Jul 28 22:57:13.561077 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.561048    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"
Jul 28 22:57:13.561308 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.561053    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk" event={"ID":"65b85c2c-de0a-4e23-b849-6b5a8f420cde","Type":"ContainerDied","Data":"b0a80a75a3a9c220099a62605f46762498eff6dd4a975aa0aa83080526428bfd"}
Jul 28 22:57:13.561308 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.561160    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk" event={"ID":"65b85c2c-de0a-4e23-b849-6b5a8f420cde","Type":"ContainerDied","Data":"6d2538d390be372b02888836de3209801f4b7f20bfa1069f40b24cb9b8670525"}
Jul 28 22:57:13.561308 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.561176    2551 scope.go:117] "RemoveContainer" containerID="b0a80a75a3a9c220099a62605f46762498eff6dd4a975aa0aa83080526428bfd"
Jul 28 22:57:13.570067 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.570049    2551 scope.go:117] "RemoveContainer" containerID="b0a80a75a3a9c220099a62605f46762498eff6dd4a975aa0aa83080526428bfd"
Jul 28 22:57:13.570327 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:57:13.570310    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0a80a75a3a9c220099a62605f46762498eff6dd4a975aa0aa83080526428bfd\": container with ID starting with b0a80a75a3a9c220099a62605f46762498eff6dd4a975aa0aa83080526428bfd not found: ID does not exist" containerID="b0a80a75a3a9c220099a62605f46762498eff6dd4a975aa0aa83080526428bfd"
Jul 28 22:57:13.570376 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.570336    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0a80a75a3a9c220099a62605f46762498eff6dd4a975aa0aa83080526428bfd"} err="failed to get container status \"b0a80a75a3a9c220099a62605f46762498eff6dd4a975aa0aa83080526428bfd\": rpc error: code = NotFound desc = could not find container \"b0a80a75a3a9c220099a62605f46762498eff6dd4a975aa0aa83080526428bfd\": container with ID starting with b0a80a75a3a9c220099a62605f46762498eff6dd4a975aa0aa83080526428bfd not found: ID does not exist"
Jul 28 22:57:13.583735 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.583701    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"]
Jul 28 22:57:13.586548 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:13.586506    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a134df/sub-test-model-a134df-kserve-5d576cdf8f-4jgvk"]
Jul 28 22:57:14.463633 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:14.463601    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65b85c2c-de0a-4e23-b849-6b5a8f420cde" path="/var/lib/kubelet/pods/65b85c2c-de0a-4e23-b849-6b5a8f420cde/volumes"
Jul 28 22:57:38.075618 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.075583    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"]
Jul 28 22:57:38.076073 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.075966    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65b85c2c-de0a-4e23-b849-6b5a8f420cde" containerName="main"
Jul 28 22:57:38.076073 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.075977    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="65b85c2c-de0a-4e23-b849-6b5a8f420cde" containerName="main"
Jul 28 22:57:38.076073 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.076039    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="65b85c2c-de0a-4e23-b849-6b5a8f420cde" containerName="main"
Jul 28 22:57:38.079062 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.079040    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:57:38.082208 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.082176    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-faf483\"/\"default-dockercfg-fnzn6\""
Jul 28 22:57:38.082318 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.082203    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-faf483\"/\"openshift-service-ca.crt\""
Jul 28 22:57:38.082463 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.082447    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-faf483\"/\"kube-root-ca.crt\""
Jul 28 22:57:38.082579 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.082563    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-faf483\"/\"rate-test-model-faf483-kserve-self-signed-certs\""
Jul 28 22:57:38.087476 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.087443    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"]
Jul 28 22:57:38.101594 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.101566    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-home\") pod \"rate-test-model-faf483-kserve-54bf59f9d7-klb6n\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") " pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:57:38.101741 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.101624    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqf76\" (UniqueName: \"kubernetes.io/projected/267a4c4b-8b64-4b89-b692-a743de091dcf-kube-api-access-pqf76\") pod \"rate-test-model-faf483-kserve-54bf59f9d7-klb6n\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") " pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:57:38.101741 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.101706    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-tmp-dir\") pod \"rate-test-model-faf483-kserve-54bf59f9d7-klb6n\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") " pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:57:38.101741 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.101732    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/267a4c4b-8b64-4b89-b692-a743de091dcf-tls-certs\") pod \"rate-test-model-faf483-kserve-54bf59f9d7-klb6n\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") " pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:57:38.101849 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.101772    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-dshm\") pod \"rate-test-model-faf483-kserve-54bf59f9d7-klb6n\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") " pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:57:38.101849 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.101821    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-model-cache\") pod \"rate-test-model-faf483-kserve-54bf59f9d7-klb6n\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") " pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:57:38.203285 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.203248    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqf76\" (UniqueName: \"kubernetes.io/projected/267a4c4b-8b64-4b89-b692-a743de091dcf-kube-api-access-pqf76\") pod \"rate-test-model-faf483-kserve-54bf59f9d7-klb6n\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") " pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:57:38.203456 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.203293    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-tmp-dir\") pod \"rate-test-model-faf483-kserve-54bf59f9d7-klb6n\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") " pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:57:38.203456 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.203312    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/267a4c4b-8b64-4b89-b692-a743de091dcf-tls-certs\") pod \"rate-test-model-faf483-kserve-54bf59f9d7-klb6n\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") " pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:57:38.203456 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.203357    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-dshm\") pod \"rate-test-model-faf483-kserve-54bf59f9d7-klb6n\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") " pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:57:38.203456 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.203388    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-model-cache\") pod \"rate-test-model-faf483-kserve-54bf59f9d7-klb6n\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") " pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:57:38.203456 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.203427    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-home\") pod \"rate-test-model-faf483-kserve-54bf59f9d7-klb6n\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") " pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:57:38.203880 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.203859    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-home\") pod \"rate-test-model-faf483-kserve-54bf59f9d7-klb6n\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") " pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:57:38.203951 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.203878    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-tmp-dir\") pod \"rate-test-model-faf483-kserve-54bf59f9d7-klb6n\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") " pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:57:38.203951 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.203924    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-model-cache\") pod \"rate-test-model-faf483-kserve-54bf59f9d7-klb6n\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") " pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:57:38.205638 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.205618    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-dshm\") pod \"rate-test-model-faf483-kserve-54bf59f9d7-klb6n\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") " pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:57:38.206014 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.205995    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/267a4c4b-8b64-4b89-b692-a743de091dcf-tls-certs\") pod \"rate-test-model-faf483-kserve-54bf59f9d7-klb6n\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") " pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:57:38.211834 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.211809    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqf76\" (UniqueName: \"kubernetes.io/projected/267a4c4b-8b64-4b89-b692-a743de091dcf-kube-api-access-pqf76\") pod \"rate-test-model-faf483-kserve-54bf59f9d7-klb6n\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") " pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:57:38.388663 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.388570    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:57:38.524246 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.524218    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"]
Jul 28 22:57:38.525603 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:57:38.525578    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod267a4c4b_8b64_4b89_b692_a743de091dcf.slice/crio-21a8e811b7befa95805a47e74667feb960a5422e9a774e45422380cd08befa61 WatchSource:0}: Error finding container 21a8e811b7befa95805a47e74667feb960a5422e9a774e45422380cd08befa61: Status 404 returned error can't find the container with id 21a8e811b7befa95805a47e74667feb960a5422e9a774e45422380cd08befa61
Jul 28 22:57:38.678473 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.678386    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n" event={"ID":"267a4c4b-8b64-4b89-b692-a743de091dcf","Type":"ContainerStarted","Data":"eaba764ed91d1f0e36fe91612185739b86c227590c6463f37fc477028d83c3b8"}
Jul 28 22:57:38.678473 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.678428    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n" event={"ID":"267a4c4b-8b64-4b89-b692-a743de091dcf","Type":"ContainerStarted","Data":"21a8e811b7befa95805a47e74667feb960a5422e9a774e45422380cd08befa61"}
Jul 28 22:57:38.697397 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:38.697339    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n" podStartSLOduration=0.697323749 podStartE2EDuration="697.323749ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:57:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:57:38.696018277 +0000 UTC m=+2866.713983971" watchObservedRunningTime="2026-07-28 22:57:38.697323749 +0000 UTC m=+2866.715289458"
Jul 28 22:57:41.679745 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:41.679708    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:57:48.389560 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:48.389461    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:57:48.389560 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:48.389504    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:57:48.402642 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:48.402614    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:57:48.738192 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:57:48.738113    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:58:12.070174 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:58:12.070142    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:58:25.967920 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:58:25.967882    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:58:40.874142 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:58:40.874109    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:58:45.556883 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:58:45.556850    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"]
Jul 28 22:58:58.165485 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:58:58.165446    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:59:00.576095 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:00.576044    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n" podUID="267a4c4b-8b64-4b89-b692-a743de091dcf" containerName="main" containerID="cri-o://eaba764ed91d1f0e36fe91612185739b86c227590c6463f37fc477028d83c3b8" gracePeriod=45
Jul 28 22:59:00.843982 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:00.843953    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:59:00.944401 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:00.944371    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-model-cache\") pod \"267a4c4b-8b64-4b89-b692-a743de091dcf\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") "
Jul 28 22:59:00.944590 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:00.944438    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/267a4c4b-8b64-4b89-b692-a743de091dcf-tls-certs\") pod \"267a4c4b-8b64-4b89-b692-a743de091dcf\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") "
Jul 28 22:59:00.944590 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:00.944457    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-home\") pod \"267a4c4b-8b64-4b89-b692-a743de091dcf\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") "
Jul 28 22:59:00.944590 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:00.944487    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-tmp-dir\") pod \"267a4c4b-8b64-4b89-b692-a743de091dcf\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") "
Jul 28 22:59:00.944590 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:00.944564    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqf76\" (UniqueName: \"kubernetes.io/projected/267a4c4b-8b64-4b89-b692-a743de091dcf-kube-api-access-pqf76\") pod \"267a4c4b-8b64-4b89-b692-a743de091dcf\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") "
Jul 28 22:59:00.944590 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:00.944590    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-dshm\") pod \"267a4c4b-8b64-4b89-b692-a743de091dcf\" (UID: \"267a4c4b-8b64-4b89-b692-a743de091dcf\") "
Jul 28 22:59:00.944871 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:00.944699    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-model-cache" (OuterVolumeSpecName: "model-cache") pod "267a4c4b-8b64-4b89-b692-a743de091dcf" (UID: "267a4c4b-8b64-4b89-b692-a743de091dcf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:59:00.944871 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:00.944728    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-home" (OuterVolumeSpecName: "home") pod "267a4c4b-8b64-4b89-b692-a743de091dcf" (UID: "267a4c4b-8b64-4b89-b692-a743de091dcf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:59:00.944871 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:00.944810    2551 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-model-cache\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:59:00.944871 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:00.944853    2551 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-home\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:59:00.945023 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:00.944882    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "267a4c4b-8b64-4b89-b692-a743de091dcf" (UID: "267a4c4b-8b64-4b89-b692-a743de091dcf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:59:00.946839 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:00.946804    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/267a4c4b-8b64-4b89-b692-a743de091dcf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "267a4c4b-8b64-4b89-b692-a743de091dcf" (UID: "267a4c4b-8b64-4b89-b692-a743de091dcf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 22:59:00.946839 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:00.946819    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-dshm" (OuterVolumeSpecName: "dshm") pod "267a4c4b-8b64-4b89-b692-a743de091dcf" (UID: "267a4c4b-8b64-4b89-b692-a743de091dcf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 22:59:00.946979 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:00.946861    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/267a4c4b-8b64-4b89-b692-a743de091dcf-kube-api-access-pqf76" (OuterVolumeSpecName: "kube-api-access-pqf76") pod "267a4c4b-8b64-4b89-b692-a743de091dcf" (UID: "267a4c4b-8b64-4b89-b692-a743de091dcf"). InnerVolumeSpecName "kube-api-access-pqf76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 22:59:01.045624 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:01.045594    2551 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/267a4c4b-8b64-4b89-b692-a743de091dcf-tls-certs\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:59:01.045771 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:01.045641    2551 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-tmp-dir\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:59:01.045771 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:01.045651    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqf76\" (UniqueName: \"kubernetes.io/projected/267a4c4b-8b64-4b89-b692-a743de091dcf-kube-api-access-pqf76\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:59:01.045771 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:01.045662    2551 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/267a4c4b-8b64-4b89-b692-a743de091dcf-dshm\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 22:59:01.059755 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:01.059729    2551 generic.go:356] "Generic (PLEG): container finished" podID="267a4c4b-8b64-4b89-b692-a743de091dcf" containerID="eaba764ed91d1f0e36fe91612185739b86c227590c6463f37fc477028d83c3b8" exitCode=0
Jul 28 22:59:01.059920 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:01.059774    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"
Jul 28 22:59:01.059920 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:01.059810    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n" event={"ID":"267a4c4b-8b64-4b89-b692-a743de091dcf","Type":"ContainerDied","Data":"eaba764ed91d1f0e36fe91612185739b86c227590c6463f37fc477028d83c3b8"}
Jul 28 22:59:01.059920 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:01.059854    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n" event={"ID":"267a4c4b-8b64-4b89-b692-a743de091dcf","Type":"ContainerDied","Data":"21a8e811b7befa95805a47e74667feb960a5422e9a774e45422380cd08befa61"}
Jul 28 22:59:01.059920 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:01.059876    2551 scope.go:117] "RemoveContainer" containerID="eaba764ed91d1f0e36fe91612185739b86c227590c6463f37fc477028d83c3b8"
Jul 28 22:59:01.068095 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:01.068080    2551 scope.go:117] "RemoveContainer" containerID="eaba764ed91d1f0e36fe91612185739b86c227590c6463f37fc477028d83c3b8"
Jul 28 22:59:01.068328 ip-10-0-141-102 kubenswrapper[2551]: E0728 22:59:01.068311    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaba764ed91d1f0e36fe91612185739b86c227590c6463f37fc477028d83c3b8\": container with ID starting with eaba764ed91d1f0e36fe91612185739b86c227590c6463f37fc477028d83c3b8 not found: ID does not exist" containerID="eaba764ed91d1f0e36fe91612185739b86c227590c6463f37fc477028d83c3b8"
Jul 28 22:59:01.068380 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:01.068334    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaba764ed91d1f0e36fe91612185739b86c227590c6463f37fc477028d83c3b8"} err="failed to get container status \"eaba764ed91d1f0e36fe91612185739b86c227590c6463f37fc477028d83c3b8\": rpc error: code = NotFound desc = could not find container \"eaba764ed91d1f0e36fe91612185739b86c227590c6463f37fc477028d83c3b8\": container with ID starting with eaba764ed91d1f0e36fe91612185739b86c227590c6463f37fc477028d83c3b8 not found: ID does not exist"
Jul 28 22:59:01.080561 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:01.080498    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"]
Jul 28 22:59:01.086326 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:01.086306    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-faf483/rate-test-model-faf483-kserve-54bf59f9d7-klb6n"]
Jul 28 22:59:02.463605 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:02.463572    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="267a4c4b-8b64-4b89-b692-a743de091dcf" path="/var/lib/kubelet/pods/267a4c4b-8b64-4b89-b692-a743de091dcf/volumes"
Jul 28 22:59:09.913010 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:09.912974    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb"]
Jul 28 22:59:09.913631 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:09.913477    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="267a4c4b-8b64-4b89-b692-a743de091dcf" containerName="main"
Jul 28 22:59:09.913631 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:09.913495    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="267a4c4b-8b64-4b89-b692-a743de091dcf" containerName="main"
Jul 28 22:59:09.913631 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:09.913624    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="267a4c4b-8b64-4b89-b692-a743de091dcf" containerName="main"
Jul 28 22:59:09.916602 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:09.916584    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb"
Jul 28 22:59:09.918373 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:09.918345    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqdg2\" (UniqueName: \"kubernetes.io/projected/5d6777e5-90ff-40a5-a17c-20918daffaab-kube-api-access-sqdg2\") pod \"payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb\" (UID: \"5d6777e5-90ff-40a5-a17c-20918daffaab\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb"
Jul 28 22:59:09.918489 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:09.918387    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d6777e5-90ff-40a5-a17c-20918daffaab-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb\" (UID: \"5d6777e5-90ff-40a5-a17c-20918daffaab\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb"
Jul 28 22:59:09.918683 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:09.918663    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-c6c731\""
Jul 28 22:59:09.918725 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:09.918682    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-c6c731-dockercfg-pq7zp\""
Jul 28 22:59:09.925579 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:09.925556    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb"]
Jul 28 22:59:10.019604 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:10.019566    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d6777e5-90ff-40a5-a17c-20918daffaab-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb\" (UID: \"5d6777e5-90ff-40a5-a17c-20918daffaab\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb"
Jul 28 22:59:10.019800 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:10.019724    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqdg2\" (UniqueName: \"kubernetes.io/projected/5d6777e5-90ff-40a5-a17c-20918daffaab-kube-api-access-sqdg2\") pod \"payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb\" (UID: \"5d6777e5-90ff-40a5-a17c-20918daffaab\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb"
Jul 28 22:59:10.020598 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:10.020563    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d6777e5-90ff-40a5-a17c-20918daffaab-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb\" (UID: \"5d6777e5-90ff-40a5-a17c-20918daffaab\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb"
Jul 28 22:59:10.027761 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:10.027701    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqdg2\" (UniqueName: \"kubernetes.io/projected/5d6777e5-90ff-40a5-a17c-20918daffaab-kube-api-access-sqdg2\") pod \"payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb\" (UID: \"5d6777e5-90ff-40a5-a17c-20918daffaab\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb"
Jul 28 22:59:10.228317 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:10.228232    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb"
Jul 28 22:59:10.361210 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:10.361185    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb"]
Jul 28 22:59:10.364165 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:59:10.364138    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d6777e5_90ff_40a5_a17c_20918daffaab.slice/crio-af9935a4a38e12448a3e01ba97cc5dc49c1857faa1686193701e02125a68d65d WatchSource:0}: Error finding container af9935a4a38e12448a3e01ba97cc5dc49c1857faa1686193701e02125a68d65d: Status 404 returned error can't find the container with id af9935a4a38e12448a3e01ba97cc5dc49c1857faa1686193701e02125a68d65d
Jul 28 22:59:11.108515 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:11.108477    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb" event={"ID":"5d6777e5-90ff-40a5-a17c-20918daffaab","Type":"ContainerStarted","Data":"0107f6833457be291bbdee8d4f4b9307fe23b916cefd50c2bafe97baab3b6f54"}
Jul 28 22:59:11.108917 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:11.108553    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb" event={"ID":"5d6777e5-90ff-40a5-a17c-20918daffaab","Type":"ContainerStarted","Data":"af9935a4a38e12448a3e01ba97cc5dc49c1857faa1686193701e02125a68d65d"}
Jul 28 22:59:11.108917 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:11.108576    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb"
Jul 28 22:59:11.123706 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:11.123647    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb" podStartSLOduration=2.123627276 podStartE2EDuration="2.123627276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:59:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:59:11.121476679 +0000 UTC m=+2959.139442374" watchObservedRunningTime="2026-07-28 22:59:11.123627276 +0000 UTC m=+2959.141592971"
Jul 28 22:59:18.368068 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:18.367989    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:59:22.117494 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:22.117461    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb"
Jul 28 22:59:35.035677 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.035643    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"]
Jul 28 22:59:35.040233 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.040206    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.049488 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.049464    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-6fb213-openshift-default-dockercfg-fhr76\""
Jul 28 22:59:35.074831 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.074799    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"]
Jul 28 22:59:35.125888 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.125854    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-credential-socket\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.126057 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.125894    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khlwc\" (UniqueName: \"kubernetes.io/projected/a2e70786-1610-43cf-8ba7-230457c157bc-kube-api-access-khlwc\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.126057 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.125928    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-istio-data\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.126057 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.125981    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2e70786-1610-43cf-8ba7-230457c157bc-istio-podinfo\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.126057 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.126025    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-istio-envoy\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.126230 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.126104    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2e70786-1610-43cf-8ba7-230457c157bc-istio-token\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.126230 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.126139    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2e70786-1610-43cf-8ba7-230457c157bc-istiod-ca-cert\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.126230 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.126194    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-workload-certs\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.126230 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.126226    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-workload-socket\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.227655 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.227616    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-istio-envoy\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.227860 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.227678    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2e70786-1610-43cf-8ba7-230457c157bc-istio-token\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.227860 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.227716    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2e70786-1610-43cf-8ba7-230457c157bc-istiod-ca-cert\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.227860 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.227760    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-workload-certs\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.228097 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.227918    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-workload-socket\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.228097 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.228005    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-credential-socket\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.228097 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.228043    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khlwc\" (UniqueName: \"kubernetes.io/projected/a2e70786-1610-43cf-8ba7-230457c157bc-kube-api-access-khlwc\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.228097 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.228076    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-istio-data\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.228315 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.228118    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2e70786-1610-43cf-8ba7-230457c157bc-istio-podinfo\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.228315 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.228129    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-workload-certs\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.228428 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.228353    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-workload-socket\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.228428 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.228403    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-istio-data\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.228563 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.228518    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2e70786-1610-43cf-8ba7-230457c157bc-istiod-ca-cert\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.228633 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.228615    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-credential-socket\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.230541 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.230507    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2e70786-1610-43cf-8ba7-230457c157bc-istio-podinfo\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.230659 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.230635    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-istio-envoy\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.239757 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.239732    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2e70786-1610-43cf-8ba7-230457c157bc-istio-token\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.248126 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.248098    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khlwc\" (UniqueName: \"kubernetes.io/projected/a2e70786-1610-43cf-8ba7-230457c157bc-kube-api-access-khlwc\") pod \"e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") " pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.350393 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.350357    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:35.491690 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.491665    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"]
Jul 28 22:59:35.493603 ip-10-0-141-102 kubenswrapper[2551]: W0728 22:59:35.493568    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2e70786_1610_43cf_8ba7_230457c157bc.slice/crio-ae6648726c5c38ecb9887712ce7d4161e590e4fd95d4aecf7f79ce82ef3e22ba WatchSource:0}: Error finding container ae6648726c5c38ecb9887712ce7d4161e590e4fd95d4aecf7f79ce82ef3e22ba: Status 404 returned error can't find the container with id ae6648726c5c38ecb9887712ce7d4161e590e4fd95d4aecf7f79ce82ef3e22ba
Jul 28 22:59:35.495835 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.495800    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 22:59:35.495961 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.495877    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 22:59:35.495961 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:35.495923    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 22:59:36.229894 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:36.229860    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh" event={"ID":"a2e70786-1610-43cf-8ba7-230457c157bc","Type":"ContainerStarted","Data":"941070a454982c2fddcf9a95659853d6b41b505b3ac86834c74a95bd12b144e6"}
Jul 28 22:59:36.230288 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:36.229919    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh" event={"ID":"a2e70786-1610-43cf-8ba7-230457c157bc","Type":"ContainerStarted","Data":"ae6648726c5c38ecb9887712ce7d4161e590e4fd95d4aecf7f79ce82ef3e22ba"}
Jul 28 22:59:36.248472 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:36.248418    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh" podStartSLOduration=1.24839956 podStartE2EDuration="1.24839956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 22:59:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 22:59:36.247629987 +0000 UTC m=+2984.265595691" watchObservedRunningTime="2026-07-28 22:59:36.24839956 +0000 UTC m=+2984.266365256"
Jul 28 22:59:36.350748 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:36.350715    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:36.355846 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:36.355823    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:37.234427 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:37.234397    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:37.235556 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:37.235518    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:47.466158 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:47.466119    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 22:59:55.123926 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:55.123899    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:59:55.131135 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:55.131117    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-222qs_481b2f34-6f30-4e75-986c-c7eba23f6214/ovn-acl-logging/0.log"
Jul 28 22:59:55.330131 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:55.330097    2551 generic.go:356] "Generic (PLEG): container finished" podID="a2e70786-1610-43cf-8ba7-230457c157bc" containerID="941070a454982c2fddcf9a95659853d6b41b505b3ac86834c74a95bd12b144e6" exitCode=0
Jul 28 22:59:55.330301 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:55.330165    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh" event={"ID":"a2e70786-1610-43cf-8ba7-230457c157bc","Type":"ContainerDied","Data":"941070a454982c2fddcf9a95659853d6b41b505b3ac86834c74a95bd12b144e6"}
Jul 28 22:59:55.330607 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:55.330587    2551 scope.go:117] "RemoveContainer" containerID="941070a454982c2fddcf9a95659853d6b41b505b3ac86834c74a95bd12b144e6"
Jul 28 22:59:55.331660 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:55.331630    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 22:59:55.331758 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:55.331697    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 22:59:55.331758 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:55.331729    2551 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 28 22:59:56.339515 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:56.339470    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh" event={"ID":"a2e70786-1610-43cf-8ba7-230457c157bc","Type":"ContainerStarted","Data":"630be4775db23ede049013b654caa1e862f3ce46e144a1577d82407c9496fad3"}
Jul 28 22:59:56.350979 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:56.350949    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 22:59:56.351190 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:56.351167    2551 patch_prober.go:28] interesting pod/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 28 22:59:56.351269 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:56.351238    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh" podUID="a2e70786-1610-43cf-8ba7-230457c157bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 28 22:59:57.351383 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:57.351347    2551 patch_prober.go:28] interesting pod/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 28 22:59:57.351789 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:57.351398    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh" podUID="a2e70786-1610-43cf-8ba7-230457c157bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 28 22:59:58.351716 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:58.351682    2551 patch_prober.go:28] interesting pod/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 28 22:59:58.352107 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:58.351737    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh" podUID="a2e70786-1610-43cf-8ba7-230457c157bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 28 22:59:59.351129 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:59.351093    2551 patch_prober.go:28] interesting pod/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 28 22:59:59.351414 ip-10-0-141-102 kubenswrapper[2551]: I0728 22:59:59.351386    2551 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh" podUID="a2e70786-1610-43cf-8ba7-230457c157bc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 28 23:00:00.354556 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:00:00.354508    2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 23:00:00.355024 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:00:00.355010    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 23:00:00.355774 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:00:00.355747    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 23:00:21.375965 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:00:21.375925    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 23:00:50.463455 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:00:50.463384    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 23:01:07.259992 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.259955    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz"]
Jul 28 23:01:07.263263 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.263246    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz"
Jul 28 23:01:07.265333 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.265315    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-030bcd\""
Jul 28 23:01:07.265470 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.265315    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-030bcd-dockercfg-5pgmf\""
Jul 28 23:01:07.277490 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.277464    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz"]
Jul 28 23:01:07.297895 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.297867    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz"]
Jul 28 23:01:07.301265 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.301246    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz"
Jul 28 23:01:07.310415 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.310390    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz"]
Jul 28 23:01:07.368319 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.368286    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl2hf\" (UniqueName: \"kubernetes.io/projected/4adfb636-6b31-47dc-ae6b-136b01cd3f00-kube-api-access-hl2hf\") pod \"payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz\" (UID: \"4adfb636-6b31-47dc-ae6b-136b01cd3f00\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz"
Jul 28 23:01:07.368495 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.368399    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4adfb636-6b31-47dc-ae6b-136b01cd3f00-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz\" (UID: \"4adfb636-6b31-47dc-ae6b-136b01cd3f00\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz"
Jul 28 23:01:07.368495 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.368451    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbb3b710-7008-4254-8b3a-db86af9bd7f1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz\" (UID: \"fbb3b710-7008-4254-8b3a-db86af9bd7f1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz"
Jul 28 23:01:07.368613 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.368502    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fj57\" (UniqueName: \"kubernetes.io/projected/fbb3b710-7008-4254-8b3a-db86af9bd7f1-kube-api-access-6fj57\") pod \"payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz\" (UID: \"fbb3b710-7008-4254-8b3a-db86af9bd7f1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz"
Jul 28 23:01:07.469189 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.469152    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl2hf\" (UniqueName: \"kubernetes.io/projected/4adfb636-6b31-47dc-ae6b-136b01cd3f00-kube-api-access-hl2hf\") pod \"payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz\" (UID: \"4adfb636-6b31-47dc-ae6b-136b01cd3f00\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz"
Jul 28 23:01:07.469189 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.469191    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4adfb636-6b31-47dc-ae6b-136b01cd3f00-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz\" (UID: \"4adfb636-6b31-47dc-ae6b-136b01cd3f00\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz"
Jul 28 23:01:07.469453 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.469214    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbb3b710-7008-4254-8b3a-db86af9bd7f1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz\" (UID: \"fbb3b710-7008-4254-8b3a-db86af9bd7f1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz"
Jul 28 23:01:07.469453 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.469251    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fj57\" (UniqueName: \"kubernetes.io/projected/fbb3b710-7008-4254-8b3a-db86af9bd7f1-kube-api-access-6fj57\") pod \"payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz\" (UID: \"fbb3b710-7008-4254-8b3a-db86af9bd7f1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz"
Jul 28 23:01:07.469963 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.469942    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4adfb636-6b31-47dc-ae6b-136b01cd3f00-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz\" (UID: \"4adfb636-6b31-47dc-ae6b-136b01cd3f00\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz"
Jul 28 23:01:07.470162 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.470138    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbb3b710-7008-4254-8b3a-db86af9bd7f1-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz\" (UID: \"fbb3b710-7008-4254-8b3a-db86af9bd7f1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz"
Jul 28 23:01:07.476664 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.476637    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fj57\" (UniqueName: \"kubernetes.io/projected/fbb3b710-7008-4254-8b3a-db86af9bd7f1-kube-api-access-6fj57\") pod \"payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz\" (UID: \"fbb3b710-7008-4254-8b3a-db86af9bd7f1\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz"
Jul 28 23:01:07.476664 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.476652    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl2hf\" (UniqueName: \"kubernetes.io/projected/4adfb636-6b31-47dc-ae6b-136b01cd3f00-kube-api-access-hl2hf\") pod \"payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz\" (UID: \"4adfb636-6b31-47dc-ae6b-136b01cd3f00\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz"
Jul 28 23:01:07.573421 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.573395    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz"
Jul 28 23:01:07.615744 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.615386    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz"
Jul 28 23:01:07.724898 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.724833    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz"]
Jul 28 23:01:07.729988 ip-10-0-141-102 kubenswrapper[2551]: W0728 23:01:07.729954    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4adfb636_6b31_47dc_ae6b_136b01cd3f00.slice/crio-28c0ba49770b56930a0aed490b4cbf5040f6fb3014f03188b2bcf7ca26367f36 WatchSource:0}: Error finding container 28c0ba49770b56930a0aed490b4cbf5040f6fb3014f03188b2bcf7ca26367f36: Status 404 returned error can't find the container with id 28c0ba49770b56930a0aed490b4cbf5040f6fb3014f03188b2bcf7ca26367f36
Jul 28 23:01:07.773746 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:07.773720    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz"]
Jul 28 23:01:07.775988 ip-10-0-141-102 kubenswrapper[2551]: W0728 23:01:07.775947    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbb3b710_7008_4254_8b3a_db86af9bd7f1.slice/crio-344e369b4c047ff3325baab4128a1b35e99bcf06ed377c99d2765419ae13c443 WatchSource:0}: Error finding container 344e369b4c047ff3325baab4128a1b35e99bcf06ed377c99d2765419ae13c443: Status 404 returned error can't find the container with id 344e369b4c047ff3325baab4128a1b35e99bcf06ed377c99d2765419ae13c443
Jul 28 23:01:08.693799 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:08.693489    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz" event={"ID":"4adfb636-6b31-47dc-ae6b-136b01cd3f00","Type":"ContainerStarted","Data":"9cc8e0b36e5e2a3158cbfe6958c85b003c19d6f2a797c3d785c695806b78ad7d"}
Jul 28 23:01:08.693799 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:08.693583    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz" event={"ID":"4adfb636-6b31-47dc-ae6b-136b01cd3f00","Type":"ContainerStarted","Data":"28c0ba49770b56930a0aed490b4cbf5040f6fb3014f03188b2bcf7ca26367f36"}
Jul 28 23:01:08.693799 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:08.693733    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz"
Jul 28 23:01:08.695399 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:08.695372    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz" event={"ID":"fbb3b710-7008-4254-8b3a-db86af9bd7f1","Type":"ContainerStarted","Data":"6f88d792d64338c8b6b0f4ba2ab66d6581987e85d18be66a4de1489181b2d31d"}
Jul 28 23:01:08.695553 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:08.695424    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz" event={"ID":"fbb3b710-7008-4254-8b3a-db86af9bd7f1","Type":"ContainerStarted","Data":"344e369b4c047ff3325baab4128a1b35e99bcf06ed377c99d2765419ae13c443"}
Jul 28 23:01:08.695553 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:08.695455    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz"
Jul 28 23:01:08.708641 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:08.708600    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz" podStartSLOduration=1.7085867179999998 podStartE2EDuration="1.708586718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:01:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:01:08.707498292 +0000 UTC m=+3076.725463987" watchObservedRunningTime="2026-07-28 23:01:08.708586718 +0000 UTC m=+3076.726552447"
Jul 28 23:01:08.722942 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:08.722890    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz" podStartSLOduration=1.722873608 podStartE2EDuration="1.722873608s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 23:01:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:01:08.720664898 +0000 UTC m=+3076.738630594" watchObservedRunningTime="2026-07-28 23:01:08.722873608 +0000 UTC m=+3076.740839302"
Jul 28 23:01:14.366637 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:14.366604    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 23:01:19.702712 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:19.702682    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz"
Jul 28 23:01:19.703151 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:19.702827    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz"
Jul 28 23:01:28.579673 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:28.579637    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz"]
Jul 28 23:01:28.580274 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:28.579857    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz" podUID="fbb3b710-7008-4254-8b3a-db86af9bd7f1" containerName="payload-processing" containerID="cri-o://6f88d792d64338c8b6b0f4ba2ab66d6581987e85d18be66a4de1489181b2d31d" gracePeriod=30
Jul 28 23:01:28.587910 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:28.587453    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz"]
Jul 28 23:01:28.587910 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:28.587702    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz" podUID="4adfb636-6b31-47dc-ae6b-136b01cd3f00" containerName="payload-pre-processing" containerID="cri-o://9cc8e0b36e5e2a3158cbfe6958c85b003c19d6f2a797c3d785c695806b78ad7d" gracePeriod=30
Jul 28 23:01:28.795594 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:28.795557    2551 generic.go:356] "Generic (PLEG): container finished" podID="4adfb636-6b31-47dc-ae6b-136b01cd3f00" containerID="9cc8e0b36e5e2a3158cbfe6958c85b003c19d6f2a797c3d785c695806b78ad7d" exitCode=0
Jul 28 23:01:28.795779 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:28.795636    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz" event={"ID":"4adfb636-6b31-47dc-ae6b-136b01cd3f00","Type":"ContainerDied","Data":"9cc8e0b36e5e2a3158cbfe6958c85b003c19d6f2a797c3d785c695806b78ad7d"}
Jul 28 23:01:28.797282 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:28.797255    2551 generic.go:356] "Generic (PLEG): container finished" podID="fbb3b710-7008-4254-8b3a-db86af9bd7f1" containerID="6f88d792d64338c8b6b0f4ba2ab66d6581987e85d18be66a4de1489181b2d31d" exitCode=0
Jul 28 23:01:28.797433 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:28.797334    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz" event={"ID":"fbb3b710-7008-4254-8b3a-db86af9bd7f1","Type":"ContainerDied","Data":"6f88d792d64338c8b6b0f4ba2ab66d6581987e85d18be66a4de1489181b2d31d"}
Jul 28 23:01:28.848737 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:28.848714    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz"
Jul 28 23:01:28.880304 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:28.880280    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz"
Jul 28 23:01:28.972497 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:28.972466    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fj57\" (UniqueName: \"kubernetes.io/projected/fbb3b710-7008-4254-8b3a-db86af9bd7f1-kube-api-access-6fj57\") pod \"fbb3b710-7008-4254-8b3a-db86af9bd7f1\" (UID: \"fbb3b710-7008-4254-8b3a-db86af9bd7f1\") "
Jul 28 23:01:28.972670 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:28.972562    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbb3b710-7008-4254-8b3a-db86af9bd7f1-plugins-config-volume\") pod \"fbb3b710-7008-4254-8b3a-db86af9bd7f1\" (UID: \"fbb3b710-7008-4254-8b3a-db86af9bd7f1\") "
Jul 28 23:01:28.972932 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:28.972907    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbb3b710-7008-4254-8b3a-db86af9bd7f1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fbb3b710-7008-4254-8b3a-db86af9bd7f1" (UID: "fbb3b710-7008-4254-8b3a-db86af9bd7f1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:01:28.974491 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:28.974470    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fbb3b710-7008-4254-8b3a-db86af9bd7f1-kube-api-access-6fj57" (OuterVolumeSpecName: "kube-api-access-6fj57") pod "fbb3b710-7008-4254-8b3a-db86af9bd7f1" (UID: "fbb3b710-7008-4254-8b3a-db86af9bd7f1"). InnerVolumeSpecName "kube-api-access-6fj57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:01:29.073030 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:29.072996    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4adfb636-6b31-47dc-ae6b-136b01cd3f00-plugins-config-volume\") pod \"4adfb636-6b31-47dc-ae6b-136b01cd3f00\" (UID: \"4adfb636-6b31-47dc-ae6b-136b01cd3f00\") "
Jul 28 23:01:29.073190 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:29.073111    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hl2hf\" (UniqueName: \"kubernetes.io/projected/4adfb636-6b31-47dc-ae6b-136b01cd3f00-kube-api-access-hl2hf\") pod \"4adfb636-6b31-47dc-ae6b-136b01cd3f00\" (UID: \"4adfb636-6b31-47dc-ae6b-136b01cd3f00\") "
Jul 28 23:01:29.073267 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:29.073256    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fj57\" (UniqueName: \"kubernetes.io/projected/fbb3b710-7008-4254-8b3a-db86af9bd7f1-kube-api-access-6fj57\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 23:01:29.073306 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:29.073270    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fbb3b710-7008-4254-8b3a-db86af9bd7f1-plugins-config-volume\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 23:01:29.073340 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:29.073313    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4adfb636-6b31-47dc-ae6b-136b01cd3f00-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4adfb636-6b31-47dc-ae6b-136b01cd3f00" (UID: "4adfb636-6b31-47dc-ae6b-136b01cd3f00"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:01:29.075045 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:29.075025    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4adfb636-6b31-47dc-ae6b-136b01cd3f00-kube-api-access-hl2hf" (OuterVolumeSpecName: "kube-api-access-hl2hf") pod "4adfb636-6b31-47dc-ae6b-136b01cd3f00" (UID: "4adfb636-6b31-47dc-ae6b-136b01cd3f00"). InnerVolumeSpecName "kube-api-access-hl2hf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:01:29.174787 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:29.174704    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hl2hf\" (UniqueName: \"kubernetes.io/projected/4adfb636-6b31-47dc-ae6b-136b01cd3f00-kube-api-access-hl2hf\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 23:01:29.174787 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:29.174733    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4adfb636-6b31-47dc-ae6b-136b01cd3f00-plugins-config-volume\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 23:01:29.804142 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:29.804109    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz"
Jul 28 23:01:29.804623 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:29.804145    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz" event={"ID":"fbb3b710-7008-4254-8b3a-db86af9bd7f1","Type":"ContainerDied","Data":"344e369b4c047ff3325baab4128a1b35e99bcf06ed377c99d2765419ae13c443"}
Jul 28 23:01:29.804623 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:29.804208    2551 scope.go:117] "RemoveContainer" containerID="6f88d792d64338c8b6b0f4ba2ab66d6581987e85d18be66a4de1489181b2d31d"
Jul 28 23:01:29.805690 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:29.805575    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz" event={"ID":"4adfb636-6b31-47dc-ae6b-136b01cd3f00","Type":"ContainerDied","Data":"28c0ba49770b56930a0aed490b4cbf5040f6fb3014f03188b2bcf7ca26367f36"}
Jul 28 23:01:29.805690 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:29.805589    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz"
Jul 28 23:01:29.813412 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:29.813386    2551 scope.go:117] "RemoveContainer" containerID="9cc8e0b36e5e2a3158cbfe6958c85b003c19d6f2a797c3d785c695806b78ad7d"
Jul 28 23:01:29.827686 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:29.827664    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz"]
Jul 28 23:01:29.830951 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:29.830930    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-030bcd-7bfd45fc76-gjgbz"]
Jul 28 23:01:29.840118 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:29.840099    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz"]
Jul 28 23:01:29.843472 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:29.843453    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-030bcd-85566bcfc6-npnnz"]
Jul 28 23:01:30.463243 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:30.463215    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4adfb636-6b31-47dc-ae6b-136b01cd3f00" path="/var/lib/kubelet/pods/4adfb636-6b31-47dc-ae6b-136b01cd3f00/volumes"
Jul 28 23:01:30.463604 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:30.463591    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbb3b710-7008-4254-8b3a-db86af9bd7f1" path="/var/lib/kubelet/pods/fbb3b710-7008-4254-8b3a-db86af9bd7f1/volumes"
Jul 28 23:01:34.666097 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:34.666056    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 23:01:53.108176 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:53.108140    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb"]
Jul 28 23:01:53.108612 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:53.108398    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb" podUID="5d6777e5-90ff-40a5-a17c-20918daffaab" containerName="payload-pre-processing" containerID="cri-o://0107f6833457be291bbdee8d4f4b9307fe23b916cefd50c2bafe97baab3b6f54" gracePeriod=30
Jul 28 23:01:53.277243 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:53.277206    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 23:01:53.391152 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:53.391125    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb"
Jul 28 23:01:53.467911 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:53.467878    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqdg2\" (UniqueName: \"kubernetes.io/projected/5d6777e5-90ff-40a5-a17c-20918daffaab-kube-api-access-sqdg2\") pod \"5d6777e5-90ff-40a5-a17c-20918daffaab\" (UID: \"5d6777e5-90ff-40a5-a17c-20918daffaab\") "
Jul 28 23:01:53.468075 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:53.467953    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d6777e5-90ff-40a5-a17c-20918daffaab-plugins-config-volume\") pod \"5d6777e5-90ff-40a5-a17c-20918daffaab\" (UID: \"5d6777e5-90ff-40a5-a17c-20918daffaab\") "
Jul 28 23:01:53.468315 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:53.468279    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d6777e5-90ff-40a5-a17c-20918daffaab-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5d6777e5-90ff-40a5-a17c-20918daffaab" (UID: "5d6777e5-90ff-40a5-a17c-20918daffaab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:01:53.469982 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:53.469956    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d6777e5-90ff-40a5-a17c-20918daffaab-kube-api-access-sqdg2" (OuterVolumeSpecName: "kube-api-access-sqdg2") pod "5d6777e5-90ff-40a5-a17c-20918daffaab" (UID: "5d6777e5-90ff-40a5-a17c-20918daffaab"). InnerVolumeSpecName "kube-api-access-sqdg2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:01:53.568573 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:53.568510    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqdg2\" (UniqueName: \"kubernetes.io/projected/5d6777e5-90ff-40a5-a17c-20918daffaab-kube-api-access-sqdg2\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 23:01:53.568573 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:53.568573    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d6777e5-90ff-40a5-a17c-20918daffaab-plugins-config-volume\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 23:01:53.928811 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:53.928778    2551 generic.go:356] "Generic (PLEG): container finished" podID="5d6777e5-90ff-40a5-a17c-20918daffaab" containerID="0107f6833457be291bbdee8d4f4b9307fe23b916cefd50c2bafe97baab3b6f54" exitCode=0
Jul 28 23:01:53.929010 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:53.928825    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb"
Jul 28 23:01:53.929010 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:53.928862    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb" event={"ID":"5d6777e5-90ff-40a5-a17c-20918daffaab","Type":"ContainerDied","Data":"0107f6833457be291bbdee8d4f4b9307fe23b916cefd50c2bafe97baab3b6f54"}
Jul 28 23:01:53.929010 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:53.928931    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb" event={"ID":"5d6777e5-90ff-40a5-a17c-20918daffaab","Type":"ContainerDied","Data":"af9935a4a38e12448a3e01ba97cc5dc49c1857faa1686193701e02125a68d65d"}
Jul 28 23:01:53.929010 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:53.928961    2551 scope.go:117] "RemoveContainer" containerID="0107f6833457be291bbdee8d4f4b9307fe23b916cefd50c2bafe97baab3b6f54"
Jul 28 23:01:53.937710 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:53.937690    2551 scope.go:117] "RemoveContainer" containerID="0107f6833457be291bbdee8d4f4b9307fe23b916cefd50c2bafe97baab3b6f54"
Jul 28 23:01:53.937980 ip-10-0-141-102 kubenswrapper[2551]: E0728 23:01:53.937960    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0107f6833457be291bbdee8d4f4b9307fe23b916cefd50c2bafe97baab3b6f54\": container with ID starting with 0107f6833457be291bbdee8d4f4b9307fe23b916cefd50c2bafe97baab3b6f54 not found: ID does not exist" containerID="0107f6833457be291bbdee8d4f4b9307fe23b916cefd50c2bafe97baab3b6f54"
Jul 28 23:01:53.938027 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:53.937989    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0107f6833457be291bbdee8d4f4b9307fe23b916cefd50c2bafe97baab3b6f54"} err="failed to get container status \"0107f6833457be291bbdee8d4f4b9307fe23b916cefd50c2bafe97baab3b6f54\": rpc error: code = NotFound desc = could not find container \"0107f6833457be291bbdee8d4f4b9307fe23b916cefd50c2bafe97baab3b6f54\": container with ID starting with 0107f6833457be291bbdee8d4f4b9307fe23b916cefd50c2bafe97baab3b6f54 not found: ID does not exist"
Jul 28 23:01:53.948789 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:53.948767    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb"]
Jul 28 23:01:53.952875 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:53.952852    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-c6c731-55dc9b4748-zgrfb"]
Jul 28 23:01:54.462175 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:01:54.462138    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d6777e5-90ff-40a5-a17c-20918daffaab" path="/var/lib/kubelet/pods/5d6777e5-90ff-40a5-a17c-20918daffaab/volumes"
Jul 28 23:02:04.371216 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:04.371179    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 23:02:51.069195 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:51.069159    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 23:02:54.241847 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:54.241808    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"]
Jul 28 23:02:54.242431 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:54.242056    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh" podUID="a2e70786-1610-43cf-8ba7-230457c157bc" containerName="istio-proxy" containerID="cri-o://630be4775db23ede049013b654caa1e862f3ce46e144a1577d82407c9496fad3" gracePeriod=30
Jul 28 23:02:59.502153 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.502129    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 23:02:59.550501 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.550399    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2e70786-1610-43cf-8ba7-230457c157bc-istiod-ca-cert\") pod \"a2e70786-1610-43cf-8ba7-230457c157bc\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") "
Jul 28 23:02:59.550501 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.550447    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2e70786-1610-43cf-8ba7-230457c157bc-istio-podinfo\") pod \"a2e70786-1610-43cf-8ba7-230457c157bc\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") "
Jul 28 23:02:59.550501 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.550498    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-credential-socket\") pod \"a2e70786-1610-43cf-8ba7-230457c157bc\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") "
Jul 28 23:02:59.550809 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.550535    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2e70786-1610-43cf-8ba7-230457c157bc-istio-token\") pod \"a2e70786-1610-43cf-8ba7-230457c157bc\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") "
Jul 28 23:02:59.550809 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.550564    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-istio-data\") pod \"a2e70786-1610-43cf-8ba7-230457c157bc\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") "
Jul 28 23:02:59.550809 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.550602    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khlwc\" (UniqueName: \"kubernetes.io/projected/a2e70786-1610-43cf-8ba7-230457c157bc-kube-api-access-khlwc\") pod \"a2e70786-1610-43cf-8ba7-230457c157bc\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") "
Jul 28 23:02:59.550809 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.550632    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-istio-envoy\") pod \"a2e70786-1610-43cf-8ba7-230457c157bc\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") "
Jul 28 23:02:59.550809 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.550684    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-workload-certs\") pod \"a2e70786-1610-43cf-8ba7-230457c157bc\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") "
Jul 28 23:02:59.550809 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.550718    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-workload-socket\") pod \"a2e70786-1610-43cf-8ba7-230457c157bc\" (UID: \"a2e70786-1610-43cf-8ba7-230457c157bc\") "
Jul 28 23:02:59.551115 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.550877    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a2e70786-1610-43cf-8ba7-230457c157bc" (UID: "a2e70786-1610-43cf-8ba7-230457c157bc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:02:59.551115 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.551086    2551 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-credential-socket\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 23:02:59.551115 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.551099    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a2e70786-1610-43cf-8ba7-230457c157bc" (UID: "a2e70786-1610-43cf-8ba7-230457c157bc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:02:59.551315 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.551290    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a2e70786-1610-43cf-8ba7-230457c157bc" (UID: "a2e70786-1610-43cf-8ba7-230457c157bc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:02:59.551475 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.551444    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2e70786-1610-43cf-8ba7-230457c157bc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a2e70786-1610-43cf-8ba7-230457c157bc" (UID: "a2e70786-1610-43cf-8ba7-230457c157bc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:02:59.551632 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.551521    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-istio-data" (OuterVolumeSpecName: "istio-data") pod "a2e70786-1610-43cf-8ba7-230457c157bc" (UID: "a2e70786-1610-43cf-8ba7-230457c157bc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:02:59.552892 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.552860    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2e70786-1610-43cf-8ba7-230457c157bc-istio-token" (OuterVolumeSpecName: "istio-token") pod "a2e70786-1610-43cf-8ba7-230457c157bc" (UID: "a2e70786-1610-43cf-8ba7-230457c157bc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:02:59.552892 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.552865    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a2e70786-1610-43cf-8ba7-230457c157bc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a2e70786-1610-43cf-8ba7-230457c157bc" (UID: "a2e70786-1610-43cf-8ba7-230457c157bc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 28 23:02:59.553273 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.553242    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a2e70786-1610-43cf-8ba7-230457c157bc" (UID: "a2e70786-1610-43cf-8ba7-230457c157bc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 23:02:59.553382 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.553250    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2e70786-1610-43cf-8ba7-230457c157bc-kube-api-access-khlwc" (OuterVolumeSpecName: "kube-api-access-khlwc") pod "a2e70786-1610-43cf-8ba7-230457c157bc" (UID: "a2e70786-1610-43cf-8ba7-230457c157bc"). InnerVolumeSpecName "kube-api-access-khlwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:02:59.651927 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.651889    2551 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-workload-certs\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 23:02:59.651927 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.651919    2551 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-workload-socket\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 23:02:59.651927 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.651932    2551 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2e70786-1610-43cf-8ba7-230457c157bc-istiod-ca-cert\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 23:02:59.652175 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.651943    2551 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2e70786-1610-43cf-8ba7-230457c157bc-istio-podinfo\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 23:02:59.652175 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.651954    2551 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2e70786-1610-43cf-8ba7-230457c157bc-istio-token\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 23:02:59.652175 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.651965    2551 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-istio-data\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 23:02:59.652175 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.651976    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khlwc\" (UniqueName: \"kubernetes.io/projected/a2e70786-1610-43cf-8ba7-230457c157bc-kube-api-access-khlwc\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 23:02:59.652175 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.651988    2551 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2e70786-1610-43cf-8ba7-230457c157bc-istio-envoy\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 23:02:59.918718 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.918684    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm"]
Jul 28 23:02:59.919452 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:02:59.918955    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm" podUID="3113fbc7-9efd-414f-9539-e603455f0de6" containerName="payload-processing" containerID="cri-o://97d95211640278a5c9733b68475ce1f2656a4c84cce27e0f628fd6df2e57d268" gracePeriod=30
Jul 28 23:03:00.170899 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.170838    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm"
Jul 28 23:03:00.256852 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.256823    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3113fbc7-9efd-414f-9539-e603455f0de6-plugins-config-volume\") pod \"3113fbc7-9efd-414f-9539-e603455f0de6\" (UID: \"3113fbc7-9efd-414f-9539-e603455f0de6\") "
Jul 28 23:03:00.257020 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.256867    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzdn2\" (UniqueName: \"kubernetes.io/projected/3113fbc7-9efd-414f-9539-e603455f0de6-kube-api-access-vzdn2\") pod \"3113fbc7-9efd-414f-9539-e603455f0de6\" (UID: \"3113fbc7-9efd-414f-9539-e603455f0de6\") "
Jul 28 23:03:00.257232 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.257209    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3113fbc7-9efd-414f-9539-e603455f0de6-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3113fbc7-9efd-414f-9539-e603455f0de6" (UID: "3113fbc7-9efd-414f-9539-e603455f0de6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:03:00.259145 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.259116    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3113fbc7-9efd-414f-9539-e603455f0de6-kube-api-access-vzdn2" (OuterVolumeSpecName: "kube-api-access-vzdn2") pod "3113fbc7-9efd-414f-9539-e603455f0de6" (UID: "3113fbc7-9efd-414f-9539-e603455f0de6"). InnerVolumeSpecName "kube-api-access-vzdn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:03:00.259332 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.259309    2551 generic.go:356] "Generic (PLEG): container finished" podID="3113fbc7-9efd-414f-9539-e603455f0de6" containerID="97d95211640278a5c9733b68475ce1f2656a4c84cce27e0f628fd6df2e57d268" exitCode=0
Jul 28 23:03:00.259411 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.259356    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm"
Jul 28 23:03:00.259411 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.259397    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm" event={"ID":"3113fbc7-9efd-414f-9539-e603455f0de6","Type":"ContainerDied","Data":"97d95211640278a5c9733b68475ce1f2656a4c84cce27e0f628fd6df2e57d268"}
Jul 28 23:03:00.259501 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.259455    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm" event={"ID":"3113fbc7-9efd-414f-9539-e603455f0de6","Type":"ContainerDied","Data":"800351bc553e6661f86dc52fadd645d187cdaf3b488f4e16c06b21d3d4f84b75"}
Jul 28 23:03:00.259501 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.259488    2551 scope.go:117] "RemoveContainer" containerID="97d95211640278a5c9733b68475ce1f2656a4c84cce27e0f628fd6df2e57d268"
Jul 28 23:03:00.261290 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.261257    2551 generic.go:356] "Generic (PLEG): container finished" podID="a2e70786-1610-43cf-8ba7-230457c157bc" containerID="630be4775db23ede049013b654caa1e862f3ce46e144a1577d82407c9496fad3" exitCode=0
Jul 28 23:03:00.261383 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.261294    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh" event={"ID":"a2e70786-1610-43cf-8ba7-230457c157bc","Type":"ContainerDied","Data":"630be4775db23ede049013b654caa1e862f3ce46e144a1577d82407c9496fad3"}
Jul 28 23:03:00.261383 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.261329    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"
Jul 28 23:03:00.261383 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.261338    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh" event={"ID":"a2e70786-1610-43cf-8ba7-230457c157bc","Type":"ContainerDied","Data":"ae6648726c5c38ecb9887712ce7d4161e590e4fd95d4aecf7f79ce82ef3e22ba"}
Jul 28 23:03:00.277883 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.277863    2551 scope.go:117] "RemoveContainer" containerID="97d95211640278a5c9733b68475ce1f2656a4c84cce27e0f628fd6df2e57d268"
Jul 28 23:03:00.278185 ip-10-0-141-102 kubenswrapper[2551]: E0728 23:03:00.278165    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97d95211640278a5c9733b68475ce1f2656a4c84cce27e0f628fd6df2e57d268\": container with ID starting with 97d95211640278a5c9733b68475ce1f2656a4c84cce27e0f628fd6df2e57d268 not found: ID does not exist" containerID="97d95211640278a5c9733b68475ce1f2656a4c84cce27e0f628fd6df2e57d268"
Jul 28 23:03:00.278240 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.278194    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97d95211640278a5c9733b68475ce1f2656a4c84cce27e0f628fd6df2e57d268"} err="failed to get container status \"97d95211640278a5c9733b68475ce1f2656a4c84cce27e0f628fd6df2e57d268\": rpc error: code = NotFound desc = could not find container \"97d95211640278a5c9733b68475ce1f2656a4c84cce27e0f628fd6df2e57d268\": container with ID starting with 97d95211640278a5c9733b68475ce1f2656a4c84cce27e0f628fd6df2e57d268 not found: ID does not exist"
Jul 28 23:03:00.278240 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.278212    2551 scope.go:117] "RemoveContainer" containerID="630be4775db23ede049013b654caa1e862f3ce46e144a1577d82407c9496fad3"
Jul 28 23:03:00.287354 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.287335    2551 scope.go:117] "RemoveContainer" containerID="941070a454982c2fddcf9a95659853d6b41b505b3ac86834c74a95bd12b144e6"
Jul 28 23:03:00.289158 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.289135    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"]
Jul 28 23:03:00.292003 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.291978    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-6fb213-openshift-default-cb8d8f5dd-fxqxh"]
Jul 28 23:03:00.297605 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.297552    2551 scope.go:117] "RemoveContainer" containerID="630be4775db23ede049013b654caa1e862f3ce46e144a1577d82407c9496fad3"
Jul 28 23:03:00.297910 ip-10-0-141-102 kubenswrapper[2551]: E0728 23:03:00.297888    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"630be4775db23ede049013b654caa1e862f3ce46e144a1577d82407c9496fad3\": container with ID starting with 630be4775db23ede049013b654caa1e862f3ce46e144a1577d82407c9496fad3 not found: ID does not exist" containerID="630be4775db23ede049013b654caa1e862f3ce46e144a1577d82407c9496fad3"
Jul 28 23:03:00.298032 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.297914    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"630be4775db23ede049013b654caa1e862f3ce46e144a1577d82407c9496fad3"} err="failed to get container status \"630be4775db23ede049013b654caa1e862f3ce46e144a1577d82407c9496fad3\": rpc error: code = NotFound desc = could not find container \"630be4775db23ede049013b654caa1e862f3ce46e144a1577d82407c9496fad3\": container with ID starting with 630be4775db23ede049013b654caa1e862f3ce46e144a1577d82407c9496fad3 not found: ID does not exist"
Jul 28 23:03:00.298032 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.297933    2551 scope.go:117] "RemoveContainer" containerID="941070a454982c2fddcf9a95659853d6b41b505b3ac86834c74a95bd12b144e6"
Jul 28 23:03:00.298194 ip-10-0-141-102 kubenswrapper[2551]: E0728 23:03:00.298153    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"941070a454982c2fddcf9a95659853d6b41b505b3ac86834c74a95bd12b144e6\": container with ID starting with 941070a454982c2fddcf9a95659853d6b41b505b3ac86834c74a95bd12b144e6 not found: ID does not exist" containerID="941070a454982c2fddcf9a95659853d6b41b505b3ac86834c74a95bd12b144e6"
Jul 28 23:03:00.298194 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.298172    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"941070a454982c2fddcf9a95659853d6b41b505b3ac86834c74a95bd12b144e6"} err="failed to get container status \"941070a454982c2fddcf9a95659853d6b41b505b3ac86834c74a95bd12b144e6\": rpc error: code = NotFound desc = could not find container \"941070a454982c2fddcf9a95659853d6b41b505b3ac86834c74a95bd12b144e6\": container with ID starting with 941070a454982c2fddcf9a95659853d6b41b505b3ac86834c74a95bd12b144e6 not found: ID does not exist"
Jul 28 23:03:00.300848 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.300799    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm"]
Jul 28 23:03:00.302813 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.302793    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-a134df-7b65b4549d-8c4sm"]
Jul 28 23:03:00.358437 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.358401    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3113fbc7-9efd-414f-9539-e603455f0de6-plugins-config-volume\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 23:03:00.358437 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.358430    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzdn2\" (UniqueName: \"kubernetes.io/projected/3113fbc7-9efd-414f-9539-e603455f0de6-kube-api-access-vzdn2\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 23:03:00.463085 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.462993    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3113fbc7-9efd-414f-9539-e603455f0de6" path="/var/lib/kubelet/pods/3113fbc7-9efd-414f-9539-e603455f0de6/volumes"
Jul 28 23:03:00.463502 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:00.463480    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2e70786-1610-43cf-8ba7-230457c157bc" path="/var/lib/kubelet/pods/a2e70786-1610-43cf-8ba7-230457c157bc/volumes"
Jul 28 23:03:03.866481 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:03.866438    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 23:03:17.321252 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:17.321217    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88"]
Jul 28 23:03:17.321706 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:17.321452    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88" podUID="799a6bb7-2cc6-45b5-b019-f0a273f26460" containerName="payload-processing" containerID="cri-o://27689bdcd7abb2cdafaca5c534579560ac201993de3e2ff5ab235ea0ddbd8204" gracePeriod=30
Jul 28 23:03:17.330949 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:17.330918    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26"]
Jul 28 23:03:17.331177 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:17.331153    2551 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26" podUID="6b110cee-435d-470f-bc40-134d389b9b6b" containerName="payload-pre-processing" containerID="cri-o://55c0916d7aefd56fefc0c686519fa5556cdf48064ddc52649110482652206d26" gracePeriod=30
Jul 28 23:03:17.611701 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:17.611627    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88"
Jul 28 23:03:17.614978 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:17.614948    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26"
Jul 28 23:03:17.712471 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:17.712428    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmvld\" (UniqueName: \"kubernetes.io/projected/799a6bb7-2cc6-45b5-b019-f0a273f26460-kube-api-access-zmvld\") pod \"799a6bb7-2cc6-45b5-b019-f0a273f26460\" (UID: \"799a6bb7-2cc6-45b5-b019-f0a273f26460\") "
Jul 28 23:03:17.712685 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:17.712507    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxxj6\" (UniqueName: \"kubernetes.io/projected/6b110cee-435d-470f-bc40-134d389b9b6b-kube-api-access-kxxj6\") pod \"6b110cee-435d-470f-bc40-134d389b9b6b\" (UID: \"6b110cee-435d-470f-bc40-134d389b9b6b\") "
Jul 28 23:03:17.712685 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:17.712572    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/799a6bb7-2cc6-45b5-b019-f0a273f26460-plugins-config-volume\") pod \"799a6bb7-2cc6-45b5-b019-f0a273f26460\" (UID: \"799a6bb7-2cc6-45b5-b019-f0a273f26460\") "
Jul 28 23:03:17.712685 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:17.712649    2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b110cee-435d-470f-bc40-134d389b9b6b-plugins-config-volume\") pod \"6b110cee-435d-470f-bc40-134d389b9b6b\" (UID: \"6b110cee-435d-470f-bc40-134d389b9b6b\") "
Jul 28 23:03:17.713202 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:17.713152    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/799a6bb7-2cc6-45b5-b019-f0a273f26460-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "799a6bb7-2cc6-45b5-b019-f0a273f26460" (UID: "799a6bb7-2cc6-45b5-b019-f0a273f26460"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:03:17.713313 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:17.713201    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b110cee-435d-470f-bc40-134d389b9b6b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6b110cee-435d-470f-bc40-134d389b9b6b" (UID: "6b110cee-435d-470f-bc40-134d389b9b6b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 23:03:17.714820 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:17.714792    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b110cee-435d-470f-bc40-134d389b9b6b-kube-api-access-kxxj6" (OuterVolumeSpecName: "kube-api-access-kxxj6") pod "6b110cee-435d-470f-bc40-134d389b9b6b" (UID: "6b110cee-435d-470f-bc40-134d389b9b6b"). InnerVolumeSpecName "kube-api-access-kxxj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:03:17.714953 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:17.714934    2551 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/799a6bb7-2cc6-45b5-b019-f0a273f26460-kube-api-access-zmvld" (OuterVolumeSpecName: "kube-api-access-zmvld") pod "799a6bb7-2cc6-45b5-b019-f0a273f26460" (UID: "799a6bb7-2cc6-45b5-b019-f0a273f26460"). InnerVolumeSpecName "kube-api-access-zmvld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 23:03:17.813789 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:17.813740    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxxj6\" (UniqueName: \"kubernetes.io/projected/6b110cee-435d-470f-bc40-134d389b9b6b-kube-api-access-kxxj6\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 23:03:17.813789 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:17.813774    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/799a6bb7-2cc6-45b5-b019-f0a273f26460-plugins-config-volume\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 23:03:17.813789 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:17.813783    2551 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6b110cee-435d-470f-bc40-134d389b9b6b-plugins-config-volume\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 23:03:17.813789 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:17.813791    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmvld\" (UniqueName: \"kubernetes.io/projected/799a6bb7-2cc6-45b5-b019-f0a273f26460-kube-api-access-zmvld\") on node \"ip-10-0-141-102.ec2.internal\" DevicePath \"\""
Jul 28 23:03:18.348489 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:18.348456    2551 generic.go:356] "Generic (PLEG): container finished" podID="6b110cee-435d-470f-bc40-134d389b9b6b" containerID="55c0916d7aefd56fefc0c686519fa5556cdf48064ddc52649110482652206d26" exitCode=0
Jul 28 23:03:18.348899 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:18.348507    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26"
Jul 28 23:03:18.348899 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:18.348556    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26" event={"ID":"6b110cee-435d-470f-bc40-134d389b9b6b","Type":"ContainerDied","Data":"55c0916d7aefd56fefc0c686519fa5556cdf48064ddc52649110482652206d26"}
Jul 28 23:03:18.348899 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:18.348635    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26" event={"ID":"6b110cee-435d-470f-bc40-134d389b9b6b","Type":"ContainerDied","Data":"797dab7d1415455e751fe36196d31d7afb0e29e71f54767fb7b750d6fa583a85"}
Jul 28 23:03:18.348899 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:18.348654    2551 scope.go:117] "RemoveContainer" containerID="55c0916d7aefd56fefc0c686519fa5556cdf48064ddc52649110482652206d26"
Jul 28 23:03:18.350114 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:18.350092    2551 generic.go:356] "Generic (PLEG): container finished" podID="799a6bb7-2cc6-45b5-b019-f0a273f26460" containerID="27689bdcd7abb2cdafaca5c534579560ac201993de3e2ff5ab235ea0ddbd8204" exitCode=0
Jul 28 23:03:18.350223 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:18.350125    2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88"
Jul 28 23:03:18.350223 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:18.350183    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88" event={"ID":"799a6bb7-2cc6-45b5-b019-f0a273f26460","Type":"ContainerDied","Data":"27689bdcd7abb2cdafaca5c534579560ac201993de3e2ff5ab235ea0ddbd8204"}
Jul 28 23:03:18.350305 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:18.350234    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88" event={"ID":"799a6bb7-2cc6-45b5-b019-f0a273f26460","Type":"ContainerDied","Data":"20071d41a6fad82bdf38ae52493ebf3e9585fbf655d6eaabf9a63d1e9bb42912"}
Jul 28 23:03:18.357769 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:18.357558    2551 scope.go:117] "RemoveContainer" containerID="55c0916d7aefd56fefc0c686519fa5556cdf48064ddc52649110482652206d26"
Jul 28 23:03:18.357961 ip-10-0-141-102 kubenswrapper[2551]: E0728 23:03:18.357939    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55c0916d7aefd56fefc0c686519fa5556cdf48064ddc52649110482652206d26\": container with ID starting with 55c0916d7aefd56fefc0c686519fa5556cdf48064ddc52649110482652206d26 not found: ID does not exist" containerID="55c0916d7aefd56fefc0c686519fa5556cdf48064ddc52649110482652206d26"
Jul 28 23:03:18.358020 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:18.357972    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55c0916d7aefd56fefc0c686519fa5556cdf48064ddc52649110482652206d26"} err="failed to get container status \"55c0916d7aefd56fefc0c686519fa5556cdf48064ddc52649110482652206d26\": rpc error: code = NotFound desc = could not find container \"55c0916d7aefd56fefc0c686519fa5556cdf48064ddc52649110482652206d26\": container with ID starting with 55c0916d7aefd56fefc0c686519fa5556cdf48064ddc52649110482652206d26 not found: ID does not exist"
Jul 28 23:03:18.358020 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:18.357992    2551 scope.go:117] "RemoveContainer" containerID="27689bdcd7abb2cdafaca5c534579560ac201993de3e2ff5ab235ea0ddbd8204"
Jul 28 23:03:18.366366 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:18.366348    2551 scope.go:117] "RemoveContainer" containerID="27689bdcd7abb2cdafaca5c534579560ac201993de3e2ff5ab235ea0ddbd8204"
Jul 28 23:03:18.366668 ip-10-0-141-102 kubenswrapper[2551]: E0728 23:03:18.366648    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27689bdcd7abb2cdafaca5c534579560ac201993de3e2ff5ab235ea0ddbd8204\": container with ID starting with 27689bdcd7abb2cdafaca5c534579560ac201993de3e2ff5ab235ea0ddbd8204 not found: ID does not exist" containerID="27689bdcd7abb2cdafaca5c534579560ac201993de3e2ff5ab235ea0ddbd8204"
Jul 28 23:03:18.366736 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:18.366676    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27689bdcd7abb2cdafaca5c534579560ac201993de3e2ff5ab235ea0ddbd8204"} err="failed to get container status \"27689bdcd7abb2cdafaca5c534579560ac201993de3e2ff5ab235ea0ddbd8204\": rpc error: code = NotFound desc = could not find container \"27689bdcd7abb2cdafaca5c534579560ac201993de3e2ff5ab235ea0ddbd8204\": container with ID starting with 27689bdcd7abb2cdafaca5c534579560ac201993de3e2ff5ab235ea0ddbd8204 not found: ID does not exist"
Jul 28 23:03:18.371122 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:18.371100    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26"]
Jul 28 23:03:18.374596 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:18.374575    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-faf483-659b7cdcd4-2xj26"]
Jul 28 23:03:18.382931 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:18.382911    2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88"]
Jul 28 23:03:18.387862 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:18.387844    2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-faf483-bb8dcb954-5gx88"]
Jul 28 23:03:18.462264 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:18.462231    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b110cee-435d-470f-bc40-134d389b9b6b" path="/var/lib/kubelet/pods/6b110cee-435d-470f-bc40-134d389b9b6b/volumes"
Jul 28 23:03:18.462602 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:18.462589    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="799a6bb7-2cc6-45b5-b019-f0a273f26460" path="/var/lib/kubelet/pods/799a6bb7-2cc6-45b5-b019-f0a273f26460/volumes"
Jul 28 23:03:20.363848 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:20.363815    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-rztht"]
Jul 28 23:03:24.598812 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:24.598775    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-675879f779-qcpzc_0f53b7be-c8b1-4ee8-b101-2c8b360b287f/authorino/0.log"
Jul 28 23:03:29.916310 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:29.916274    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-2nrlv_6ee65165-d6a4-423b-8595-fc5f815a8041/manager/0.log"
Jul 28 23:03:30.043326 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:30.043296    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-vcdcb_5b429457-3746-4fe4-9991-df23a899369b/manager/0.log"
Jul 28 23:03:30.280400 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:30.280320    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-w2q5c_32913f56-4ade-44a2-a3ff-c137229ddea5/server/0.log"
Jul 28 23:03:30.403618 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:30.403585    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-lfcxw_220c022c-101e-4f2e-8bfc-79514b31a768/manager/0.log"
Jul 28 23:03:30.781979 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:30.781948    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b7ccc5769-hcb2x_1f196092-c49f-41f1-ac37-b92510ba4cd9/manager/0.log"
Jul 28 23:03:30.788354 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:30.788333    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b7ccc5769-hcb2x_1f196092-c49f-41f1-ac37-b92510ba4cd9/copy-manifests/0.log"
Jul 28 23:03:32.833707 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:32.833675    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-675879f779-qcpzc_0f53b7be-c8b1-4ee8-b101-2c8b360b287f/authorino/0.log"
Jul 28 23:03:32.957301 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:32.957264    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-5mgx4_3b6a0f70-a698-484a-b6c9-4f69b5cde601/manager/0.log"
Jul 28 23:03:33.070961 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:33.070938    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-68n4l_7bf1110c-d048-42ce-a459-3c18aadb8c09/manager/0.log"
Jul 28 23:03:33.288967 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:33.288886    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-dd5hz_f6718c70-80d3-43f8-a684-aa08b0ce7a5c/registry-server/0.log"
Jul 28 23:03:33.416044 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:33.416015    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qxkqm_068f572e-5833-41e8-9bc7-0bdfaaecd84b/manager/0.log"
Jul 28 23:03:33.526798 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:33.526766    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-rztht_38118b91-a8ba-47a3-9fb6-bac0869b17ab/limitador/0.log"
Jul 28 23:03:33.651232 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:33.651201    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-7dk88_7947ae81-e535-495f-b66f-72510cf91f65/manager/0.log"
Jul 28 23:03:34.004683 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:34.004609    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6_b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8/istio-proxy/0.log"
Jul 28 23:03:34.356891 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:34.356860    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d4df5cb96-q778h_6fe78ca4-ba3e-4377-accc-d2056694218b/kube-auth-proxy/0.log"
Jul 28 23:03:35.639223 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:35.639191    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-ffjt6_16e277d1-9dd8-42a8-91fe-4f35fde4c4b2/main/0.log"
Jul 28 23:03:35.749720 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:35.749678    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-bt5zm_d0131959-e6ab-4db7-b8dd-8f876c8d3d88/main/0.log"
Jul 28 23:03:44.179643 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:44.179563    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7cn74_be77a0c4-e19b-49d2-b547-c36c3d79dd33/global-pull-secret-syncer/0.log"
Jul 28 23:03:44.360791 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:44.360757    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-sm9g2_934b288b-0f1f-4bd5-a697-d41516122892/konnectivity-agent/0.log"
Jul 28 23:03:44.408429 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:44.408392    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-102.ec2.internal_76decd8d884af4bc21a6a287671d52e8/haproxy/0.log"
Jul 28 23:03:50.301987 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:50.301956    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-675879f779-qcpzc_0f53b7be-c8b1-4ee8-b101-2c8b360b287f/authorino/0.log"
Jul 28 23:03:50.334135 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:50.334108    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-5mgx4_3b6a0f70-a698-484a-b6c9-4f69b5cde601/manager/0.log"
Jul 28 23:03:50.361543 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:50.361508    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-68n4l_7bf1110c-d048-42ce-a459-3c18aadb8c09/manager/0.log"
Jul 28 23:03:50.459463 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:50.459434    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-dd5hz_f6718c70-80d3-43f8-a684-aa08b0ce7a5c/registry-server/0.log"
Jul 28 23:03:50.695735 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:50.695696    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qxkqm_068f572e-5833-41e8-9bc7-0bdfaaecd84b/manager/0.log"
Jul 28 23:03:50.732715 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:50.732681    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-rztht_38118b91-a8ba-47a3-9fb6-bac0869b17ab/limitador/0.log"
Jul 28 23:03:50.848773 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:50.848740    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-7dk88_7947ae81-e535-495f-b66f-72510cf91f65/manager/0.log"
Jul 28 23:03:52.785955 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:52.785921    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-76b4h_ed198125-3b0c-47de-b461-a0b8ee9083e6/node-exporter/0.log"
Jul 28 23:03:52.809875 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:52.809846    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-76b4h_ed198125-3b0c-47de-b461-a0b8ee9083e6/kube-rbac-proxy/0.log"
Jul 28 23:03:52.831407 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:52.831387    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-76b4h_ed198125-3b0c-47de-b461-a0b8ee9083e6/init-textfile/0.log"
Jul 28 23:03:54.168853 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.168807    2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf"]
Jul 28 23:03:54.169359 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.169338    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2e70786-1610-43cf-8ba7-230457c157bc" containerName="istio-proxy"
Jul 28 23:03:54.169451 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.169362    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2e70786-1610-43cf-8ba7-230457c157bc" containerName="istio-proxy"
Jul 28 23:03:54.169451 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.169376    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4adfb636-6b31-47dc-ae6b-136b01cd3f00" containerName="payload-pre-processing"
Jul 28 23:03:54.169451 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.169385    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="4adfb636-6b31-47dc-ae6b-136b01cd3f00" containerName="payload-pre-processing"
Jul 28 23:03:54.169451 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.169407    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbb3b710-7008-4254-8b3a-db86af9bd7f1" containerName="payload-processing"
Jul 28 23:03:54.169451 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.169416    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbb3b710-7008-4254-8b3a-db86af9bd7f1" containerName="payload-processing"
Jul 28 23:03:54.169451 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.169433    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="799a6bb7-2cc6-45b5-b019-f0a273f26460" containerName="payload-processing"
Jul 28 23:03:54.169451 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.169443    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="799a6bb7-2cc6-45b5-b019-f0a273f26460" containerName="payload-processing"
Jul 28 23:03:54.169846 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.169455    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d6777e5-90ff-40a5-a17c-20918daffaab" containerName="payload-pre-processing"
Jul 28 23:03:54.169846 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.169465    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d6777e5-90ff-40a5-a17c-20918daffaab" containerName="payload-pre-processing"
Jul 28 23:03:54.169846 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.169477    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b110cee-435d-470f-bc40-134d389b9b6b" containerName="payload-pre-processing"
Jul 28 23:03:54.169846 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.169487    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b110cee-435d-470f-bc40-134d389b9b6b" containerName="payload-pre-processing"
Jul 28 23:03:54.169846 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.169508    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3113fbc7-9efd-414f-9539-e603455f0de6" containerName="payload-processing"
Jul 28 23:03:54.169846 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.169516    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="3113fbc7-9efd-414f-9539-e603455f0de6" containerName="payload-processing"
Jul 28 23:03:54.169846 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.169623    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbb3b710-7008-4254-8b3a-db86af9bd7f1" containerName="payload-processing"
Jul 28 23:03:54.169846 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.169638    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2e70786-1610-43cf-8ba7-230457c157bc" containerName="istio-proxy"
Jul 28 23:03:54.169846 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.169650    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="4adfb636-6b31-47dc-ae6b-136b01cd3f00" containerName="payload-pre-processing"
Jul 28 23:03:54.169846 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.169661    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="3113fbc7-9efd-414f-9539-e603455f0de6" containerName="payload-processing"
Jul 28 23:03:54.169846 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.169672    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d6777e5-90ff-40a5-a17c-20918daffaab" containerName="payload-pre-processing"
Jul 28 23:03:54.169846 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.169683    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b110cee-435d-470f-bc40-134d389b9b6b" containerName="payload-pre-processing"
Jul 28 23:03:54.169846 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.169692    2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="799a6bb7-2cc6-45b5-b019-f0a273f26460" containerName="payload-processing"
Jul 28 23:03:54.172896 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.172875    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf"
Jul 28 23:03:54.174725 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.174698    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tzj6t\"/\"openshift-service-ca.crt\""
Jul 28 23:03:54.175292 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.175270    2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tzj6t\"/\"default-dockercfg-gslgj\""
Jul 28 23:03:54.175394 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.175291    2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tzj6t\"/\"kube-root-ca.crt\""
Jul 28 23:03:54.178884 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.178859    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf"]
Jul 28 23:03:54.228410 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.228366    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/364f3cca-8064-4e23-8232-7b4d98127dfe-proc\") pod \"perf-node-gather-daemonset-w9mgf\" (UID: \"364f3cca-8064-4e23-8232-7b4d98127dfe\") " pod="openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf"
Jul 28 23:03:54.228410 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.228409    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/364f3cca-8064-4e23-8232-7b4d98127dfe-podres\") pod \"perf-node-gather-daemonset-w9mgf\" (UID: \"364f3cca-8064-4e23-8232-7b4d98127dfe\") " pod="openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf"
Jul 28 23:03:54.228690 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.228444    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x52m\" (UniqueName: \"kubernetes.io/projected/364f3cca-8064-4e23-8232-7b4d98127dfe-kube-api-access-8x52m\") pod \"perf-node-gather-daemonset-w9mgf\" (UID: \"364f3cca-8064-4e23-8232-7b4d98127dfe\") " pod="openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf"
Jul 28 23:03:54.228690 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.228479    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/364f3cca-8064-4e23-8232-7b4d98127dfe-lib-modules\") pod \"perf-node-gather-daemonset-w9mgf\" (UID: \"364f3cca-8064-4e23-8232-7b4d98127dfe\") " pod="openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf"
Jul 28 23:03:54.228690 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.228567    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/364f3cca-8064-4e23-8232-7b4d98127dfe-sys\") pod \"perf-node-gather-daemonset-w9mgf\" (UID: \"364f3cca-8064-4e23-8232-7b4d98127dfe\") " pod="openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf"
Jul 28 23:03:54.329909 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.329806    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/364f3cca-8064-4e23-8232-7b4d98127dfe-lib-modules\") pod \"perf-node-gather-daemonset-w9mgf\" (UID: \"364f3cca-8064-4e23-8232-7b4d98127dfe\") " pod="openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf"
Jul 28 23:03:54.330106 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.329939    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/364f3cca-8064-4e23-8232-7b4d98127dfe-lib-modules\") pod \"perf-node-gather-daemonset-w9mgf\" (UID: \"364f3cca-8064-4e23-8232-7b4d98127dfe\") " pod="openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf"
Jul 28 23:03:54.330106 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.329975    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/364f3cca-8064-4e23-8232-7b4d98127dfe-sys\") pod \"perf-node-gather-daemonset-w9mgf\" (UID: \"364f3cca-8064-4e23-8232-7b4d98127dfe\") " pod="openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf"
Jul 28 23:03:54.330106 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.330060    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/364f3cca-8064-4e23-8232-7b4d98127dfe-proc\") pod \"perf-node-gather-daemonset-w9mgf\" (UID: \"364f3cca-8064-4e23-8232-7b4d98127dfe\") " pod="openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf"
Jul 28 23:03:54.330106 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.330083    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/364f3cca-8064-4e23-8232-7b4d98127dfe-sys\") pod \"perf-node-gather-daemonset-w9mgf\" (UID: \"364f3cca-8064-4e23-8232-7b4d98127dfe\") " pod="openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf"
Jul 28 23:03:54.330106 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.330088    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/364f3cca-8064-4e23-8232-7b4d98127dfe-podres\") pod \"perf-node-gather-daemonset-w9mgf\" (UID: \"364f3cca-8064-4e23-8232-7b4d98127dfe\") " pod="openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf"
Jul 28 23:03:54.330322 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.330127    2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x52m\" (UniqueName: \"kubernetes.io/projected/364f3cca-8064-4e23-8232-7b4d98127dfe-kube-api-access-8x52m\") pod \"perf-node-gather-daemonset-w9mgf\" (UID: \"364f3cca-8064-4e23-8232-7b4d98127dfe\") " pod="openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf"
Jul 28 23:03:54.330322 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.330150    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/364f3cca-8064-4e23-8232-7b4d98127dfe-proc\") pod \"perf-node-gather-daemonset-w9mgf\" (UID: \"364f3cca-8064-4e23-8232-7b4d98127dfe\") " pod="openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf"
Jul 28 23:03:54.330322 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.330183    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/364f3cca-8064-4e23-8232-7b4d98127dfe-podres\") pod \"perf-node-gather-daemonset-w9mgf\" (UID: \"364f3cca-8064-4e23-8232-7b4d98127dfe\") " pod="openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf"
Jul 28 23:03:54.337559 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.337518    2551 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x52m\" (UniqueName: \"kubernetes.io/projected/364f3cca-8064-4e23-8232-7b4d98127dfe-kube-api-access-8x52m\") pod \"perf-node-gather-daemonset-w9mgf\" (UID: \"364f3cca-8064-4e23-8232-7b4d98127dfe\") " pod="openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf"
Jul 28 23:03:54.483611 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.483513    2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf"
Jul 28 23:03:54.612004 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:54.611940    2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf"]
Jul 28 23:03:54.614312 ip-10-0-141-102 kubenswrapper[2551]: W0728 23:03:54.614287    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod364f3cca_8064_4e23_8232_7b4d98127dfe.slice/crio-f0c3f7ab677b24664dd4e34d0053aac06a755ced882abb0fd0e2203c4ab38d78 WatchSource:0}: Error finding container f0c3f7ab677b24664dd4e34d0053aac06a755ced882abb0fd0e2203c4ab38d78: Status 404 returned error can't find the container with id f0c3f7ab677b24664dd4e34d0053aac06a755ced882abb0fd0e2203c4ab38d78
Jul 28 23:03:55.521889 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:55.521852    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf" event={"ID":"364f3cca-8064-4e23-8232-7b4d98127dfe","Type":"ContainerStarted","Data":"79728674d68cec449950cd80fa9161d05401c51719a88d2d8312fd5a220b32b0"}
Jul 28 23:03:55.522342 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:55.521904    2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf" event={"ID":"364f3cca-8064-4e23-8232-7b4d98127dfe","Type":"ContainerStarted","Data":"f0c3f7ab677b24664dd4e34d0053aac06a755ced882abb0fd0e2203c4ab38d78"}
Jul 28 23:03:55.522342 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:55.521932    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf"
Jul 28 23:03:55.535303 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:55.535249    2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf" podStartSLOduration=1.5352339430000002 podStartE2EDuration="1.535233943s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 23:03:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 23:03:55.535197535 +0000 UTC m=+3243.553163230" watchObservedRunningTime="2026-07-28 23:03:55.535233943 +0000 UTC m=+3243.553199637"
Jul 28 23:03:57.519793 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:57.519763    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x8fct_31f5aa14-3e3d-445b-b646-1ac37987f977/dns/0.log"
Jul 28 23:03:57.540018 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:57.539987    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-x8fct_31f5aa14-3e3d-445b-b646-1ac37987f977/kube-rbac-proxy/0.log"
Jul 28 23:03:57.591635 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:57.591610    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-trrlj_03112039-98fd-478f-b873-2504ce8aae02/dns-node-resolver/0.log"
Jul 28 23:03:58.172387 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:58.172353    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-55dcc5fc8-nr27g_e6b58763-25f6-42cf-ab80-50b19539628b/registry/0.log"
Jul 28 23:03:58.196348 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:58.196317    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-25fc9_9c7c65c8-313a-4e85-9c0b-4c190cf1cbea/node-ca/0.log"
Jul 28 23:03:59.191276 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:59.191245    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6_b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8/istio-proxy/0.log"
Jul 28 23:03:59.399790 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:03:59.399762    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d4df5cb96-q778h_6fe78ca4-ba3e-4377-accc-d2056694218b/kube-auth-proxy/0.log"
Jul 28 23:04:00.403750 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:04:00.403713    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-r6wpx_0ff7f02a-ec84-4e1c-b29d-65063a1dd278/serve-healthcheck-canary/0.log"
Jul 28 23:04:00.989480 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:04:00.989446    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nmfk9_511a4d5c-39cf-4a64-971d-1ee43a729c90/kube-rbac-proxy/0.log"
Jul 28 23:04:01.009147 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:04:01.009122    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nmfk9_511a4d5c-39cf-4a64-971d-1ee43a729c90/exporter/0.log"
Jul 28 23:04:01.030802 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:04:01.030775    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nmfk9_511a4d5c-39cf-4a64-971d-1ee43a729c90/extractor/0.log"
Jul 28 23:04:01.536748 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:04:01.536708    2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tzj6t/perf-node-gather-daemonset-w9mgf"
Jul 28 23:04:03.454698 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:04:03.454661    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-2nrlv_6ee65165-d6a4-423b-8595-fc5f815a8041/manager/0.log"
Jul 28 23:04:03.856325 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:04:03.856290    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-vcdcb_5b429457-3746-4fe4-9991-df23a899369b/manager/0.log"
Jul 28 23:04:04.051637 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:04:04.051607    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-w2q5c_32913f56-4ade-44a2-a3ff-c137229ddea5/server/0.log"
Jul 28 23:04:04.189754 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:04:04.189710    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-lfcxw_220c022c-101e-4f2e-8bfc-79514b31a768/manager/0.log"
Jul 28 23:04:04.438800 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:04:04.438704    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b7ccc5769-hcb2x_1f196092-c49f-41f1-ac37-b92510ba4cd9/manager/0.log"
Jul 28 23:04:04.455005 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:04:04.454980    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b7ccc5769-hcb2x_1f196092-c49f-41f1-ac37-b92510ba4cd9/copy-manifests/0.log"
Jul 28 23:04:04.730718 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:04:04.730639    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6_b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8/istio-proxy/0.log"
Jul 28 23:04:04.905943 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:04:04.905910    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d4df5cb96-q778h_6fe78ca4-ba3e-4377-accc-d2056694218b/kube-auth-proxy/0.log"
Jul 28 23:04:06.469328 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:04:06.469288    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-hrbp6_b1b84c4f-f3ba-41ed-92d6-1c58aa9dadc8/istio-proxy/0.log"
Jul 28 23:04:06.640739 ip-10-0-141-102 kubenswrapper[2551]: I0728 23:04:06.640706    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5d4df5cb96-q778h_6fe78ca4-ba3e-4377-accc-d2056694218b/kube-auth-proxy/0.log"