Jul 10 15:55:10.912649 ip-10-0-129-101 systemd[1]: Starting Kubernetes Kubelet... Jul 10 15:55:11.322610 ip-10-0-129-101 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 15:55:11.322610 ip-10-0-129-101 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 10 15:55:11.322610 ip-10-0-129-101 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 15:55:11.323235 ip-10-0-129-101 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 10 15:55:11.323235 ip-10-0-129-101 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 10 15:55:11.323235 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.322686 2560 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 10 15:55:11.326182 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326167 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 15:55:11.326182 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326182 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 15:55:11.326240 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326186 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 15:55:11.326240 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326189 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 15:55:11.326240 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326192 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 15:55:11.326240 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326195 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 15:55:11.326240 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326198 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 15:55:11.326240 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326201 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 15:55:11.326240 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326203 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 15:55:11.326240 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326206 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 15:55:11.326240 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326213 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 15:55:11.326240 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326217 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 15:55:11.326240 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326220 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 15:55:11.326240 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326223 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 15:55:11.326240 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326225 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 15:55:11.326240 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326228 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 15:55:11.326240 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326231 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 15:55:11.326240 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326233 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 15:55:11.326240 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326235 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 15:55:11.326240 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326238 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 15:55:11.326240 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326240 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 15:55:11.326687 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326243 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 15:55:11.326687 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326245 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 15:55:11.326687 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326248 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 15:55:11.326687 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326251 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 15:55:11.326687 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326253 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 15:55:11.326687 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326256 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 15:55:11.326687 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326258 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 15:55:11.326687 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326261 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 15:55:11.326687 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326264 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 15:55:11.326687 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326266 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 15:55:11.326687 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326268 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 15:55:11.326687 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326271 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 15:55:11.326687 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326274 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 15:55:11.326687 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326276 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 15:55:11.326687 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326279 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 15:55:11.326687 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326281 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 15:55:11.326687 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326285 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 15:55:11.326687 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326289 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 15:55:11.326687 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326292 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 15:55:11.327172 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326294 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 15:55:11.327172 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326297 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 15:55:11.327172 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326301 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 15:55:11.327172 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326303 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 15:55:11.327172 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326306 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 15:55:11.327172 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326309 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 15:55:11.327172 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326311 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 15:55:11.327172 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326314 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 15:55:11.327172 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326316 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 15:55:11.327172 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326319 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 15:55:11.327172 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326321 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 15:55:11.327172 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326323 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 15:55:11.327172 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326326 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 15:55:11.327172 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326328 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 15:55:11.327172 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326331 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 15:55:11.327172 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326334 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 15:55:11.327172 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326337 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 10 15:55:11.327172 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326339 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 15:55:11.327172 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326342 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 15:55:11.327172 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326344 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 15:55:11.327669 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326347 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 15:55:11.327669 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326349 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 15:55:11.327669 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326352 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 15:55:11.327669 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326354 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 15:55:11.327669 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326356 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 15:55:11.327669 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326359 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 15:55:11.327669 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326361 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 15:55:11.327669 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326364 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 15:55:11.327669 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326366 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 15:55:11.327669 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326368 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 15:55:11.327669 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326371 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 15:55:11.327669 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326374 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 15:55:11.327669 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326377 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 15:55:11.327669 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326379 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 15:55:11.327669 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326382 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 15:55:11.327669 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326384 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 15:55:11.327669 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326387 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 15:55:11.327669 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326390 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 15:55:11.327669 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326392 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 15:55:11.327669 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326395 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 15:55:11.328162 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326398 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 15:55:11.328162 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326401 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 15:55:11.328162 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326403 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 15:55:11.328162 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326406 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 15:55:11.328162 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326408 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 15:55:11.328162 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326411 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 15:55:11.328162 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326414 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 15:55:11.328162 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326416 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 15:55:11.328162 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326419 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 15:55:11.328162 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326421 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 15:55:11.328162 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326424 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 15:55:11.328162 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326426 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 15:55:11.328162 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326430 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 15:55:11.328162 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326433 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 15:55:11.328162 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326437 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 15:55:11.328162 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326440 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 15:55:11.328162 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326444 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 15:55:11.328162 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326447 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 15:55:11.328162 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.326450 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 15:55:11.329795 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329782 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 15:55:11.329795 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329794 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 15:55:11.329854 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329797 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 15:55:11.329854 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329800 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 15:55:11.329854 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329803 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 15:55:11.329854 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329811 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 15:55:11.329854 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329814 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 15:55:11.329854 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329817 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 15:55:11.329854 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329820 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 15:55:11.329854 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329823 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 15:55:11.329854 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329825 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 15:55:11.329854 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329828 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 15:55:11.329854 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329830 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 15:55:11.329854 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329833 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 15:55:11.329854 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329835 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 15:55:11.329854 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329838 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 15:55:11.329854 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329841 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 15:55:11.329854 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329849 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 15:55:11.329854 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329851 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 15:55:11.329854 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329854 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 15:55:11.329854 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329856 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 15:55:11.330347 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329859 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 15:55:11.330347 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329862 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 15:55:11.330347 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329865 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 15:55:11.330347 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329869 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 15:55:11.330347 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329871 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 15:55:11.330347 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329876 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 15:55:11.330347 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329879 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 15:55:11.330347 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329882 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 15:55:11.330347 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329885 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 15:55:11.330347 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329887 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 15:55:11.330347 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329890 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 15:55:11.330347 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329892 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 15:55:11.330347 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329894 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 10 15:55:11.330347 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329897 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 15:55:11.330347 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329899 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 15:55:11.330347 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329902 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 15:55:11.330347 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329905 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 15:55:11.330347 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329908 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 15:55:11.330347 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329910 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 15:55:11.330347 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329913 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 15:55:11.330812 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329915 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 15:55:11.330812 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329918 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 15:55:11.330812 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329920 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 15:55:11.330812 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329922 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 15:55:11.330812 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329925 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 15:55:11.330812 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329927 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 15:55:11.330812 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329929 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 15:55:11.330812 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329932 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 15:55:11.330812 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329934 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 15:55:11.330812 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329937 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 15:55:11.330812 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329939 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 15:55:11.330812 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329941 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 15:55:11.330812 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329944 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 15:55:11.330812 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329961 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 15:55:11.330812 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329964 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 15:55:11.330812 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329966 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 15:55:11.330812 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329969 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 15:55:11.330812 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329971 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 15:55:11.330812 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329974 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 15:55:11.330812 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329976 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 15:55:11.331331 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329979 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 15:55:11.331331 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329982 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 15:55:11.331331 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329984 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 15:55:11.331331 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329987 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 15:55:11.331331 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329989 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 15:55:11.331331 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329992 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 15:55:11.331331 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329994 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 15:55:11.331331 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.329997 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 15:55:11.331331 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330001 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 15:55:11.331331 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330005 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 15:55:11.331331 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330014 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 15:55:11.331331 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330017 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 15:55:11.331331 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330019 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 15:55:11.331331 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330021 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 15:55:11.331331 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330024 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 15:55:11.331331 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330026 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 15:55:11.331331 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330031 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 15:55:11.331331 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330034 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 15:55:11.331331 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330037 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 15:55:11.331788 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330040 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 15:55:11.331788 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330044 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 15:55:11.331788 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330047 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 15:55:11.331788 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330049 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 15:55:11.331788 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330052 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 15:55:11.331788 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330055 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 15:55:11.331788 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330058 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 15:55:11.331788 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330061 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 15:55:11.331788 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330064 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 15:55:11.331788 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330067 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 15:55:11.331788 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330069 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 15:55:11.331788 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330072 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 15:55:11.331788 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330074 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 15:55:11.331788 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330077 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 15:55:11.331788 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330080 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 15:55:11.331788 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330082 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 15:55:11.331788 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330084 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 15:55:11.331788 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330087 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 15:55:11.331788 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330089 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 15:55:11.332317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330462 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 10 15:55:11.332317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330467 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 15:55:11.332317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330469 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 15:55:11.332317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330472 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 15:55:11.332317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330475 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 15:55:11.332317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330478 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 15:55:11.332317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330480 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 15:55:11.332317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330483 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 15:55:11.332317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330486 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 15:55:11.332317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330491 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 15:55:11.332317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330494 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 15:55:11.332317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330497 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 15:55:11.332317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330500 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 15:55:11.332317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330503 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 15:55:11.332317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330506 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 15:55:11.332317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330508 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 15:55:11.332317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330512 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 15:55:11.332317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330516 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 15:55:11.332317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330519 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 15:55:11.332767 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330522 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 15:55:11.332767 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330525 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 15:55:11.332767 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330528 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 15:55:11.332767 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330531 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 15:55:11.332767 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330533 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 15:55:11.332767 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330536 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 15:55:11.332767 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330538 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 15:55:11.332767 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330540 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 15:55:11.332767 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330543 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 15:55:11.332767 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330546 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 15:55:11.332767 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330548 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 15:55:11.332767 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330551 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 15:55:11.332767 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330554 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 15:55:11.332767 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330556 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 15:55:11.332767 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330559 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 15:55:11.332767 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330561 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 15:55:11.332767 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330564 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 15:55:11.332767 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330566 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 15:55:11.332767 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330569 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 15:55:11.332767 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330572 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 15:55:11.333242 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330575 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 15:55:11.333242 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330578 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 15:55:11.333242 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330581 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 15:55:11.333242 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330583 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 15:55:11.333242 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330586 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 15:55:11.333242 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330588 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 15:55:11.333242 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330591 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 15:55:11.333242 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330593 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 15:55:11.333242 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330596 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 15:55:11.333242 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330598 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 15:55:11.333242 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330601 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 15:55:11.333242 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330603 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 15:55:11.333242 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330606 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 15:55:11.333242 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330609 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 15:55:11.333242 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330611 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 15:55:11.333242 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330614 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 15:55:11.333242 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330616 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 15:55:11.333242 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330618 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 15:55:11.333242 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330621 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 15:55:11.333242 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330623 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 15:55:11.333700 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330626 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 15:55:11.333700 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330628 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 15:55:11.333700 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330631 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 15:55:11.333700 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330634 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 15:55:11.333700 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330638 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 15:55:11.333700 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330640 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 15:55:11.333700 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330643 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 15:55:11.333700 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330645 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 15:55:11.333700 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330649 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 15:55:11.333700 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330651 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 15:55:11.333700 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330654 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 15:55:11.333700 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330657 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 15:55:11.333700 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330659 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 15:55:11.333700 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330662 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 15:55:11.333700 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330665 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 15:55:11.333700 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330667 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 15:55:11.333700 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330669 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 15:55:11.333700 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330672 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 15:55:11.333700 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330674 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 15:55:11.333700 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330677 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 15:55:11.334176 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330679 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 15:55:11.334176 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330681 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 15:55:11.334176 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330684 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 15:55:11.334176 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330689 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 15:55:11.334176 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330692 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 15:55:11.334176 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330694 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 15:55:11.334176 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330697 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 15:55:11.334176 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330699 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 15:55:11.334176 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330701 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 15:55:11.334176 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330704 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 15:55:11.334176 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330706 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 15:55:11.334176 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330709 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 15:55:11.334176 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330711 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 15:55:11.334176 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330714 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 15:55:11.334176 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330716 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 15:55:11.334176 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330719 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 15:55:11.334176 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330725 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 15:55:11.334176 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330728 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 15:55:11.334176 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330730 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 15:55:11.334176 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.330733 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330798 2560 flags.go:64] FLAG: --address="0.0.0.0" Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330805 2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330811 2560 flags.go:64] FLAG: --anonymous-auth="true" Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330816 2560 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330820 2560 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330823 2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330827 2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330831 2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330834 2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330837 2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330841 2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330844 2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330847 2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330849 2560 flags.go:64] FLAG: --cgroup-root="" Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330852 2560 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330856 2560 flags.go:64] FLAG: --client-ca-file="" Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330859 2560 flags.go:64] FLAG: --cloud-provider="external" Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330862 2560 flags.go:64] FLAG: --cluster-dns="[]" Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330866 2560 flags.go:64] FLAG: --cluster-domain="" Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330869 2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330872 2560 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330875 2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330878 2560 flags.go:64] FLAG: --container-log-max-files="5" Jul 10 15:55:11.334642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330882 2560 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330884 2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330887 2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330890 2560 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330893 2560 flags.go:64] FLAG: --contention-profiling="false" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330896 2560 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330899 2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330902 2560 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330905 2560 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330909 2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330911 2560 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330914 2560 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330917 2560 flags.go:64] FLAG: --enable-load-reader="false" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330920 2560 flags.go:64] FLAG: --enable-server="true" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330923 2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330928 2560 flags.go:64] FLAG: --event-burst="100" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330931 2560 flags.go:64] FLAG: --event-qps="50" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330933 2560 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330936 2560 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330939 2560 flags.go:64] FLAG: --eviction-hard="" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330943 2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330945 2560 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330948 2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330965 2560 flags.go:64] FLAG: --eviction-soft="" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330969 2560 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 10 15:55:11.335232 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330972 2560 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330975 2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330978 2560 flags.go:64] FLAG: --experimental-mounter-path="" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330980 2560 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330983 2560 flags.go:64] FLAG: --fail-swap-on="true" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330986 2560 flags.go:64] FLAG: --feature-gates="" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330989 2560 flags.go:64] FLAG: --file-check-frequency="20s" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330992 2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330995 2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.330998 2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331001 2560 flags.go:64] FLAG: --healthz-port="10248" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331004 2560 flags.go:64] FLAG: --help="false" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331007 2560 flags.go:64] FLAG: --hostname-override="ip-10-0-129-101.ec2.internal" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331010 2560 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331013 2560 flags.go:64] FLAG: --http-check-frequency="20s" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331015 2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331019 2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331022 2560 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331025 2560 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331027 2560 flags.go:64] FLAG: --image-service-endpoint="" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331030 2560 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331033 2560 flags.go:64] FLAG: --kube-api-burst="100" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331036 2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331039 2560 flags.go:64] FLAG: --kube-api-qps="50" Jul 10 15:55:11.335862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331042 2560 flags.go:64] FLAG: --kube-reserved="" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331045 2560 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331048 2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331050 2560 flags.go:64] FLAG: --kubelet-cgroups="" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331064 2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331068 2560 flags.go:64] FLAG: --lock-file="" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331070 2560 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331075 2560 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331078 2560 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331083 2560 flags.go:64] FLAG: --log-json-split-stream="false" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331086 2560 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331089 2560 flags.go:64] FLAG: --log-text-split-stream="false" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331110 2560 flags.go:64] FLAG: --logging-format="text" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331113 2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331117 2560 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331120 2560 flags.go:64] FLAG: --manifest-url="" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331122 2560 flags.go:64] FLAG: --manifest-url-header="" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331127 2560 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331130 2560 flags.go:64] FLAG: --max-open-files="1000000" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331133 2560 flags.go:64] FLAG: --max-pods="110" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331137 2560 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331140 2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331142 2560 flags.go:64] FLAG: --memory-manager-policy="None" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331145 2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331148 2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 10 15:55:11.336484 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331151 2560 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331154 2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331161 2560 flags.go:64] FLAG: --node-status-max-images="50" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331164 2560 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331167 2560 flags.go:64] FLAG: --oom-score-adj="-999" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331170 2560 flags.go:64] FLAG: --pod-cidr="" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331174 2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331180 2560 flags.go:64] FLAG: --pod-manifest-path="" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331182 2560 flags.go:64] FLAG: --pod-max-pids="-1" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331185 2560 flags.go:64] FLAG: --pods-per-core="0" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331188 2560 flags.go:64] FLAG: --port="10250" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331191 2560 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331194 2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e98202f7293bd619" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331197 2560 flags.go:64] FLAG: --qos-reserved="" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331200 2560 flags.go:64] FLAG: --read-only-port="10255" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331204 2560 flags.go:64] FLAG: --register-node="true" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331207 2560 flags.go:64] FLAG: --register-with-taints="" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331210 2560 flags.go:64] FLAG: --registry-burst="10" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331213 2560 flags.go:64] FLAG: --registry-qps="5" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331216 2560 flags.go:64] FLAG: --reserved-cpus="" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331218 2560 flags.go:64] FLAG: --reserved-memory="" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331222 2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331225 2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331228 2560 flags.go:64] FLAG: --rotate-certificates="false" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331231 2560 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 10 15:55:11.337107 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331234 2560 flags.go:64] FLAG: --runonce="false" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331237 2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331240 2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331243 2560 flags.go:64] FLAG: --seccomp-default="false" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331246 2560 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331249 2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331252 2560 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331255 2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331258 2560 flags.go:64] FLAG: --storage-driver-password="root" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331261 2560 flags.go:64] FLAG: --storage-driver-secure="false" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331264 2560 flags.go:64] FLAG: --storage-driver-table="stats" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331267 2560 flags.go:64] FLAG: --storage-driver-user="root" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331270 2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331273 2560 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331276 2560 flags.go:64] FLAG: --system-cgroups="" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331279 2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331284 2560 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331287 2560 flags.go:64] FLAG: --tls-cert-file="" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331289 2560 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331293 2560 flags.go:64] FLAG: --tls-min-version="" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331296 2560 flags.go:64] FLAG: --tls-private-key-file="" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331299 2560 flags.go:64] FLAG: --topology-manager-policy="none" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331304 2560 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331307 2560 flags.go:64] FLAG: --topology-manager-scope="container" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331309 2560 flags.go:64] FLAG: --v="2" Jul 10 15:55:11.337687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331313 2560 flags.go:64] FLAG: --version="false" Jul 10 15:55:11.338281 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331317 2560 flags.go:64] FLAG: --vmodule="" Jul 10 15:55:11.338281 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331320 2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 10 15:55:11.338281 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331324 2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 10 15:55:11.338281 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331440 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 15:55:11.338281 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331444 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 15:55:11.338281 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331447 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 15:55:11.338281 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331451 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 15:55:11.338281 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331454 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 15:55:11.338281 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331458 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 15:55:11.338281 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331461 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 15:55:11.338281 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331463 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 15:55:11.338281 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331466 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 15:55:11.338281 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331468 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 15:55:11.338281 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331471 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 15:55:11.338281 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331473 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 15:55:11.338281 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331476 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 15:55:11.338281 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331478 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 15:55:11.338281 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331481 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 15:55:11.338281 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331483 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 15:55:11.338281 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331486 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 15:55:11.338946 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331488 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 15:55:11.338946 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331491 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 15:55:11.338946 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331494 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 15:55:11.338946 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331496 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 15:55:11.338946 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331499 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 15:55:11.338946 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331501 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 15:55:11.338946 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331504 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 15:55:11.338946 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331507 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 15:55:11.338946 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331512 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 15:55:11.338946 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331515 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 15:55:11.338946 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331518 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 15:55:11.338946 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331521 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 15:55:11.338946 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331524 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 15:55:11.338946 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331526 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 10 15:55:11.338946 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331529 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 15:55:11.338946 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331532 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 15:55:11.338946 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331534 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 15:55:11.338946 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331537 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 15:55:11.338946 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331539 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 15:55:11.339764 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331542 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 15:55:11.339764 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331545 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 15:55:11.339764 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331548 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 15:55:11.339764 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331551 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 15:55:11.339764 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331553 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 15:55:11.339764 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331556 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 15:55:11.339764 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331559 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 15:55:11.339764 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331561 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 15:55:11.339764 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331564 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 15:55:11.339764 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331566 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 15:55:11.339764 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331570 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 15:55:11.339764 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331575 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 15:55:11.339764 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331578 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 15:55:11.339764 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331581 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 15:55:11.339764 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331584 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 15:55:11.339764 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331588 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 15:55:11.339764 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331591 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 15:55:11.339764 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331594 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 15:55:11.339764 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331597 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 15:55:11.340267 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331599 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 15:55:11.340267 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331602 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 15:55:11.340267 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331607 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 15:55:11.340267 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331609 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 15:55:11.340267 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331612 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 15:55:11.340267 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331614 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 15:55:11.340267 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331617 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 15:55:11.340267 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331620 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 15:55:11.340267 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331622 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 15:55:11.340267 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331625 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 15:55:11.340267 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331627 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 15:55:11.340267 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331630 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 15:55:11.340267 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331632 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 15:55:11.340267 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331635 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 15:55:11.340267 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331639 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 15:55:11.340267 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331641 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 15:55:11.340267 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331644 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 15:55:11.340267 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331646 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 15:55:11.340267 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331649 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 15:55:11.340267 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331651 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 15:55:11.340808 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331653 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 15:55:11.340808 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331656 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 15:55:11.340808 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331658 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 15:55:11.340808 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331661 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 15:55:11.340808 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331663 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 15:55:11.340808 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331666 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 15:55:11.340808 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331669 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 15:55:11.340808 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331671 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 15:55:11.340808 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331673 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 15:55:11.340808 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331677 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 15:55:11.340808 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331680 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 15:55:11.340808 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331682 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 15:55:11.340808 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331685 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 15:55:11.340808 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331687 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 15:55:11.340808 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331691 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 15:55:11.340808 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331694 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 15:55:11.340808 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331697 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 15:55:11.340808 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331699 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 15:55:11.340808 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331702 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 15:55:11.341271 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331704 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 15:55:11.341271 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331707 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 15:55:11.341271 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331709 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 15:55:11.341271 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331712 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 15:55:11.341271 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.331714 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 15:55:11.341271 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.331719 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 15:55:11.341271 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.338321 2560 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 10 15:55:11.341271 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.338336 2560 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 10 15:55:11.341271 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.338355 2560 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 10 15:55:11.341271 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.338360 2560 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 10 15:55:11.341271 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339551 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 15:55:11.341271 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339564 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 15:55:11.341271 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339569 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 15:55:11.341271 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339574 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 15:55:11.341271 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339579 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 15:55:11.341641 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339583 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 15:55:11.341641 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339587 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 15:55:11.341641 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339592 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 15:55:11.341641 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339596 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 15:55:11.341641 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339606 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 15:55:11.341641 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339611 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 15:55:11.341641 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339615 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 15:55:11.341641 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339620 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 15:55:11.341641 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339624 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 15:55:11.341641 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339629 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 15:55:11.341641 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339634 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 15:55:11.341641 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339639 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 15:55:11.341641 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339643 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 15:55:11.341641 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339648 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 15:55:11.341641 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339652 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 15:55:11.341641 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339657 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 15:55:11.341641 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339661 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 15:55:11.341641 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339670 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 15:55:11.341641 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339676 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 15:55:11.341641 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339681 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 15:55:11.342127 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339685 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 15:55:11.342127 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339690 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 15:55:11.342127 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339695 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 15:55:11.342127 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339699 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 15:55:11.342127 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339706 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 15:55:11.342127 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339711 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 15:55:11.342127 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339715 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 15:55:11.342127 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339720 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 15:55:11.342127 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339727 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 15:55:11.342127 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339738 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 15:55:11.342127 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339743 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 15:55:11.342127 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339747 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 15:55:11.342127 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339754 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 15:55:11.342127 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339760 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 15:55:11.342127 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339765 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 15:55:11.342127 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339769 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 15:55:11.342127 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339774 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 15:55:11.342127 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339779 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 15:55:11.342127 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339784 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 15:55:11.342573 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339791 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 15:55:11.342573 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339797 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 15:55:11.342573 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339807 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 15:55:11.342573 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339811 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 15:55:11.342573 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339814 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 15:55:11.342573 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339817 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 15:55:11.342573 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339820 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 15:55:11.342573 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339824 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 15:55:11.342573 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339826 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 15:55:11.342573 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339829 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 15:55:11.342573 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339832 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 15:55:11.342573 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339835 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 15:55:11.342573 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339838 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 10 15:55:11.342573 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339841 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 15:55:11.342573 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339846 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 15:55:11.342573 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339849 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 15:55:11.342573 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339852 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 15:55:11.342573 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339855 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 15:55:11.342573 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339876 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 15:55:11.342573 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339893 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 15:55:11.343041 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339907 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 15:55:11.343041 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339912 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 15:55:11.343041 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339916 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 15:55:11.343041 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339919 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 15:55:11.343041 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339923 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 15:55:11.343041 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.339937 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 15:55:11.343041 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340077 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 15:55:11.343041 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340081 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 15:55:11.343041 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340084 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 15:55:11.343041 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340087 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 15:55:11.343041 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340090 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 15:55:11.343041 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340092 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 15:55:11.343041 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340095 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 15:55:11.343041 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340098 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 15:55:11.343041 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340100 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 15:55:11.343041 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340103 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 15:55:11.343041 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340106 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 15:55:11.343041 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340109 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 15:55:11.343041 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340111 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 15:55:11.343041 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340114 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 15:55:11.343555 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340117 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 15:55:11.343555 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340119 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 15:55:11.343555 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340122 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 15:55:11.343555 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340125 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 15:55:11.343555 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340127 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 15:55:11.343555 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340130 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 15:55:11.343555 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340133 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 15:55:11.343555 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340135 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 15:55:11.343555 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340138 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 15:55:11.343555 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340140 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 15:55:11.343555 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340143 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 15:55:11.343555 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340145 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 15:55:11.343555 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340152 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 15:55:11.343555 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340155 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 15:55:11.343555 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340157 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 15:55:11.343924 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.340164 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 15:55:11.343924 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340286 2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 10 15:55:11.343924 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340303 2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 10 15:55:11.343924 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340307 2560 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 10 15:55:11.343924 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340310 2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 10 15:55:11.343924 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340313 2560 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 10 15:55:11.343924 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340316 2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 10 15:55:11.343924 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340318 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 10 15:55:11.343924 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340321 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 10 15:55:11.343924 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340324 2560 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 10 15:55:11.343924 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340326 2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 10 15:55:11.343924 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340329 2560 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 10 15:55:11.343924 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340331 2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 10 15:55:11.343924 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340334 2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 10 15:55:11.343924 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340336 2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 10 15:55:11.344317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340339 2560 feature_gate.go:328] unrecognized feature gate: Example Jul 10 15:55:11.344317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340342 2560 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 10 15:55:11.344317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340344 2560 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 10 15:55:11.344317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340347 2560 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 10 15:55:11.344317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340349 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 10 15:55:11.344317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340351 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 10 15:55:11.344317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340353 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 10 15:55:11.344317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340357 2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 10 15:55:11.344317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340361 2560 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 10 15:55:11.344317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340364 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 10 15:55:11.344317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340367 2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 10 15:55:11.344317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340370 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 10 15:55:11.344317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340372 2560 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 10 15:55:11.344317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340375 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 10 15:55:11.344317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340377 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 10 15:55:11.344317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340380 2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 10 15:55:11.344317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340383 2560 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 10 15:55:11.344317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340385 2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 10 15:55:11.344317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340388 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 10 15:55:11.344317 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340391 2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 10 15:55:11.344784 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340394 2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 10 15:55:11.344784 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340396 2560 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 10 15:55:11.344784 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340398 2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 10 15:55:11.344784 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340401 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 10 15:55:11.344784 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340403 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 10 15:55:11.344784 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340406 2560 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 10 15:55:11.344784 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340409 2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 10 15:55:11.344784 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340411 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 10 15:55:11.344784 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340414 2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 10 15:55:11.344784 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340416 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 10 15:55:11.344784 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340419 2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 10 15:55:11.344784 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340421 2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 10 15:55:11.344784 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340425 2560 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 10 15:55:11.344784 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340429 2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 10 15:55:11.344784 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340432 2560 feature_gate.go:328] unrecognized feature gate: Example2 Jul 10 15:55:11.344784 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340435 2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 10 15:55:11.344784 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340438 2560 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 10 15:55:11.344784 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340442 2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 10 15:55:11.344784 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340445 2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 10 15:55:11.345274 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340448 2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 10 15:55:11.345274 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340450 2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 10 15:55:11.345274 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340453 2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 10 15:55:11.345274 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340455 2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 10 15:55:11.345274 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340458 2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 10 15:55:11.345274 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340461 2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 10 15:55:11.345274 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340463 2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 10 15:55:11.345274 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340466 2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 10 15:55:11.345274 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340468 2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 10 15:55:11.345274 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340471 2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 10 15:55:11.345274 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340474 2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 10 15:55:11.345274 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340476 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 10 15:55:11.345274 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340479 2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 10 15:55:11.345274 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340481 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 10 15:55:11.345274 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340483 2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 10 15:55:11.345274 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340486 2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 10 15:55:11.345274 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340488 2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 10 15:55:11.345274 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340491 2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 10 15:55:11.345274 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340493 2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 10 15:55:11.345274 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340496 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 10 15:55:11.345738 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340498 2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 10 15:55:11.345738 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340501 2560 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 10 15:55:11.345738 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340503 2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 10 15:55:11.345738 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340506 2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 10 15:55:11.345738 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340508 2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 10 15:55:11.345738 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340511 2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 10 15:55:11.345738 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340514 2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 10 15:55:11.345738 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340516 2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 10 15:55:11.345738 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340519 2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 10 15:55:11.345738 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340523 2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 10 15:55:11.345738 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340526 2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 10 15:55:11.345738 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340529 2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 10 15:55:11.345738 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340531 2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 10 15:55:11.345738 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340534 2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 10 15:55:11.345738 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340536 2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 10 15:55:11.345738 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340539 2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 10 15:55:11.345738 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340541 2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 10 15:55:11.345738 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340544 2560 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 10 15:55:11.345738 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340546 2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 10 15:55:11.345738 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340549 2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 10 15:55:11.346202 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340551 2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 10 15:55:11.346202 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340553 2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 10 15:55:11.346202 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340556 2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 10 15:55:11.346202 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340558 2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 10 15:55:11.346202 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340561 2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 10 15:55:11.346202 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:11.340563 2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 10 15:55:11.346202 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.340568 2560 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 10 15:55:11.346202 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.340677 2560 server.go:962] "Client rotation is on, will bootstrap in background" Jul 10 15:55:11.346202 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.343657 2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 10 15:55:11.346202 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.344976 2560 server.go:1019] "Starting client certificate rotation" Jul 10 15:55:11.346202 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.345072 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 15:55:11.346202 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.345111 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 10 15:55:11.347356 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.347343 2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 15:55:11.350137 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.350124 2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 10 15:55:11.358155 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.358136 2560 log.go:25] "Validated CRI v1 runtime API" Jul 10 15:55:11.360040 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.360025 2560 log.go:25] "Validated CRI v1 image API" Jul 10 15:55:11.360811 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.360792 2560 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 10 15:55:11.363180 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.363160 2560 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8ac269e0-3860-4816-bbe6-08f6db217fd6:/dev/nvme0n1p4 dc2713ec-5b85-4b89-bea5-05c7560ad685:/dev/nvme0n1p3] Jul 10 15:55:11.363262 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.363178 2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 10 15:55:11.368383 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.368241 2560 manager.go:216] Machine: {Timestamp:2026-07-10 15:55:11.366591101 +0000 UTC m=+0.338624466 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3193784 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a41bc56fb0f90b3e707dd3927ba38 SystemUUID:ec2a41bc-56fb-0f90-b3e7-07dd3927ba38 BootID:7b1a7a57-9502-46be-b842-17b2a7b78fee Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:86:ab:49:87:cd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:86:ab:49:87:cd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d2:47:2d:4d:b6:8b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 10 15:55:11.368383 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.368370 2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 10 15:55:11.368504 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.368443 2560 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 10 15:55:11.370546 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.370526 2560 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 10 15:55:11.370670 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.370549 2560 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-101.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 10 15:55:11.371382 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.371371 2560 topology_manager.go:138] "Creating topology manager with none policy" Jul 10 15:55:11.371425 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.371384 2560 container_manager_linux.go:309] "Creating device plugin manager" Jul 10 15:55:11.371425 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.371392 2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 15:55:11.372120 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.372110 2560 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 10 15:55:11.373259 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.373251 2560 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 10 15:55:11.373849 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.373834 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 15:55:11.375835 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.375824 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 10 15:55:11.375965 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.375945 2560 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 10 15:55:11.376000 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.375989 2560 kubelet.go:486] "Attempting to sync node with API server" Jul 10 15:55:11.376453 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.376443 2560 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 10 15:55:11.376517 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.376464 2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 10 15:55:11.376517 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.376472 2560 kubelet.go:398] "Adding apiserver pod source" Jul 10 15:55:11.376517 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.376480 2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 10 15:55:11.376835 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.376816 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 15:55:11.376901 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.376850 2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 10 15:55:11.380126 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.380104 2560 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 10 15:55:11.381810 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.381794 2560 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 10 15:55:11.381896 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.381832 2560 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 10 15:55:11.382035 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.382023 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 10 15:55:11.382092 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.382042 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 10 15:55:11.382092 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.382053 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 10 15:55:11.382092 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.382061 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 10 15:55:11.382092 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.382072 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 10 15:55:11.382092 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.382080 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 10 15:55:11.382893 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.382882 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 10 15:55:11.382940 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.382894 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 10 15:55:11.382940 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.382906 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 10 15:55:11.382940 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.382915 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 10 15:55:11.382940 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.382927 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 10 15:55:11.382940 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.382940 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 10 15:55:11.383844 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.383834 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 10 15:55:11.383897 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.383847 2560 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 10 15:55:11.386889 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.386873 2560 server.go:1268] "Started kubelet" Jul 10 15:55:11.387006 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.386971 2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 10 15:55:11.387006 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.386967 2560 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 10 15:55:11.387184 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.387034 2560 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 10 15:55:11.387685 ip-10-0-129-101 systemd[1]: Started Kubernetes Kubelet. Jul 10 15:55:11.388566 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.388494 2560 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 10 15:55:11.389128 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.389113 2560 server.go:310] "Adding debug handlers to kubelet server" Jul 10 15:55:11.390101 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:11.390075 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 10 15:55:11.390101 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:11.390074 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-101.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 10 15:55:11.390213 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.390152 2560 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-101.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 10 15:55:11.393704 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:11.393683 2560 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 10 15:55:11.395010 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.394995 2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 10 15:55:11.395087 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.395013 2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 10 15:55:11.395253 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:11.395209 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found" Jul 10 15:55:11.395253 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:11.394286 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-101.ec2.internal.18c0f8b59f76cba5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-101.ec2.internal,UID:ip-10-0-129-101.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-101.ec2.internal,},FirstTimestamp:2026-07-10 15:55:11.386848165 +0000 UTC m=+0.358881531,LastTimestamp:2026-07-10 15:55:11.386848165 +0000 UTC m=+0.358881531,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-101.ec2.internal,}" Jul 10 15:55:11.395645 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.395628 2560 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 10 15:55:11.395645 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.395628 2560 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 10 15:55:11.395772 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.395656 2560 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 10 15:55:11.396001 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.395848 2560 reconstruct.go:97] "Volume reconstruction finished" Jul 10 15:55:11.396001 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.395861 2560 reconciler.go:29] "Reconciler: start to sync state" Jul 10 15:55:11.396170 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.396154 2560 factory.go:55] Registering systemd factory Jul 10 15:55:11.396224 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.396174 2560 factory.go:223] Registration of the systemd container factory successfully Jul 10 15:55:11.396325 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.396296 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8lzq7" Jul 10 15:55:11.396388 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.396376 2560 factory.go:153] Registering CRI-O factory Jul 10 15:55:11.396439 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.396391 2560 factory.go:223] Registration of the crio container factory successfully Jul 10 15:55:11.396439 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.396436 2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 10 15:55:11.396529 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.396453 2560 factory.go:103] Registering Raw factory Jul 10 15:55:11.396529 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.396463 2560 manager.go:1196] Started watching for new ooms in manager Jul 10 15:55:11.397275 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.397229 2560 manager.go:349] Starting recovery of all containers Jul 10 15:55:11.401482 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:11.401449 2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-101.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 10 15:55:11.401482 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:11.401451 2560 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 10 15:55:11.402391 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.402367 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8lzq7" Jul 10 15:55:11.407756 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.407693 2560 manager.go:354] Recovery completed Jul 10 15:55:11.409664 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:11.409561 2560 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 10 15:55:11.412568 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.412555 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 15:55:11.414323 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.414311 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientMemory" Jul 10 15:55:11.414361 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.414337 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 15:55:11.414361 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.414347 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientPID" Jul 10 15:55:11.414840 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.414827 2560 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 10 15:55:11.414840 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.414838 2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 10 15:55:11.414918 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.414852 2560 state_mem.go:36] "Initialized new in-memory state store" Jul 10 15:55:11.414918 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.414878 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 15:55:11.415613 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.415600 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientMemory" Jul 10 15:55:11.415679 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.415624 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 15:55:11.415679 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.415633 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientPID" Jul 10 15:55:11.416006 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.415991 2560 policy_none.go:49] "None policy: Start" Jul 10 15:55:11.416070 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.416010 2560 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 10 15:55:11.416070 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.416023 2560 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 10 15:55:11.416437 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:11.416378 2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-101.ec2.internal.18c0f8b5a11a0f9c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-101.ec2.internal,UID:ip-10-0-129-101.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-101.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-101.ec2.internal,},FirstTimestamp:2026-07-10 15:55:11.414325148 +0000 UTC m=+0.386358515,LastTimestamp:2026-07-10 15:55:11.414325148 +0000 UTC m=+0.386358515,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-101.ec2.internal,}" Jul 10 15:55:11.417096 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.417086 2560 policy_none.go:47] "Start" Jul 10 15:55:11.453640 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.453626 2560 manager.go:337] "Starting Device Plugin manager" Jul 10 15:55:11.453700 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:11.453659 2560 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 10 15:55:11.453700 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.453672 2560 server.go:87] "Starting device plugin registration server" Jul 10 15:55:11.453901 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.453890 2560 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 10 15:55:11.453967 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.453903 2560 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 10 15:55:11.454169 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.454016 2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 10 15:55:11.454169 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.454095 2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 10 15:55:11.454169 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.454105 2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 10 15:55:11.454668 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:11.454648 2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 10 15:55:11.454755 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:11.454704 2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-101.ec2.internal\" not found" Jul 10 15:55:11.544542 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.544496 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 10 15:55:11.545840 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.545821 2560 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 10 15:55:11.545939 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.545860 2560 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 10 15:55:11.545939 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.545883 2560 kubelet.go:2448] "Starting kubelet main sync loop" Jul 10 15:55:11.545939 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:11.545921 2560 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 10 15:55:11.548683 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.548665 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 15:55:11.554492 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.554479 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 15:55:11.555309 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.555290 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientMemory" Jul 10 15:55:11.555397 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.555329 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 15:55:11.555397 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.555345 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientPID" Jul 10 15:55:11.555397 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.555376 2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-101.ec2.internal" Jul 10 15:55:11.562570 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.562554 2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-101.ec2.internal" Jul 10 15:55:11.562629 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:11.562577 2560 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-101.ec2.internal\": node \"ip-10-0-129-101.ec2.internal\" not found" Jul 10 15:55:11.583811 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:11.583763 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found" Jul 10 15:55:11.647021 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.646996 2560 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-101.ec2.internal"] Jul 10 15:55:11.647120 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.647061 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 15:55:11.648520 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.648499 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientMemory" Jul 10 15:55:11.648620 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.648529 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 15:55:11.648620 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.648543 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientPID" Jul 10 15:55:11.650069 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.650058 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 15:55:11.650151 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.650136 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal" Jul 10 15:55:11.650254 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.650179 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 15:55:11.650800 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.650774 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientMemory" Jul 10 15:55:11.650800 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.650793 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientMemory" Jul 10 15:55:11.650800 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.650800 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 15:55:11.650980 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.650813 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientPID" Jul 10 15:55:11.650980 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.650816 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 15:55:11.650980 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.650827 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientPID" Jul 10 15:55:11.651830 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.651816 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-101.ec2.internal" Jul 10 15:55:11.651901 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.651843 2560 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 10 15:55:11.652492 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.652477 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientMemory" Jul 10 15:55:11.652564 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.652498 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasNoDiskPressure" Jul 10 15:55:11.652564 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.652511 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeHasSufficientPID" Jul 10 15:55:11.676795 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:11.676774 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-101.ec2.internal\" not found" node="ip-10-0-129-101.ec2.internal" Jul 10 15:55:11.680174 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:11.680158 2560 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-101.ec2.internal\" not found" node="ip-10-0-129-101.ec2.internal" Jul 10 15:55:11.684274 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:11.684259 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found" Jul 10 15:55:11.697236 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.697215 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c72755a90b814da38c5f2e7b423a2466-config\") pod \"kube-apiserver-proxy-ip-10-0-129-101.ec2.internal\" (UID: \"c72755a90b814da38c5f2e7b423a2466\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-101.ec2.internal" Jul 10 15:55:11.697320 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.697238 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/239fe417768071dd08b1b2c414e86a71-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal\" (UID: \"239fe417768071dd08b1b2c414e86a71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal" Jul 10 15:55:11.697320 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.697259 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/239fe417768071dd08b1b2c414e86a71-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal\" (UID: \"239fe417768071dd08b1b2c414e86a71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal" Jul 10 15:55:11.785101 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:11.785075 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found" Jul 10 15:55:11.797890 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.797872 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c72755a90b814da38c5f2e7b423a2466-config\") pod \"kube-apiserver-proxy-ip-10-0-129-101.ec2.internal\" (UID: \"c72755a90b814da38c5f2e7b423a2466\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-101.ec2.internal" Jul 10 15:55:11.798028 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.797895 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/239fe417768071dd08b1b2c414e86a71-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal\" (UID: \"239fe417768071dd08b1b2c414e86a71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal" Jul 10 15:55:11.798028 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.797920 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/239fe417768071dd08b1b2c414e86a71-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal\" (UID: \"239fe417768071dd08b1b2c414e86a71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal" Jul 10 15:55:11.798028 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.797947 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/239fe417768071dd08b1b2c414e86a71-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal\" (UID: \"239fe417768071dd08b1b2c414e86a71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal" Jul 10 15:55:11.798028 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.798000 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c72755a90b814da38c5f2e7b423a2466-config\") pod \"kube-apiserver-proxy-ip-10-0-129-101.ec2.internal\" (UID: \"c72755a90b814da38c5f2e7b423a2466\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-101.ec2.internal" Jul 10 15:55:11.798187 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.798074 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/239fe417768071dd08b1b2c414e86a71-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal\" (UID: \"239fe417768071dd08b1b2c414e86a71\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal" Jul 10 15:55:11.885444 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:11.885373 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found" Jul 10 15:55:11.985046 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.985010 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-101.ec2.internal" Jul 10 15:55:11.985187 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:11.985072 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal" Jul 10 15:55:11.986089 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:11.986067 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found" Jul 10 15:55:12.086655 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:12.086624 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found" Jul 10 15:55:12.187243 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:12.187183 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found" Jul 10 15:55:12.287696 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:12.287668 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found" Jul 10 15:55:12.287842 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:12.287788 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 15:55:12.347343 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:12.347315 2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 10 15:55:12.347849 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:12.347442 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 15:55:12.347849 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:12.347452 2560 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 10 15:55:12.388770 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:12.388752 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found" Jul 10 15:55:12.407741 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:12.407711 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-09 15:50:11 +0000 UTC" deadline="2028-01-01 01:58:10.666049725 +0000 UTC" Jul 10 15:55:12.407741 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:12.407735 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12946h2m58.258317602s" Jul 10 15:55:12.489178 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:12.489158 2560 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-101.ec2.internal\" not found" Jul 10 15:55:12.557712 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:12.557684 2560 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 15:55:12.570339 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:12.570318 2560 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 15:55:12.571352 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:12.571330 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc72755a90b814da38c5f2e7b423a2466.slice/crio-b8ef29d45e5be353f66b461fcc87a53c2eb9aed0103fe275e5bf9c1ffc48648b WatchSource:0}: Error finding container b8ef29d45e5be353f66b461fcc87a53c2eb9aed0103fe275e5bf9c1ffc48648b: Status 404 returned error can't find the container with id b8ef29d45e5be353f66b461fcc87a53c2eb9aed0103fe275e5bf9c1ffc48648b Jul 10 15:55:12.575597 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:12.575584 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 15:55:12.596161 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:12.596143 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal" Jul 10 15:55:12.610043 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:12.610027 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 15:55:12.611199 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:12.611187 2560 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-101.ec2.internal" Jul 10 15:55:12.619388 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:12.619373 2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 10 15:55:12.703717 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:12.703692 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod239fe417768071dd08b1b2c414e86a71.slice/crio-63bcd1bdc00493fb6b97f90a657cc38da79a52937a3774ae892e5bcc5cfb0e97 WatchSource:0}: Error finding container 63bcd1bdc00493fb6b97f90a657cc38da79a52937a3774ae892e5bcc5cfb0e97: Status 404 returned error can't find the container with id 63bcd1bdc00493fb6b97f90a657cc38da79a52937a3774ae892e5bcc5cfb0e97 Jul 10 15:55:13.379794 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.379764 2560 apiserver.go:52] "Watching apiserver" Jul 10 15:55:13.389763 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.389730 2560 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 10 15:55:13.390254 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.390223 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-mjxwx","kube-system/global-pull-secret-syncer-h9z25","openshift-image-registry/node-ca-8vfqs","openshift-multus/multus-additional-cni-plugins-cfcr7","openshift-multus/multus-z6k5g","openshift-network-diagnostics/network-check-target-2h276","openshift-network-operator/iptables-alerter-7prkb","openshift-ovn-kubernetes/ovnkube-node-6s2sn","kube-system/konnectivity-agent-8bbgg","kube-system/kube-apiserver-proxy-ip-10-0-129-101.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh","openshift-cluster-node-tuning-operator/tuned-7cvfh","openshift-dns/node-resolver-ngcrw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal"] Jul 10 15:55:13.392953 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.392648 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:13.392953 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:13.392743 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mjxwx" podUID="049e242d-f002-42b0-b116-e9baa5f8eb87" Jul 10 15:55:13.394783 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.394666 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" Jul 10 15:55:13.395160 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.395116 2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 10 15:55:13.396939 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.396915 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8bbgg" Jul 10 15:55:13.397478 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.397445 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 10 15:55:13.397478 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.397463 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5dj4q\"" Jul 10 15:55:13.397627 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.397529 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 10 15:55:13.397627 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.397553 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 10 15:55:13.399273 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.399255 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.399994 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.399568 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-w655z\"" Jul 10 15:55:13.400259 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.400240 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 10 15:55:13.400349 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.400287 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 10 15:55:13.401679 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.401503 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.402358 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.402338 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 10 15:55:13.402454 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.402388 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 10 15:55:13.402617 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.402604 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 10 15:55:13.402692 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.402626 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 10 15:55:13.402906 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.402888 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-646gw\"" Jul 10 15:55:13.402985 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.402946 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 10 15:55:13.403815 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.403796 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:13.403901 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:13.403869 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h276" podUID="ff0e4720-f5ab-4832-b118-eb5fd37dd8ab" Jul 10 15:55:13.404857 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.404801 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 10 15:55:13.405413 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.405171 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 10 15:55:13.405506 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.405413 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlqcb\" (UniqueName: \"kubernetes.io/projected/049e242d-f002-42b0-b116-e9baa5f8eb87-kube-api-access-xlqcb\") pod \"network-metrics-daemon-mjxwx\" (UID: \"049e242d-f002-42b0-b116-e9baa5f8eb87\") " pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:13.405506 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.405448 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c6e830d4-3ff4-4c7d-9066-379949f26ffa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qrpqh\" (UID: \"c6e830d4-3ff4-4c7d-9066-379949f26ffa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" Jul 10 15:55:13.405506 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.405475 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c6e830d4-3ff4-4c7d-9066-379949f26ffa-socket-dir\") pod \"aws-ebs-csi-driver-node-qrpqh\" (UID: \"c6e830d4-3ff4-4c7d-9066-379949f26ffa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" Jul 10 15:55:13.405506 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.405500 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c6e830d4-3ff4-4c7d-9066-379949f26ffa-registration-dir\") pod \"aws-ebs-csi-driver-node-qrpqh\" (UID: \"c6e830d4-3ff4-4c7d-9066-379949f26ffa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" Jul 10 15:55:13.405721 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.405536 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c6e830d4-3ff4-4c7d-9066-379949f26ffa-sys-fs\") pod \"aws-ebs-csi-driver-node-qrpqh\" (UID: \"c6e830d4-3ff4-4c7d-9066-379949f26ffa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" Jul 10 15:55:13.405721 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.405586 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs\") pod \"network-metrics-daemon-mjxwx\" (UID: \"049e242d-f002-42b0-b116-e9baa5f8eb87\") " pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:13.405721 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.405634 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c6e830d4-3ff4-4c7d-9066-379949f26ffa-device-dir\") pod \"aws-ebs-csi-driver-node-qrpqh\" (UID: \"c6e830d4-3ff4-4c7d-9066-379949f26ffa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" Jul 10 15:55:13.405721 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.405663 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c6e830d4-3ff4-4c7d-9066-379949f26ffa-etc-selinux\") pod \"aws-ebs-csi-driver-node-qrpqh\" (UID: \"c6e830d4-3ff4-4c7d-9066-379949f26ffa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" Jul 10 15:55:13.405721 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.405700 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcjl4\" (UniqueName: \"kubernetes.io/projected/c6e830d4-3ff4-4c7d-9066-379949f26ffa-kube-api-access-tcjl4\") pod \"aws-ebs-csi-driver-node-qrpqh\" (UID: \"c6e830d4-3ff4-4c7d-9066-379949f26ffa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" Jul 10 15:55:13.405975 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.405728 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d5c8652b-3f54-4b1d-8bbb-597347aba689-agent-certs\") pod \"konnectivity-agent-8bbgg\" (UID: \"d5c8652b-3f54-4b1d-8bbb-597347aba689\") " pod="kube-system/konnectivity-agent-8bbgg" Jul 10 15:55:13.405975 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.405754 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d5c8652b-3f54-4b1d-8bbb-597347aba689-konnectivity-ca\") pod \"konnectivity-agent-8bbgg\" (UID: \"d5c8652b-3f54-4b1d-8bbb-597347aba689\") " pod="kube-system/konnectivity-agent-8bbgg" Jul 10 15:55:13.405975 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.405864 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 10 15:55:13.406163 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.406110 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 10 15:55:13.406219 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.406200 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 10 15:55:13.406520 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.406326 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 10 15:55:13.406520 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.406339 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xxfdc\"" Jul 10 15:55:13.406687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.406539 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7prkb" Jul 10 15:55:13.409441 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.409183 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.409441 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.409294 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 10 15:55:13.409441 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.409372 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 10 15:55:13.409679 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.409535 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 10 15:55:13.409679 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.409536 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xvvj8\"" Jul 10 15:55:13.409970 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.409951 2560 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 10 15:55:13.411889 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.411868 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-tq4h6\"" Jul 10 15:55:13.412133 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.412116 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8vfqs" Jul 10 15:55:13.412217 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.412135 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 10 15:55:13.415542 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.415502 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:13.415663 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:13.415618 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9z25" podUID="7511b558-f953-4c6b-a90b-c2843281f2cc" Jul 10 15:55:13.416512 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.416486 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 10 15:55:13.416609 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.416589 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 10 15:55:13.416688 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.416668 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4nkn2\"" Jul 10 15:55:13.417178 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.417155 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 10 15:55:13.419388 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.419368 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.422411 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.422388 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 10 15:55:13.422498 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.422484 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-j4m2z\"" Jul 10 15:55:13.422746 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.422733 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 10 15:55:13.423424 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.423405 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ngcrw" Jul 10 15:55:13.426070 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.426029 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 10 15:55:13.426295 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.426278 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 10 15:55:13.427682 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.427665 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-n2rnx\"" Jul 10 15:55:13.460401 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.460289 2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-nljx6" Jul 10 15:55:13.478009 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.477970 2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-nljx6" Jul 10 15:55:13.496759 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.496736 2560 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 10 15:55:13.506041 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506007 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-var-lib-openvswitch\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.506146 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506043 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-etc-openvswitch\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.506146 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506069 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.506146 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506097 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a7a2569d-7313-49a4-b283-aa416f5fdd9e-ovn-node-metrics-cert\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.506309 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506140 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9h4t\" (UniqueName: \"kubernetes.io/projected/a7a2569d-7313-49a4-b283-aa416f5fdd9e-kube-api-access-l9h4t\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.506309 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506215 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlqcb\" (UniqueName: \"kubernetes.io/projected/049e242d-f002-42b0-b116-e9baa5f8eb87-kube-api-access-xlqcb\") pod \"network-metrics-daemon-mjxwx\" (UID: \"049e242d-f002-42b0-b116-e9baa5f8eb87\") " pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:13.506309 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506263 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-sys\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.506454 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506401 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-host-run-netns\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.506504 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506451 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/017c05d8-db4b-427e-b6de-19e1d1264b97-multus-daemon-config\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.506557 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506518 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7511b558-f953-4c6b-a90b-c2843281f2cc-kubelet-config\") pod \"global-pull-secret-syncer-h9z25\" (UID: \"7511b558-f953-4c6b-a90b-c2843281f2cc\") " pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:13.506610 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506556 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/81e1bb75-b227-4f4a-81d4-6843cebccc61-hosts-file\") pod \"node-resolver-ngcrw\" (UID: \"81e1bb75-b227-4f4a-81d4-6843cebccc61\") " pod="openshift-dns/node-resolver-ngcrw" Jul 10 15:55:13.506610 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506584 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92d76a94-65a8-457e-8a57-bc3729beeb80-cnibin\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.506714 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506608 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d5c8652b-3f54-4b1d-8bbb-597347aba689-agent-certs\") pod \"konnectivity-agent-8bbgg\" (UID: \"d5c8652b-3f54-4b1d-8bbb-597347aba689\") " pod="kube-system/konnectivity-agent-8bbgg" Jul 10 15:55:13.506714 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506633 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-host-var-lib-kubelet\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.506714 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506660 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d5c8652b-3f54-4b1d-8bbb-597347aba689-konnectivity-ca\") pod \"konnectivity-agent-8bbgg\" (UID: \"d5c8652b-3f54-4b1d-8bbb-597347aba689\") " pod="kube-system/konnectivity-agent-8bbgg" Jul 10 15:55:13.506714 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506697 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdgtm\" (UniqueName: \"kubernetes.io/projected/81e1bb75-b227-4f4a-81d4-6843cebccc61-kube-api-access-mdgtm\") pod \"node-resolver-ngcrw\" (UID: \"81e1bb75-b227-4f4a-81d4-6843cebccc61\") " pod="openshift-dns/node-resolver-ngcrw" Jul 10 15:55:13.506901 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506721 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/18946076-1464-4aac-bbe1-3c4d2b4b6ae9-serviceca\") pod \"node-ca-8vfqs\" (UID: \"18946076-1464-4aac-bbe1-3c4d2b4b6ae9\") " pod="openshift-image-registry/node-ca-8vfqs" Jul 10 15:55:13.506901 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506773 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-var-lib-kubelet\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.506901 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506798 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtwpc\" (UniqueName: \"kubernetes.io/projected/017c05d8-db4b-427e-b6de-19e1d1264b97-kube-api-access-mtwpc\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.506901 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506825 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a7a2569d-7313-49a4-b283-aa416f5fdd9e-env-overrides\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.506901 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506852 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd8td\" (UniqueName: \"kubernetes.io/projected/78e16a25-054d-4cd6-8578-f101a0e47f77-kube-api-access-cd8td\") pod \"iptables-alerter-7prkb\" (UID: \"78e16a25-054d-4cd6-8578-f101a0e47f77\") " pod="openshift-network-operator/iptables-alerter-7prkb" Jul 10 15:55:13.506901 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506877 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-etc-sysctl-d\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.507208 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506905 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z98ml\" (UniqueName: \"kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml\") pod \"network-check-target-2h276\" (UID: \"ff0e4720-f5ab-4832-b118-eb5fd37dd8ab\") " pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:13.507208 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506944 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-host-var-lib-cni-multus\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.507208 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506960 2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 10 15:55:13.507208 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.506992 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-host-kubelet\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.507208 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507036 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-run-openvswitch\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.507208 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507062 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81e1bb75-b227-4f4a-81d4-6843cebccc61-tmp-dir\") pod \"node-resolver-ngcrw\" (UID: \"81e1bb75-b227-4f4a-81d4-6843cebccc61\") " pod="openshift-dns/node-resolver-ngcrw" Jul 10 15:55:13.507208 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507093 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcjl4\" (UniqueName: \"kubernetes.io/projected/c6e830d4-3ff4-4c7d-9066-379949f26ffa-kube-api-access-tcjl4\") pod \"aws-ebs-csi-driver-node-qrpqh\" (UID: \"c6e830d4-3ff4-4c7d-9066-379949f26ffa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" Jul 10 15:55:13.507208 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507122 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/017c05d8-db4b-427e-b6de-19e1d1264b97-cni-binary-copy\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.507208 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507156 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/92d76a94-65a8-457e-8a57-bc3729beeb80-os-release\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.507634 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507243 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-run\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.507634 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507276 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-multus-cni-dir\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.507634 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507312 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-hostroot\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.507634 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507338 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-host-run-multus-certs\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.507634 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507364 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-node-log\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.507634 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507387 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/18946076-1464-4aac-bbe1-3c4d2b4b6ae9-host\") pod \"node-ca-8vfqs\" (UID: \"18946076-1464-4aac-bbe1-3c4d2b4b6ae9\") " pod="openshift-image-registry/node-ca-8vfqs" Jul 10 15:55:13.507634 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507411 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-etc-modprobe-d\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.507634 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507435 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-etc-systemd\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.507634 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507457 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eecb1845-50d9-4878-8434-a13b10aa2833-tmp\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.507634 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507481 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-host-var-lib-cni-bin\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.507634 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507547 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-etc-sysctl-conf\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.507634 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507569 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92d76a94-65a8-457e-8a57-bc3729beeb80-cni-binary-copy\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.507634 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507592 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92d76a94-65a8-457e-8a57-bc3729beeb80-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.507634 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507609 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qglk6\" (UniqueName: \"kubernetes.io/projected/92d76a94-65a8-457e-8a57-bc3729beeb80-kube-api-access-qglk6\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.508307 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507646 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-host-run-ovn-kubernetes\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.508307 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507690 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d5c8652b-3f54-4b1d-8bbb-597347aba689-konnectivity-ca\") pod \"konnectivity-agent-8bbgg\" (UID: \"d5c8652b-3f54-4b1d-8bbb-597347aba689\") " pod="kube-system/konnectivity-agent-8bbgg" Jul 10 15:55:13.508307 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507722 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c6e830d4-3ff4-4c7d-9066-379949f26ffa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qrpqh\" (UID: \"c6e830d4-3ff4-4c7d-9066-379949f26ffa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" Jul 10 15:55:13.508307 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507783 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v76l8\" (UniqueName: \"kubernetes.io/projected/eecb1845-50d9-4878-8434-a13b10aa2833-kube-api-access-v76l8\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.508307 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507795 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c6e830d4-3ff4-4c7d-9066-379949f26ffa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qrpqh\" (UID: \"c6e830d4-3ff4-4c7d-9066-379949f26ffa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" Jul 10 15:55:13.508307 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507814 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-os-release\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.508307 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507842 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92d76a94-65a8-457e-8a57-bc3729beeb80-system-cni-dir\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.508307 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507894 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/92d76a94-65a8-457e-8a57-bc3729beeb80-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.508307 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507930 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-host-slash\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.508307 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507954 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-host-run-netns\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.508307 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.507975 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-lib-modules\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.508307 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508002 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-host\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.508307 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508027 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/eecb1845-50d9-4878-8434-a13b10aa2833-etc-tuned\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.508307 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508066 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-cnibin\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.508307 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508087 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-multus-socket-dir-parent\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.508307 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508107 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-host-run-k8s-cni-cncf-io\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.509092 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508143 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret\") pod \"global-pull-secret-syncer-h9z25\" (UID: \"7511b558-f953-4c6b-a90b-c2843281f2cc\") " pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:13.509092 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508191 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/92d76a94-65a8-457e-8a57-bc3729beeb80-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.509092 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508222 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c6e830d4-3ff4-4c7d-9066-379949f26ffa-socket-dir\") pod \"aws-ebs-csi-driver-node-qrpqh\" (UID: \"c6e830d4-3ff4-4c7d-9066-379949f26ffa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" Jul 10 15:55:13.509092 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508247 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-etc-sysconfig\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.509092 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508270 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-etc-kubernetes\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.509092 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508315 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-multus-conf-dir\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.509092 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508358 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-run-systemd\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.509092 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508375 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c6e830d4-3ff4-4c7d-9066-379949f26ffa-socket-dir\") pod \"aws-ebs-csi-driver-node-qrpqh\" (UID: \"c6e830d4-3ff4-4c7d-9066-379949f26ffa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" Jul 10 15:55:13.509092 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508385 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-run-ovn\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.509092 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508410 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a7a2569d-7313-49a4-b283-aa416f5fdd9e-ovnkube-script-lib\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.509092 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508435 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/78e16a25-054d-4cd6-8578-f101a0e47f77-iptables-alerter-script\") pod \"iptables-alerter-7prkb\" (UID: \"78e16a25-054d-4cd6-8578-f101a0e47f77\") " pod="openshift-network-operator/iptables-alerter-7prkb" Jul 10 15:55:13.509092 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508462 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs\") pod \"network-metrics-daemon-mjxwx\" (UID: \"049e242d-f002-42b0-b116-e9baa5f8eb87\") " pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:13.509092 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:13.508544 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 15:55:13.509092 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508550 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c6e830d4-3ff4-4c7d-9066-379949f26ffa-device-dir\") pod \"aws-ebs-csi-driver-node-qrpqh\" (UID: \"c6e830d4-3ff4-4c7d-9066-379949f26ffa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" Jul 10 15:55:13.509092 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508660 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c6e830d4-3ff4-4c7d-9066-379949f26ffa-etc-selinux\") pod \"aws-ebs-csi-driver-node-qrpqh\" (UID: \"c6e830d4-3ff4-4c7d-9066-379949f26ffa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" Jul 10 15:55:13.509092 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:13.508678 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs podName:049e242d-f002-42b0-b116-e9baa5f8eb87 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:14.008642451 +0000 UTC m=+2.980675825 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs") pod "network-metrics-daemon-mjxwx" (UID: "049e242d-f002-42b0-b116-e9baa5f8eb87") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 15:55:13.509774 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508699 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7511b558-f953-4c6b-a90b-c2843281f2cc-dbus\") pod \"global-pull-secret-syncer-h9z25\" (UID: \"7511b558-f953-4c6b-a90b-c2843281f2cc\") " pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:13.509774 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508734 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-log-socket\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.509774 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508765 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-host-cni-bin\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.509774 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508768 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c6e830d4-3ff4-4c7d-9066-379949f26ffa-etc-selinux\") pod \"aws-ebs-csi-driver-node-qrpqh\" (UID: \"c6e830d4-3ff4-4c7d-9066-379949f26ffa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" Jul 10 15:55:13.509774 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508793 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c6e830d4-3ff4-4c7d-9066-379949f26ffa-device-dir\") pod \"aws-ebs-csi-driver-node-qrpqh\" (UID: \"c6e830d4-3ff4-4c7d-9066-379949f26ffa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" Jul 10 15:55:13.509774 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508820 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-host-cni-netd\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.509774 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508836 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a7a2569d-7313-49a4-b283-aa416f5fdd9e-ovnkube-config\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.509774 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508853 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c6e830d4-3ff4-4c7d-9066-379949f26ffa-registration-dir\") pod \"aws-ebs-csi-driver-node-qrpqh\" (UID: \"c6e830d4-3ff4-4c7d-9066-379949f26ffa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" Jul 10 15:55:13.509774 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508869 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c6e830d4-3ff4-4c7d-9066-379949f26ffa-sys-fs\") pod \"aws-ebs-csi-driver-node-qrpqh\" (UID: \"c6e830d4-3ff4-4c7d-9066-379949f26ffa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" Jul 10 15:55:13.509774 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508903 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c6e830d4-3ff4-4c7d-9066-379949f26ffa-sys-fs\") pod \"aws-ebs-csi-driver-node-qrpqh\" (UID: \"c6e830d4-3ff4-4c7d-9066-379949f26ffa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" Jul 10 15:55:13.509774 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508902 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-system-cni-dir\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.509774 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508941 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-systemd-units\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.509774 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508948 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c6e830d4-3ff4-4c7d-9066-379949f26ffa-registration-dir\") pod \"aws-ebs-csi-driver-node-qrpqh\" (UID: \"c6e830d4-3ff4-4c7d-9066-379949f26ffa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" Jul 10 15:55:13.509774 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.508971 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/78e16a25-054d-4cd6-8578-f101a0e47f77-host-slash\") pod \"iptables-alerter-7prkb\" (UID: \"78e16a25-054d-4cd6-8578-f101a0e47f77\") " pod="openshift-network-operator/iptables-alerter-7prkb" Jul 10 15:55:13.509774 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.509020 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z29gv\" (UniqueName: \"kubernetes.io/projected/18946076-1464-4aac-bbe1-3c4d2b4b6ae9-kube-api-access-z29gv\") pod \"node-ca-8vfqs\" (UID: \"18946076-1464-4aac-bbe1-3c4d2b4b6ae9\") " pod="openshift-image-registry/node-ca-8vfqs" Jul 10 15:55:13.509774 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.509039 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-etc-kubernetes\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.511255 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.511239 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d5c8652b-3f54-4b1d-8bbb-597347aba689-agent-certs\") pod \"konnectivity-agent-8bbgg\" (UID: \"d5c8652b-3f54-4b1d-8bbb-597347aba689\") " pod="kube-system/konnectivity-agent-8bbgg" Jul 10 15:55:13.532837 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.532810 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcjl4\" (UniqueName: \"kubernetes.io/projected/c6e830d4-3ff4-4c7d-9066-379949f26ffa-kube-api-access-tcjl4\") pod \"aws-ebs-csi-driver-node-qrpqh\" (UID: \"c6e830d4-3ff4-4c7d-9066-379949f26ffa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" Jul 10 15:55:13.536697 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.536674 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlqcb\" (UniqueName: \"kubernetes.io/projected/049e242d-f002-42b0-b116-e9baa5f8eb87-kube-api-access-xlqcb\") pod \"network-metrics-daemon-mjxwx\" (UID: \"049e242d-f002-42b0-b116-e9baa5f8eb87\") " pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:13.552864 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.552811 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal" event={"ID":"239fe417768071dd08b1b2c414e86a71","Type":"ContainerStarted","Data":"63bcd1bdc00493fb6b97f90a657cc38da79a52937a3774ae892e5bcc5cfb0e97"} Jul 10 15:55:13.554310 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.554286 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-101.ec2.internal" event={"ID":"c72755a90b814da38c5f2e7b423a2466","Type":"ContainerStarted","Data":"b8ef29d45e5be353f66b461fcc87a53c2eb9aed0103fe275e5bf9c1ffc48648b"} Jul 10 15:55:13.609408 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609377 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v76l8\" (UniqueName: \"kubernetes.io/projected/eecb1845-50d9-4878-8434-a13b10aa2833-kube-api-access-v76l8\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.609554 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609422 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-os-release\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.609554 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609447 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92d76a94-65a8-457e-8a57-bc3729beeb80-system-cni-dir\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.609554 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609473 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/92d76a94-65a8-457e-8a57-bc3729beeb80-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.609554 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609501 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-host-slash\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.609554 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609526 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-host-run-netns\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.609946 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609542 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-os-release\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.609946 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609551 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-lib-modules\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.609946 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609585 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-host\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.609946 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609624 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-host-slash\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.609946 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609624 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/eecb1845-50d9-4878-8434-a13b10aa2833-etc-tuned\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.609946 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609665 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-cnibin\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.609946 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609690 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-multus-socket-dir-parent\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.609946 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609717 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-host-run-k8s-cni-cncf-io\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.609946 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609742 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret\") pod \"global-pull-secret-syncer-h9z25\" (UID: \"7511b558-f953-4c6b-a90b-c2843281f2cc\") " pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:13.609946 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609749 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-host-run-netns\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.609946 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609750 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-host\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.609946 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609768 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/92d76a94-65a8-457e-8a57-bc3729beeb80-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.609946 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609829 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-etc-sysconfig\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.609946 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609848 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/92d76a94-65a8-457e-8a57-bc3729beeb80-system-cni-dir\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.609946 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609860 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-etc-kubernetes\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.609946 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609890 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-lib-modules\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.610857 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609968 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-multus-conf-dir\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.610857 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.609989 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-multus-socket-dir-parent\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.610857 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.610054 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-etc-sysconfig\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.610857 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.610043 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-etc-kubernetes\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.610857 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.610064 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-run-systemd\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.610857 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.610086 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-cnibin\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.610857 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.610103 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-run-ovn\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.610857 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.610128 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-run-systemd\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.610857 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.610132 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-host-run-k8s-cni-cncf-io\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.610857 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.610175 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a7a2569d-7313-49a4-b283-aa416f5fdd9e-ovnkube-script-lib\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.610857 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.610192 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-multus-conf-dir\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.610857 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.610218 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/78e16a25-054d-4cd6-8578-f101a0e47f77-iptables-alerter-script\") pod \"iptables-alerter-7prkb\" (UID: \"78e16a25-054d-4cd6-8578-f101a0e47f77\") " pod="openshift-network-operator/iptables-alerter-7prkb" Jul 10 15:55:13.610857 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.610276 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-run-ovn\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.610857 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.610349 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7511b558-f953-4c6b-a90b-c2843281f2cc-dbus\") pod \"global-pull-secret-syncer-h9z25\" (UID: \"7511b558-f953-4c6b-a90b-c2843281f2cc\") " pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:13.610857 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.610355 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/92d76a94-65a8-457e-8a57-bc3729beeb80-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.610857 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.610422 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-log-socket\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.610857 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.610492 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-host-cni-bin\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.610857 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:13.610514 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 15:55:13.612052 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:13.610586 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret podName:7511b558-f953-4c6b-a90b-c2843281f2cc nodeName:}" failed. No retries permitted until 2026-07-10 15:55:14.110568281 +0000 UTC m=+3.082601653 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret") pod "global-pull-secret-syncer-h9z25" (UID: "7511b558-f953-4c6b-a90b-c2843281f2cc") : object "kube-system"/"original-pull-secret" not registered Jul 10 15:55:13.612052 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.610508 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-host-cni-netd\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.612052 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611182 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a7a2569d-7313-49a4-b283-aa416f5fdd9e-ovnkube-config\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.612052 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611215 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-system-cni-dir\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.612052 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611237 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-host-cni-bin\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.612052 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611207 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/92d76a94-65a8-457e-8a57-bc3729beeb80-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.612052 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611245 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-systemd-units\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.612052 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611282 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-systemd-units\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.612052 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611304 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/78e16a25-054d-4cd6-8578-f101a0e47f77-host-slash\") pod \"iptables-alerter-7prkb\" (UID: \"78e16a25-054d-4cd6-8578-f101a0e47f77\") " pod="openshift-network-operator/iptables-alerter-7prkb" Jul 10 15:55:13.612052 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611334 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-host-cni-netd\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.612052 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611333 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z29gv\" (UniqueName: \"kubernetes.io/projected/18946076-1464-4aac-bbe1-3c4d2b4b6ae9-kube-api-access-z29gv\") pod \"node-ca-8vfqs\" (UID: \"18946076-1464-4aac-bbe1-3c4d2b4b6ae9\") " pod="openshift-image-registry/node-ca-8vfqs" Jul 10 15:55:13.612052 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611347 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7511b558-f953-4c6b-a90b-c2843281f2cc-dbus\") pod \"global-pull-secret-syncer-h9z25\" (UID: \"7511b558-f953-4c6b-a90b-c2843281f2cc\") " pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:13.612052 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611374 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-etc-kubernetes\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.612052 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611401 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-var-lib-openvswitch\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.612052 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611408 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-log-socket\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.612052 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611427 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/78e16a25-054d-4cd6-8578-f101a0e47f77-iptables-alerter-script\") pod \"iptables-alerter-7prkb\" (UID: \"78e16a25-054d-4cd6-8578-f101a0e47f77\") " pod="openshift-network-operator/iptables-alerter-7prkb" Jul 10 15:55:13.612052 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611455 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-etc-openvswitch\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.612861 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611491 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.612861 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611498 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-system-cni-dir\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.612861 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611456 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-etc-kubernetes\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.612861 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611498 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-var-lib-openvswitch\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.612861 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611528 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-etc-openvswitch\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.612861 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611556 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a7a2569d-7313-49a4-b283-aa416f5fdd9e-ovnkube-script-lib\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.612861 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611568 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/78e16a25-054d-4cd6-8578-f101a0e47f77-host-slash\") pod \"iptables-alerter-7prkb\" (UID: \"78e16a25-054d-4cd6-8578-f101a0e47f77\") " pod="openshift-network-operator/iptables-alerter-7prkb" Jul 10 15:55:13.612861 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611576 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.612861 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611579 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a7a2569d-7313-49a4-b283-aa416f5fdd9e-ovn-node-metrics-cert\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.612861 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611611 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9h4t\" (UniqueName: \"kubernetes.io/projected/a7a2569d-7313-49a4-b283-aa416f5fdd9e-kube-api-access-l9h4t\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.612861 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611641 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-sys\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.612861 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611667 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-host-run-netns\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.612861 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611709 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-host-run-netns\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.612861 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611728 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-sys\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.612861 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611754 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/017c05d8-db4b-427e-b6de-19e1d1264b97-multus-daemon-config\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.612861 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611779 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7511b558-f953-4c6b-a90b-c2843281f2cc-kubelet-config\") pod \"global-pull-secret-syncer-h9z25\" (UID: \"7511b558-f953-4c6b-a90b-c2843281f2cc\") " pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:13.612861 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611785 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a7a2569d-7313-49a4-b283-aa416f5fdd9e-ovnkube-config\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.612861 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611811 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/81e1bb75-b227-4f4a-81d4-6843cebccc61-hosts-file\") pod \"node-resolver-ngcrw\" (UID: \"81e1bb75-b227-4f4a-81d4-6843cebccc61\") " pod="openshift-dns/node-resolver-ngcrw" Jul 10 15:55:13.613730 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611835 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92d76a94-65a8-457e-8a57-bc3729beeb80-cnibin\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.613730 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611862 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7511b558-f953-4c6b-a90b-c2843281f2cc-kubelet-config\") pod \"global-pull-secret-syncer-h9z25\" (UID: \"7511b558-f953-4c6b-a90b-c2843281f2cc\") " pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:13.613730 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611862 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-host-var-lib-kubelet\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.613730 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611899 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdgtm\" (UniqueName: \"kubernetes.io/projected/81e1bb75-b227-4f4a-81d4-6843cebccc61-kube-api-access-mdgtm\") pod \"node-resolver-ngcrw\" (UID: \"81e1bb75-b227-4f4a-81d4-6843cebccc61\") " pod="openshift-dns/node-resolver-ngcrw" Jul 10 15:55:13.613730 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611902 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/92d76a94-65a8-457e-8a57-bc3729beeb80-cnibin\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.613730 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611922 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/81e1bb75-b227-4f4a-81d4-6843cebccc61-hosts-file\") pod \"node-resolver-ngcrw\" (UID: \"81e1bb75-b227-4f4a-81d4-6843cebccc61\") " pod="openshift-dns/node-resolver-ngcrw" Jul 10 15:55:13.613730 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611938 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-host-var-lib-kubelet\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.614318 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.611922 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/18946076-1464-4aac-bbe1-3c4d2b4b6ae9-serviceca\") pod \"node-ca-8vfqs\" (UID: \"18946076-1464-4aac-bbe1-3c4d2b4b6ae9\") " pod="openshift-image-registry/node-ca-8vfqs" Jul 10 15:55:13.614425 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.614328 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-var-lib-kubelet\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.614425 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.614377 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtwpc\" (UniqueName: \"kubernetes.io/projected/017c05d8-db4b-427e-b6de-19e1d1264b97-kube-api-access-mtwpc\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.614425 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.614415 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a7a2569d-7313-49a4-b283-aa416f5fdd9e-env-overrides\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.614606 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.614450 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd8td\" (UniqueName: \"kubernetes.io/projected/78e16a25-054d-4cd6-8578-f101a0e47f77-kube-api-access-cd8td\") pod \"iptables-alerter-7prkb\" (UID: \"78e16a25-054d-4cd6-8578-f101a0e47f77\") " pod="openshift-network-operator/iptables-alerter-7prkb" Jul 10 15:55:13.614606 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.614479 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-etc-sysctl-d\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.614606 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.614514 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z98ml\" (UniqueName: \"kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml\") pod \"network-check-target-2h276\" (UID: \"ff0e4720-f5ab-4832-b118-eb5fd37dd8ab\") " pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:13.614606 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.614567 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-host-var-lib-cni-multus\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.614791 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.614606 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-host-kubelet\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.614791 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.614639 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-run-openvswitch\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.614791 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.614674 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81e1bb75-b227-4f4a-81d4-6843cebccc61-tmp-dir\") pod \"node-resolver-ngcrw\" (UID: \"81e1bb75-b227-4f4a-81d4-6843cebccc61\") " pod="openshift-dns/node-resolver-ngcrw" Jul 10 15:55:13.614791 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.614702 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/017c05d8-db4b-427e-b6de-19e1d1264b97-cni-binary-copy\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.614791 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.614735 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/92d76a94-65a8-457e-8a57-bc3729beeb80-os-release\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.614791 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.614768 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-run\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.615046 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.614801 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-multus-cni-dir\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.615046 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.614836 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-hostroot\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.615046 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.614863 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-host-run-multus-certs\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.615046 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.614896 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-node-log\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.615046 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.614928 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/18946076-1464-4aac-bbe1-3c4d2b4b6ae9-host\") pod \"node-ca-8vfqs\" (UID: \"18946076-1464-4aac-bbe1-3c4d2b4b6ae9\") " pod="openshift-image-registry/node-ca-8vfqs" Jul 10 15:55:13.615046 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.614959 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-etc-modprobe-d\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.615046 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.614985 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-etc-systemd\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.615046 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.615038 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eecb1845-50d9-4878-8434-a13b10aa2833-tmp\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.615355 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.615071 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-host-var-lib-cni-bin\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.615355 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.615103 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-etc-sysctl-conf\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.615355 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.615140 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92d76a94-65a8-457e-8a57-bc3729beeb80-cni-binary-copy\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.615355 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.615170 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92d76a94-65a8-457e-8a57-bc3729beeb80-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.615355 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.615204 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qglk6\" (UniqueName: \"kubernetes.io/projected/92d76a94-65a8-457e-8a57-bc3729beeb80-kube-api-access-qglk6\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.615355 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.615232 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-host-run-ovn-kubernetes\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.615355 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.615346 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-host-run-ovn-kubernetes\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.615804 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.615785 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/18946076-1464-4aac-bbe1-3c4d2b4b6ae9-serviceca\") pod \"node-ca-8vfqs\" (UID: \"18946076-1464-4aac-bbe1-3c4d2b4b6ae9\") " pod="openshift-image-registry/node-ca-8vfqs" Jul 10 15:55:13.615911 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.615868 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-var-lib-kubelet\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.617809 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.616367 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a7a2569d-7313-49a4-b283-aa416f5fdd9e-ovn-node-metrics-cert\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.617809 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.616389 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/017c05d8-db4b-427e-b6de-19e1d1264b97-multus-daemon-config\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.617809 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.616462 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-hostroot\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.617809 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.616568 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/eecb1845-50d9-4878-8434-a13b10aa2833-etc-tuned\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.617809 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.616573 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a7a2569d-7313-49a4-b283-aa416f5fdd9e-env-overrides\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.617809 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.616658 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-host-run-multus-certs\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.617809 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.616741 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-node-log\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.617809 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.616793 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-etc-sysctl-d\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.617809 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.616812 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-run-openvswitch\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.617809 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.616821 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-host-var-lib-cni-bin\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.617809 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.616813 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/92d76a94-65a8-457e-8a57-bc3729beeb80-os-release\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.617809 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.616890 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-multus-cni-dir\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.617809 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.616984 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/92d76a94-65a8-457e-8a57-bc3729beeb80-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.617809 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.616993 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-etc-sysctl-conf\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.617809 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.617064 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/18946076-1464-4aac-bbe1-3c4d2b4b6ae9-host\") pod \"node-ca-8vfqs\" (UID: \"18946076-1464-4aac-bbe1-3c4d2b4b6ae9\") " pod="openshift-image-registry/node-ca-8vfqs" Jul 10 15:55:13.617809 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.617178 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-etc-modprobe-d\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.617809 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.617230 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-etc-systemd\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.617809 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.617308 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/017c05d8-db4b-427e-b6de-19e1d1264b97-host-var-lib-cni-multus\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.618750 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.617356 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/017c05d8-db4b-427e-b6de-19e1d1264b97-cni-binary-copy\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.618750 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.617403 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/92d76a94-65a8-457e-8a57-bc3729beeb80-cni-binary-copy\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.618750 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.617426 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7a2569d-7313-49a4-b283-aa416f5fdd9e-host-kubelet\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.618750 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.617467 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/eecb1845-50d9-4878-8434-a13b10aa2833-run\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.618750 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.617643 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81e1bb75-b227-4f4a-81d4-6843cebccc61-tmp-dir\") pod \"node-resolver-ngcrw\" (UID: \"81e1bb75-b227-4f4a-81d4-6843cebccc61\") " pod="openshift-dns/node-resolver-ngcrw" Jul 10 15:55:13.619722 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.619684 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eecb1845-50d9-4878-8434-a13b10aa2833-tmp\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.645478 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:13.645422 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 15:55:13.645478 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:13.645445 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 15:55:13.645478 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:13.645459 2560 projected.go:196] Error preparing data for projected volume kube-api-access-z98ml for pod openshift-network-diagnostics/network-check-target-2h276: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 15:55:13.645695 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:13.645518 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml podName:ff0e4720-f5ab-4832-b118-eb5fd37dd8ab nodeName:}" failed. No retries permitted until 2026-07-10 15:55:14.145500871 +0000 UTC m=+3.117534228 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-z98ml" (UniqueName: "kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml") pod "network-check-target-2h276" (UID: "ff0e4720-f5ab-4832-b118-eb5fd37dd8ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 15:55:13.646505 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.646480 2560 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 15:55:13.647639 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.647618 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z29gv\" (UniqueName: \"kubernetes.io/projected/18946076-1464-4aac-bbe1-3c4d2b4b6ae9-kube-api-access-z29gv\") pod \"node-ca-8vfqs\" (UID: \"18946076-1464-4aac-bbe1-3c4d2b4b6ae9\") " pod="openshift-image-registry/node-ca-8vfqs" Jul 10 15:55:13.653124 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.653097 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtwpc\" (UniqueName: \"kubernetes.io/projected/017c05d8-db4b-427e-b6de-19e1d1264b97-kube-api-access-mtwpc\") pod \"multus-z6k5g\" (UID: \"017c05d8-db4b-427e-b6de-19e1d1264b97\") " pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.653248 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.653191 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdgtm\" (UniqueName: \"kubernetes.io/projected/81e1bb75-b227-4f4a-81d4-6843cebccc61-kube-api-access-mdgtm\") pod \"node-resolver-ngcrw\" (UID: \"81e1bb75-b227-4f4a-81d4-6843cebccc61\") " pod="openshift-dns/node-resolver-ngcrw" Jul 10 15:55:13.653317 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.653270 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9h4t\" (UniqueName: \"kubernetes.io/projected/a7a2569d-7313-49a4-b283-aa416f5fdd9e-kube-api-access-l9h4t\") pod \"ovnkube-node-6s2sn\" (UID: \"a7a2569d-7313-49a4-b283-aa416f5fdd9e\") " pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.655341 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.655308 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v76l8\" (UniqueName: \"kubernetes.io/projected/eecb1845-50d9-4878-8434-a13b10aa2833-kube-api-access-v76l8\") pod \"tuned-7cvfh\" (UID: \"eecb1845-50d9-4878-8434-a13b10aa2833\") " pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.655537 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.655519 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qglk6\" (UniqueName: \"kubernetes.io/projected/92d76a94-65a8-457e-8a57-bc3729beeb80-kube-api-access-qglk6\") pod \"multus-additional-cni-plugins-cfcr7\" (UID: \"92d76a94-65a8-457e-8a57-bc3729beeb80\") " pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.655807 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.655790 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd8td\" (UniqueName: \"kubernetes.io/projected/78e16a25-054d-4cd6-8578-f101a0e47f77-kube-api-access-cd8td\") pod \"iptables-alerter-7prkb\" (UID: \"78e16a25-054d-4cd6-8578-f101a0e47f77\") " pod="openshift-network-operator/iptables-alerter-7prkb" Jul 10 15:55:13.708933 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.708902 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" Jul 10 15:55:13.717479 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.717457 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8bbgg" Jul 10 15:55:13.730655 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.730629 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cfcr7" Jul 10 15:55:13.738360 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.738341 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:13.748703 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.748682 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7prkb" Jul 10 15:55:13.756264 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.756220 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-z6k5g" Jul 10 15:55:13.765170 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.765153 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-8vfqs" Jul 10 15:55:13.779991 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.779955 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" Jul 10 15:55:13.785839 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.785820 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ngcrw" Jul 10 15:55:13.984894 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:13.984816 2560 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 10 15:55:14.018939 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:14.018905 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs\") pod \"network-metrics-daemon-mjxwx\" (UID: \"049e242d-f002-42b0-b116-e9baa5f8eb87\") " pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:14.019123 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:14.019055 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 15:55:14.019123 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:14.019121 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs podName:049e242d-f002-42b0-b116-e9baa5f8eb87 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:15.019101974 +0000 UTC m=+3.991135347 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs") pod "network-metrics-daemon-mjxwx" (UID: "049e242d-f002-42b0-b116-e9baa5f8eb87") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 15:55:14.119921 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:14.119892 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret\") pod \"global-pull-secret-syncer-h9z25\" (UID: \"7511b558-f953-4c6b-a90b-c2843281f2cc\") " pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:14.120086 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:14.120024 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 15:55:14.120086 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:14.120082 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret podName:7511b558-f953-4c6b-a90b-c2843281f2cc nodeName:}" failed. No retries permitted until 2026-07-10 15:55:15.120069234 +0000 UTC m=+4.092102589 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret") pod "global-pull-secret-syncer-h9z25" (UID: "7511b558-f953-4c6b-a90b-c2843281f2cc") : object "kube-system"/"original-pull-secret" not registered Jul 10 15:55:14.220203 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:14.220166 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z98ml\" (UniqueName: \"kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml\") pod \"network-check-target-2h276\" (UID: \"ff0e4720-f5ab-4832-b118-eb5fd37dd8ab\") " pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:14.220356 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:14.220291 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 15:55:14.220356 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:14.220305 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 15:55:14.220356 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:14.220314 2560 projected.go:196] Error preparing data for projected volume kube-api-access-z98ml for pod openshift-network-diagnostics/network-check-target-2h276: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 15:55:14.220485 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:14.220359 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml podName:ff0e4720-f5ab-4832-b118-eb5fd37dd8ab nodeName:}" failed. No retries permitted until 2026-07-10 15:55:15.220345565 +0000 UTC m=+4.192378923 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-z98ml" (UniqueName: "kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml") pod "network-check-target-2h276" (UID: "ff0e4720-f5ab-4832-b118-eb5fd37dd8ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 15:55:14.481675 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:14.481576 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 15:50:13 +0000 UTC" deadline="2028-03-22 04:15:41.659976862 +0000 UTC" Jul 10 15:55:14.481675 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:14.481625 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14892h20m27.178355336s" Jul 10 15:55:14.546331 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:14.546290 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:14.546331 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:14.546321 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:14.546520 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:14.546415 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h276" podUID="ff0e4720-f5ab-4832-b118-eb5fd37dd8ab" Jul 10 15:55:14.546567 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:14.546535 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9z25" podUID="7511b558-f953-4c6b-a90b-c2843281f2cc" Jul 10 15:55:14.616072 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:14.616043 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5c8652b_3f54_4b1d_8bbb_597347aba689.slice/crio-aafe890547618da6593fe0dde4c616c7e4b94dbfeebeebcc1e14948608b9e5d1 WatchSource:0}: Error finding container aafe890547618da6593fe0dde4c616c7e4b94dbfeebeebcc1e14948608b9e5d1: Status 404 returned error can't find the container with id aafe890547618da6593fe0dde4c616c7e4b94dbfeebeebcc1e14948608b9e5d1 Jul 10 15:55:14.619121 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:14.619092 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81e1bb75_b227_4f4a_81d4_6843cebccc61.slice/crio-9f31a3f2596034b449a46ebd2e245304da28e54b4d0c69ad7a981cf183c816c2 WatchSource:0}: Error finding container 9f31a3f2596034b449a46ebd2e245304da28e54b4d0c69ad7a981cf183c816c2: Status 404 returned error can't find the container with id 9f31a3f2596034b449a46ebd2e245304da28e54b4d0c69ad7a981cf183c816c2 Jul 10 15:55:14.621820 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:14.621801 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6e830d4_3ff4_4c7d_9066_379949f26ffa.slice/crio-fce6dffb18926311b91927e129cc0da2e0a909048d6c4b9dcd751983187bd931 WatchSource:0}: Error finding container fce6dffb18926311b91927e129cc0da2e0a909048d6c4b9dcd751983187bd931: Status 404 returned error can't find the container with id fce6dffb18926311b91927e129cc0da2e0a909048d6c4b9dcd751983187bd931 Jul 10 15:55:14.623119 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:14.623086 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeecb1845_50d9_4878_8434_a13b10aa2833.slice/crio-d6f849d6ad606758cb68b255cbbb91cda512366f29cc797c07b3922084d7052f WatchSource:0}: Error finding container d6f849d6ad606758cb68b255cbbb91cda512366f29cc797c07b3922084d7052f: Status 404 returned error can't find the container with id d6f849d6ad606758cb68b255cbbb91cda512366f29cc797c07b3922084d7052f Jul 10 15:55:14.624400 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:14.624373 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod017c05d8_db4b_427e_b6de_19e1d1264b97.slice/crio-ae502310b396623607e02a67811c688b3cae4cdcd6d60dca5a392dfd712bdad8 WatchSource:0}: Error finding container ae502310b396623607e02a67811c688b3cae4cdcd6d60dca5a392dfd712bdad8: Status 404 returned error can't find the container with id ae502310b396623607e02a67811c688b3cae4cdcd6d60dca5a392dfd712bdad8 Jul 10 15:55:14.626012 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:14.625990 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18946076_1464_4aac_bbe1_3c4d2b4b6ae9.slice/crio-278f4b4faabf7f6ffc766d6954e1c5a8162545aa2212ceb9644948f9e95c57ee WatchSource:0}: Error finding container 278f4b4faabf7f6ffc766d6954e1c5a8162545aa2212ceb9644948f9e95c57ee: Status 404 returned error can't find the container with id 278f4b4faabf7f6ffc766d6954e1c5a8162545aa2212ceb9644948f9e95c57ee Jul 10 15:55:14.627822 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:14.627664 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92d76a94_65a8_457e_8a57_bc3729beeb80.slice/crio-a51a54f0023c0d0355d43f06a492221a7bd88ce3948365e3921eda12f9d6b4cc WatchSource:0}: Error finding container a51a54f0023c0d0355d43f06a492221a7bd88ce3948365e3921eda12f9d6b4cc: Status 404 returned error can't find the container with id a51a54f0023c0d0355d43f06a492221a7bd88ce3948365e3921eda12f9d6b4cc Jul 10 15:55:15.027447 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:15.027234 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs\") pod \"network-metrics-daemon-mjxwx\" (UID: \"049e242d-f002-42b0-b116-e9baa5f8eb87\") " pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:15.027619 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:15.027421 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 15:55:15.027619 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:15.027598 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs podName:049e242d-f002-42b0-b116-e9baa5f8eb87 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:17.027536779 +0000 UTC m=+5.999570137 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs") pod "network-metrics-daemon-mjxwx" (UID: "049e242d-f002-42b0-b116-e9baa5f8eb87") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 15:55:15.131482 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:15.131385 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret\") pod \"global-pull-secret-syncer-h9z25\" (UID: \"7511b558-f953-4c6b-a90b-c2843281f2cc\") " pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:15.131649 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:15.131589 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 15:55:15.131715 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:15.131658 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret podName:7511b558-f953-4c6b-a90b-c2843281f2cc nodeName:}" failed. No retries permitted until 2026-07-10 15:55:17.131638756 +0000 UTC m=+6.103672136 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret") pod "global-pull-secret-syncer-h9z25" (UID: "7511b558-f953-4c6b-a90b-c2843281f2cc") : object "kube-system"/"original-pull-secret" not registered Jul 10 15:55:15.232812 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:15.232765 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z98ml\" (UniqueName: \"kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml\") pod \"network-check-target-2h276\" (UID: \"ff0e4720-f5ab-4832-b118-eb5fd37dd8ab\") " pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:15.233012 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:15.232972 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 15:55:15.233012 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:15.233010 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 15:55:15.233136 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:15.233025 2560 projected.go:196] Error preparing data for projected volume kube-api-access-z98ml for pod openshift-network-diagnostics/network-check-target-2h276: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 15:55:15.233136 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:15.233088 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml podName:ff0e4720-f5ab-4832-b118-eb5fd37dd8ab nodeName:}" failed. No retries permitted until 2026-07-10 15:55:17.233067793 +0000 UTC m=+6.205101154 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-z98ml" (UniqueName: "kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml") pod "network-check-target-2h276" (UID: "ff0e4720-f5ab-4832-b118-eb5fd37dd8ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 15:55:15.483080 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:15.482911 2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-09 15:50:13 +0000 UTC" deadline="2028-02-27 08:07:02.628133411 +0000 UTC" Jul 10 15:55:15.483080 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:15.482999 2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14320h11m47.145142445s" Jul 10 15:55:15.547667 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:15.547148 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:15.547667 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:15.547306 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mjxwx" podUID="049e242d-f002-42b0-b116-e9baa5f8eb87" Jul 10 15:55:15.575779 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:15.575743 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-101.ec2.internal" event={"ID":"c72755a90b814da38c5f2e7b423a2466","Type":"ContainerStarted","Data":"3b8db908e2864638ee08b4efe4eb3a17f1aa9f25d0eb59e5279ab838031223d4"} Jul 10 15:55:15.584180 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:15.584084 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8vfqs" event={"ID":"18946076-1464-4aac-bbe1-3c4d2b4b6ae9","Type":"ContainerStarted","Data":"278f4b4faabf7f6ffc766d6954e1c5a8162545aa2212ceb9644948f9e95c57ee"} Jul 10 15:55:15.608256 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:15.608225 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" event={"ID":"eecb1845-50d9-4878-8434-a13b10aa2833","Type":"ContainerStarted","Data":"d6f849d6ad606758cb68b255cbbb91cda512366f29cc797c07b3922084d7052f"} Jul 10 15:55:15.611917 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:15.611852 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ngcrw" event={"ID":"81e1bb75-b227-4f4a-81d4-6843cebccc61","Type":"ContainerStarted","Data":"9f31a3f2596034b449a46ebd2e245304da28e54b4d0c69ad7a981cf183c816c2"} Jul 10 15:55:15.623682 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:15.623576 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" event={"ID":"a7a2569d-7313-49a4-b283-aa416f5fdd9e","Type":"ContainerStarted","Data":"b4aa5941b49bbcdfb51387d42cea2757604188ac829973232bb3394741ed4525"} Jul 10 15:55:15.635603 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:15.635563 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7prkb" event={"ID":"78e16a25-054d-4cd6-8578-f101a0e47f77","Type":"ContainerStarted","Data":"724935c393d0e9e2395e864c650b1abfb38d9cc4d78ced5374ea68165ecc5c61"} Jul 10 15:55:15.656055 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:15.655979 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cfcr7" event={"ID":"92d76a94-65a8-457e-8a57-bc3729beeb80","Type":"ContainerStarted","Data":"a51a54f0023c0d0355d43f06a492221a7bd88ce3948365e3921eda12f9d6b4cc"} Jul 10 15:55:15.660915 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:15.660822 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z6k5g" event={"ID":"017c05d8-db4b-427e-b6de-19e1d1264b97","Type":"ContainerStarted","Data":"ae502310b396623607e02a67811c688b3cae4cdcd6d60dca5a392dfd712bdad8"} Jul 10 15:55:15.669294 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:15.669073 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" event={"ID":"c6e830d4-3ff4-4c7d-9066-379949f26ffa","Type":"ContainerStarted","Data":"fce6dffb18926311b91927e129cc0da2e0a909048d6c4b9dcd751983187bd931"} Jul 10 15:55:15.673306 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:15.673234 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8bbgg" event={"ID":"d5c8652b-3f54-4b1d-8bbb-597347aba689","Type":"ContainerStarted","Data":"aafe890547618da6593fe0dde4c616c7e4b94dbfeebeebcc1e14948608b9e5d1"} Jul 10 15:55:15.679247 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:15.679219 2560 generic.go:356] "Generic (PLEG): container finished" podID="239fe417768071dd08b1b2c414e86a71" containerID="28c9863bf9830b4a174393a840e8b7898eb08be97b9777f8a9eed152a3a3bf22" exitCode=0 Jul 10 15:55:15.679356 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:15.679261 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal" event={"ID":"239fe417768071dd08b1b2c414e86a71","Type":"ContainerDied","Data":"28c9863bf9830b4a174393a840e8b7898eb08be97b9777f8a9eed152a3a3bf22"} Jul 10 15:55:15.708497 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:15.707948 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-101.ec2.internal" podStartSLOduration=3.7079272420000002 podStartE2EDuration="3.707927242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:15.594435281 +0000 UTC m=+4.566468661" watchObservedRunningTime="2026-07-10 15:55:15.707927242 +0000 UTC m=+4.679960622" Jul 10 15:55:16.546808 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:16.546777 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:16.547294 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:16.546906 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9z25" podUID="7511b558-f953-4c6b-a90b-c2843281f2cc" Jul 10 15:55:16.547294 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:16.547097 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:16.547294 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:16.547183 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h276" podUID="ff0e4720-f5ab-4832-b118-eb5fd37dd8ab" Jul 10 15:55:16.713153 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:16.713080 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal" event={"ID":"239fe417768071dd08b1b2c414e86a71","Type":"ContainerStarted","Data":"902584ce8551dc90fba5317eb836a9e88edbe9a23e0b8e5f97a7dde2293c9aca"} Jul 10 15:55:16.730359 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:16.730304 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-101.ec2.internal" podStartSLOduration=4.730282745 podStartE2EDuration="4.730282745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:16.73020566 +0000 UTC m=+5.702239035" watchObservedRunningTime="2026-07-10 15:55:16.730282745 +0000 UTC m=+5.702316122" Jul 10 15:55:17.048943 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:17.048379 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs\") pod \"network-metrics-daemon-mjxwx\" (UID: \"049e242d-f002-42b0-b116-e9baa5f8eb87\") " pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:17.048943 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:17.048530 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 15:55:17.048943 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:17.048595 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs podName:049e242d-f002-42b0-b116-e9baa5f8eb87 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:21.048577977 +0000 UTC m=+10.020611335 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs") pod "network-metrics-daemon-mjxwx" (UID: "049e242d-f002-42b0-b116-e9baa5f8eb87") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 15:55:17.149708 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:17.149670 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret\") pod \"global-pull-secret-syncer-h9z25\" (UID: \"7511b558-f953-4c6b-a90b-c2843281f2cc\") " pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:17.149884 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:17.149847 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 15:55:17.149945 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:17.149937 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret podName:7511b558-f953-4c6b-a90b-c2843281f2cc nodeName:}" failed. No retries permitted until 2026-07-10 15:55:21.14988949 +0000 UTC m=+10.121922844 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret") pod "global-pull-secret-syncer-h9z25" (UID: "7511b558-f953-4c6b-a90b-c2843281f2cc") : object "kube-system"/"original-pull-secret" not registered Jul 10 15:55:17.250355 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:17.250315 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z98ml\" (UniqueName: \"kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml\") pod \"network-check-target-2h276\" (UID: \"ff0e4720-f5ab-4832-b118-eb5fd37dd8ab\") " pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:17.250592 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:17.250525 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 15:55:17.250592 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:17.250549 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 15:55:17.250592 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:17.250562 2560 projected.go:196] Error preparing data for projected volume kube-api-access-z98ml for pod openshift-network-diagnostics/network-check-target-2h276: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 15:55:17.250769 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:17.250622 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml podName:ff0e4720-f5ab-4832-b118-eb5fd37dd8ab nodeName:}" failed. No retries permitted until 2026-07-10 15:55:21.250602731 +0000 UTC m=+10.222636101 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-z98ml" (UniqueName: "kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml") pod "network-check-target-2h276" (UID: "ff0e4720-f5ab-4832-b118-eb5fd37dd8ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 15:55:17.548739 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:17.548663 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:17.549163 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:17.548791 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mjxwx" podUID="049e242d-f002-42b0-b116-e9baa5f8eb87" Jul 10 15:55:18.546308 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:18.546274 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:18.546515 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:18.546390 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9z25" podUID="7511b558-f953-4c6b-a90b-c2843281f2cc" Jul 10 15:55:18.546806 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:18.546787 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:18.546894 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:18.546879 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h276" podUID="ff0e4720-f5ab-4832-b118-eb5fd37dd8ab" Jul 10 15:55:19.546531 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:19.546492 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:19.547019 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:19.546647 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mjxwx" podUID="049e242d-f002-42b0-b116-e9baa5f8eb87" Jul 10 15:55:20.546587 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:20.546089 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:20.546587 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:20.546127 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:20.546587 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:20.546229 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9z25" podUID="7511b558-f953-4c6b-a90b-c2843281f2cc" Jul 10 15:55:20.546587 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:20.546343 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h276" podUID="ff0e4720-f5ab-4832-b118-eb5fd37dd8ab" Jul 10 15:55:21.085025 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:21.084991 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs\") pod \"network-metrics-daemon-mjxwx\" (UID: \"049e242d-f002-42b0-b116-e9baa5f8eb87\") " pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:21.085193 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:21.085142 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 15:55:21.085271 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:21.085204 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs podName:049e242d-f002-42b0-b116-e9baa5f8eb87 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:29.085185091 +0000 UTC m=+18.057218447 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs") pod "network-metrics-daemon-mjxwx" (UID: "049e242d-f002-42b0-b116-e9baa5f8eb87") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 15:55:21.185904 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:21.185816 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret\") pod \"global-pull-secret-syncer-h9z25\" (UID: \"7511b558-f953-4c6b-a90b-c2843281f2cc\") " pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:21.186109 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:21.185992 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 15:55:21.186109 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:21.186065 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret podName:7511b558-f953-4c6b-a90b-c2843281f2cc nodeName:}" failed. No retries permitted until 2026-07-10 15:55:29.186045702 +0000 UTC m=+18.158079073 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret") pod "global-pull-secret-syncer-h9z25" (UID: "7511b558-f953-4c6b-a90b-c2843281f2cc") : object "kube-system"/"original-pull-secret" not registered Jul 10 15:55:21.287234 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:21.286709 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z98ml\" (UniqueName: \"kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml\") pod \"network-check-target-2h276\" (UID: \"ff0e4720-f5ab-4832-b118-eb5fd37dd8ab\") " pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:21.287234 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:21.286865 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 15:55:21.287234 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:21.286883 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 15:55:21.287234 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:21.286897 2560 projected.go:196] Error preparing data for projected volume kube-api-access-z98ml for pod openshift-network-diagnostics/network-check-target-2h276: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 15:55:21.287234 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:21.286978 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml podName:ff0e4720-f5ab-4832-b118-eb5fd37dd8ab nodeName:}" failed. No retries permitted until 2026-07-10 15:55:29.286942871 +0000 UTC m=+18.258976227 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-z98ml" (UniqueName: "kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml") pod "network-check-target-2h276" (UID: "ff0e4720-f5ab-4832-b118-eb5fd37dd8ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 15:55:21.548566 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:21.548037 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:21.548566 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:21.548165 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mjxwx" podUID="049e242d-f002-42b0-b116-e9baa5f8eb87" Jul 10 15:55:22.546987 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:22.546939 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:22.547170 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:22.546939 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:22.547170 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:22.547090 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9z25" podUID="7511b558-f953-4c6b-a90b-c2843281f2cc" Jul 10 15:55:22.547170 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:22.547115 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h276" podUID="ff0e4720-f5ab-4832-b118-eb5fd37dd8ab" Jul 10 15:55:23.546913 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:23.546884 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:23.547337 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:23.547038 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mjxwx" podUID="049e242d-f002-42b0-b116-e9baa5f8eb87" Jul 10 15:55:24.546305 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:24.546266 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:24.546507 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:24.546268 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:24.546507 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:24.546373 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9z25" podUID="7511b558-f953-4c6b-a90b-c2843281f2cc" Jul 10 15:55:24.546507 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:24.546490 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h276" podUID="ff0e4720-f5ab-4832-b118-eb5fd37dd8ab" Jul 10 15:55:25.547046 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:25.546945 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:25.547487 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:25.547117 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mjxwx" podUID="049e242d-f002-42b0-b116-e9baa5f8eb87" Jul 10 15:55:26.546756 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:26.546715 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:26.546930 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:26.546763 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:26.546930 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:26.546845 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9z25" podUID="7511b558-f953-4c6b-a90b-c2843281f2cc" Jul 10 15:55:26.547082 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:26.546992 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h276" podUID="ff0e4720-f5ab-4832-b118-eb5fd37dd8ab" Jul 10 15:55:27.546716 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:27.546673 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:27.546892 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:27.546832 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mjxwx" podUID="049e242d-f002-42b0-b116-e9baa5f8eb87" Jul 10 15:55:28.546352 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:28.546323 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:28.546352 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:28.546338 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:28.546932 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:28.546452 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9z25" podUID="7511b558-f953-4c6b-a90b-c2843281f2cc" Jul 10 15:55:28.546932 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:28.546567 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h276" podUID="ff0e4720-f5ab-4832-b118-eb5fd37dd8ab" Jul 10 15:55:29.150265 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:29.150233 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs\") pod \"network-metrics-daemon-mjxwx\" (UID: \"049e242d-f002-42b0-b116-e9baa5f8eb87\") " pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:29.150436 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:29.150389 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 15:55:29.150495 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:29.150476 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs podName:049e242d-f002-42b0-b116-e9baa5f8eb87 nodeName:}" failed. No retries permitted until 2026-07-10 15:55:45.150443812 +0000 UTC m=+34.122477166 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs") pod "network-metrics-daemon-mjxwx" (UID: "049e242d-f002-42b0-b116-e9baa5f8eb87") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 15:55:29.250752 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:29.250662 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret\") pod \"global-pull-secret-syncer-h9z25\" (UID: \"7511b558-f953-4c6b-a90b-c2843281f2cc\") " pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:29.250908 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:29.250826 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 15:55:29.250908 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:29.250905 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret podName:7511b558-f953-4c6b-a90b-c2843281f2cc nodeName:}" failed. No retries permitted until 2026-07-10 15:55:45.250885164 +0000 UTC m=+34.222918531 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret") pod "global-pull-secret-syncer-h9z25" (UID: "7511b558-f953-4c6b-a90b-c2843281f2cc") : object "kube-system"/"original-pull-secret" not registered Jul 10 15:55:29.351345 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:29.351296 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z98ml\" (UniqueName: \"kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml\") pod \"network-check-target-2h276\" (UID: \"ff0e4720-f5ab-4832-b118-eb5fd37dd8ab\") " pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:29.351496 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:29.351442 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 15:55:29.351496 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:29.351461 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 15:55:29.351496 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:29.351470 2560 projected.go:196] Error preparing data for projected volume kube-api-access-z98ml for pod openshift-network-diagnostics/network-check-target-2h276: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 15:55:29.351625 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:29.351524 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml podName:ff0e4720-f5ab-4832-b118-eb5fd37dd8ab nodeName:}" failed. No retries permitted until 2026-07-10 15:55:45.351507927 +0000 UTC m=+34.323541302 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-z98ml" (UniqueName: "kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml") pod "network-check-target-2h276" (UID: "ff0e4720-f5ab-4832-b118-eb5fd37dd8ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 15:55:29.548934 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:29.548866 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:29.549339 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:29.548983 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mjxwx" podUID="049e242d-f002-42b0-b116-e9baa5f8eb87" Jul 10 15:55:30.546221 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:30.546184 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:30.546412 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:30.546188 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:30.546412 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:30.546318 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9z25" podUID="7511b558-f953-4c6b-a90b-c2843281f2cc" Jul 10 15:55:30.546547 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:30.546425 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h276" podUID="ff0e4720-f5ab-4832-b118-eb5fd37dd8ab" Jul 10 15:55:31.547928 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:31.547890 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:31.548393 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:31.548042 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mjxwx" podUID="049e242d-f002-42b0-b116-e9baa5f8eb87" Jul 10 15:55:32.546969 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:32.546918 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:32.546969 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:32.546946 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:32.547206 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:32.547074 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9z25" podUID="7511b558-f953-4c6b-a90b-c2843281f2cc" Jul 10 15:55:32.547263 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:32.547207 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h276" podUID="ff0e4720-f5ab-4832-b118-eb5fd37dd8ab" Jul 10 15:55:33.546751 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:33.546591 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:33.547424 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:33.546841 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mjxwx" podUID="049e242d-f002-42b0-b116-e9baa5f8eb87" Jul 10 15:55:33.747905 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:33.747746 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6s2sn_a7a2569d-7313-49a4-b283-aa416f5fdd9e/ovn-acl-logging/0.log" Jul 10 15:55:33.748253 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:33.748233 2560 generic.go:356] "Generic (PLEG): container finished" podID="a7a2569d-7313-49a4-b283-aa416f5fdd9e" containerID="b8d7634e867c03a2c173debe1d205c57098a53a289662891341e1d3865bd5e80" exitCode=1 Jul 10 15:55:33.748320 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:33.748301 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" event={"ID":"a7a2569d-7313-49a4-b283-aa416f5fdd9e","Type":"ContainerStarted","Data":"003050d4b52e172e68d201724a34a47dd16454855ddabe9ab60dba78d8af2051"} Jul 10 15:55:33.748387 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:33.748374 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" event={"ID":"a7a2569d-7313-49a4-b283-aa416f5fdd9e","Type":"ContainerStarted","Data":"f08f79bfe912da83fe8e7047b551cad5b6ea5770cfc9e0b373a5eae1a9d4a6fd"} Jul 10 15:55:33.748447 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:33.748412 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" event={"ID":"a7a2569d-7313-49a4-b283-aa416f5fdd9e","Type":"ContainerDied","Data":"b8d7634e867c03a2c173debe1d205c57098a53a289662891341e1d3865bd5e80"} Jul 10 15:55:33.748493 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:33.748458 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" event={"ID":"a7a2569d-7313-49a4-b283-aa416f5fdd9e","Type":"ContainerStarted","Data":"66d2e8ad96a9bbac4fd7426c211adafdca70ee591136c58ef195cd8f9f4c58fd"} Jul 10 15:55:33.749753 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:33.749732 2560 generic.go:356] "Generic (PLEG): container finished" podID="92d76a94-65a8-457e-8a57-bc3729beeb80" containerID="b2625e032c7207ae4276a1ff2003bc003601300b1533a0e72ac8383eb386dd86" exitCode=0 Jul 10 15:55:33.749852 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:33.749814 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cfcr7" event={"ID":"92d76a94-65a8-457e-8a57-bc3729beeb80","Type":"ContainerDied","Data":"b2625e032c7207ae4276a1ff2003bc003601300b1533a0e72ac8383eb386dd86"} Jul 10 15:55:33.751474 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:33.751327 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-z6k5g" event={"ID":"017c05d8-db4b-427e-b6de-19e1d1264b97","Type":"ContainerStarted","Data":"f11f28544c4a8294a47f523771831383b7ac3eb9bf6ea41603dfada050346f77"} Jul 10 15:55:33.752770 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:33.752750 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" event={"ID":"c6e830d4-3ff4-4c7d-9066-379949f26ffa","Type":"ContainerStarted","Data":"527d38d2aff2152939f3bfd9ff325acb0cf1f4adf8aceb8bb5e046ce9aaab4df"} Jul 10 15:55:33.754092 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:33.754068 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8bbgg" event={"ID":"d5c8652b-3f54-4b1d-8bbb-597347aba689","Type":"ContainerStarted","Data":"d2ecf96e52f252ee669c69a47da023c097f809a905854c1dda6f12e24f0af8ae"} Jul 10 15:55:33.755837 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:33.755816 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-8vfqs" event={"ID":"18946076-1464-4aac-bbe1-3c4d2b4b6ae9","Type":"ContainerStarted","Data":"bf7da0fdcc65915472fad300c17a3d60f1cda121275a6081874643d205de0469"} Jul 10 15:55:33.757291 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:33.757273 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" event={"ID":"eecb1845-50d9-4878-8434-a13b10aa2833","Type":"ContainerStarted","Data":"b99fdfa6b5e968bd34c4995fd99a9e2fa1d5c98c6f20902815ba285f57c11f1f"} Jul 10 15:55:33.758499 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:33.758479 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ngcrw" event={"ID":"81e1bb75-b227-4f4a-81d4-6843cebccc61","Type":"ContainerStarted","Data":"ae903aa994967f5202375d006f3698417be731370e5e79e006e00ccf6daaff7d"} Jul 10 15:55:33.787670 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:33.787573 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8bbgg" podStartSLOduration=4.11992922 podStartE2EDuration="22.78755927s" totalImagesPullingTime="18.66763005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:55:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:33.787406888 +0000 UTC m=+22.759440263" watchObservedRunningTime="2026-07-10 15:55:33.78755927 +0000 UTC m=+22.759592645" Jul 10 15:55:33.805279 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:33.805241 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-z6k5g" podStartSLOduration=3.092492224 podStartE2EDuration="21.805226651s" totalImagesPullingTime="18.712734427s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:33.80507627 +0000 UTC m=+22.777109662" watchObservedRunningTime="2026-07-10 15:55:33.805226651 +0000 UTC m=+22.777260025" Jul 10 15:55:33.820273 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:33.820233 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-8vfqs" podStartSLOduration=3.162339915 podStartE2EDuration="21.820217782s" totalImagesPullingTime="18.657877867s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:33.819250053 +0000 UTC m=+22.791283428" watchObservedRunningTime="2026-07-10 15:55:33.820217782 +0000 UTC m=+22.792251154" Jul 10 15:55:33.839562 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:33.839513 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7cvfh" podStartSLOduration=3.17774032 podStartE2EDuration="21.839493693s" totalImagesPullingTime="18.661753373s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:33.838617058 +0000 UTC m=+22.810650435" watchObservedRunningTime="2026-07-10 15:55:33.839493693 +0000 UTC m=+22.811527069" Jul 10 15:55:33.854147 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:33.854110 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ngcrw" podStartSLOduration=3.169946179 podStartE2EDuration="21.854096018s" totalImagesPullingTime="18.684149839s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:33.853742894 +0000 UTC m=+22.825776277" watchObservedRunningTime="2026-07-10 15:55:33.854096018 +0000 UTC m=+22.826129391" Jul 10 15:55:34.546403 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:34.546351 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:34.546403 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:34.546401 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:34.546770 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:34.546538 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9z25" podUID="7511b558-f953-4c6b-a90b-c2843281f2cc" Jul 10 15:55:34.546770 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:34.546654 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h276" podUID="ff0e4720-f5ab-4832-b118-eb5fd37dd8ab" Jul 10 15:55:34.763601 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:34.763523 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6s2sn_a7a2569d-7313-49a4-b283-aa416f5fdd9e/ovn-acl-logging/0.log" Jul 10 15:55:34.764065 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:34.764029 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" event={"ID":"a7a2569d-7313-49a4-b283-aa416f5fdd9e","Type":"ContainerStarted","Data":"4f5092902ad083ea5c26f4cf85c9206ec947d012b57904e3d956cd13c5f9d564"} Jul 10 15:55:34.764209 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:34.764118 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" event={"ID":"a7a2569d-7313-49a4-b283-aa416f5fdd9e","Type":"ContainerStarted","Data":"ce7c67cb400b9618539c7c6f8da5b6dee7b8cb845e745640a8d0de711e290800"} Jul 10 15:55:34.765713 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:34.765617 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7prkb" event={"ID":"78e16a25-054d-4cd6-8578-f101a0e47f77","Type":"ContainerStarted","Data":"f47a979f5a3e3e816fac12d52756961c77689ae567fbe936f40b7867a7a38da5"} Jul 10 15:55:34.781858 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:34.781805 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7prkb" podStartSLOduration=4.128150829 podStartE2EDuration="22.781787528s" totalImagesPullingTime="18.653636699s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:34.781619765 +0000 UTC m=+23.753653161" watchObservedRunningTime="2026-07-10 15:55:34.781787528 +0000 UTC m=+23.753820904" Jul 10 15:55:35.132091 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:35.132062 2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 10 15:55:35.469190 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:35.469048 2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-10T15:55:35.132084412Z","UUID":"5ff6e3a1-00fe-4409-b89f-15fcd72cfb04","Handler":null,"Name":"","Endpoint":""} Jul 10 15:55:35.470520 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:35.470493 2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 10 15:55:35.470644 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:35.470530 2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 10 15:55:35.547130 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:35.547096 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:35.547652 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:35.547253 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mjxwx" podUID="049e242d-f002-42b0-b116-e9baa5f8eb87" Jul 10 15:55:35.769994 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:35.769894 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" event={"ID":"c6e830d4-3ff4-4c7d-9066-379949f26ffa","Type":"ContainerStarted","Data":"13b4c4640402453fa23eb08debf41515a29f799a921fcf45bd5ada383170d9db"} Jul 10 15:55:36.546236 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:36.546079 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:36.546373 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:36.546085 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:36.546373 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:36.546320 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9z25" podUID="7511b558-f953-4c6b-a90b-c2843281f2cc" Jul 10 15:55:36.546475 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:36.546461 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h276" podUID="ff0e4720-f5ab-4832-b118-eb5fd37dd8ab" Jul 10 15:55:36.775653 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:36.775617 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6s2sn_a7a2569d-7313-49a4-b283-aa416f5fdd9e/ovn-acl-logging/0.log" Jul 10 15:55:36.776088 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:36.776049 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" event={"ID":"a7a2569d-7313-49a4-b283-aa416f5fdd9e","Type":"ContainerStarted","Data":"72cec33f4c7f1e34a6d220edc91a98a3258b6be306f593c7e243f79b2c791445"} Jul 10 15:55:37.048720 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:37.048683 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8bbgg" Jul 10 15:55:37.049315 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:37.049292 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8bbgg" Jul 10 15:55:37.546808 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:37.546731 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:37.546996 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:37.546847 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mjxwx" podUID="049e242d-f002-42b0-b116-e9baa5f8eb87" Jul 10 15:55:37.779022 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:37.778983 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8bbgg" Jul 10 15:55:37.779449 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:37.779437 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8bbgg" Jul 10 15:55:38.546685 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:38.546413 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:38.546685 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:38.546453 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:38.546879 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:38.546716 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9z25" podUID="7511b558-f953-4c6b-a90b-c2843281f2cc" Jul 10 15:55:38.546879 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:38.546749 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h276" podUID="ff0e4720-f5ab-4832-b118-eb5fd37dd8ab" Jul 10 15:55:38.784270 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:38.784232 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6s2sn_a7a2569d-7313-49a4-b283-aa416f5fdd9e/ovn-acl-logging/0.log" Jul 10 15:55:38.785125 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:38.784589 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" event={"ID":"a7a2569d-7313-49a4-b283-aa416f5fdd9e","Type":"ContainerStarted","Data":"6bc5ef072f8f1a1bdd47f7e9a99399f010f0dadce48c4e9fe6d7d491faa3575d"} Jul 10 15:55:38.785125 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:38.784891 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:38.785125 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:38.784970 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:38.785125 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:38.785111 2560 scope.go:117] "RemoveContainer" containerID="b8d7634e867c03a2c173debe1d205c57098a53a289662891341e1d3865bd5e80" Jul 10 15:55:38.786775 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:38.786753 2560 generic.go:356] "Generic (PLEG): container finished" podID="92d76a94-65a8-457e-8a57-bc3729beeb80" containerID="f089206024ef3202f5afdccd627956b408c58f8580abc37b97163b53e3659f3a" exitCode=0 Jul 10 15:55:38.786889 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:38.786803 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cfcr7" event={"ID":"92d76a94-65a8-457e-8a57-bc3729beeb80","Type":"ContainerDied","Data":"f089206024ef3202f5afdccd627956b408c58f8580abc37b97163b53e3659f3a"} Jul 10 15:55:38.789687 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:38.789667 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" event={"ID":"c6e830d4-3ff4-4c7d-9066-379949f26ffa","Type":"ContainerStarted","Data":"8c7e7db67fd0fa6e5a15aacd818edbd1096a16c49790bc0432794fb9800fa42b"} Jul 10 15:55:38.803263 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:38.803202 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:38.803440 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:38.803420 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:38.858873 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:38.858800 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qrpqh" podStartSLOduration=4.726670785 podStartE2EDuration="27.858779975s" totalImagesPullingTime="23.13210919s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:38.832181816 +0000 UTC m=+27.804215200" watchObservedRunningTime="2026-07-10 15:55:38.858779975 +0000 UTC m=+27.830813362" Jul 10 15:55:39.546845 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:39.546754 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:39.547118 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:39.546973 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mjxwx" podUID="049e242d-f002-42b0-b116-e9baa5f8eb87" Jul 10 15:55:39.796101 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:39.796076 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6s2sn_a7a2569d-7313-49a4-b283-aa416f5fdd9e/ovn-acl-logging/0.log" Jul 10 15:55:39.796619 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:39.796439 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" event={"ID":"a7a2569d-7313-49a4-b283-aa416f5fdd9e","Type":"ContainerStarted","Data":"01c90e389e243dd07b4778a7a77bae594c4a01be961f8a465c1d9459aa27c74f"} Jul 10 15:55:39.796619 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:39.796515 2560 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 10 15:55:39.825841 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:39.825747 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" podStartSLOduration=10.089670878 podStartE2EDuration="28.825730898s" totalImagesPullingTime="18.73606002s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:39.824208433 +0000 UTC m=+28.796241827" watchObservedRunningTime="2026-07-10 15:55:39.825730898 +0000 UTC m=+28.797764272" Jul 10 15:55:40.206715 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:40.206547 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:55:40.228448 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:40.228418 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2h276"] Jul 10 15:55:40.228599 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:40.228540 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:40.228666 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:40.228645 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h276" podUID="ff0e4720-f5ab-4832-b118-eb5fd37dd8ab" Jul 10 15:55:40.231593 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:40.231563 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h9z25"] Jul 10 15:55:40.231753 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:40.231708 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:40.231825 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:40.231804 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9z25" podUID="7511b558-f953-4c6b-a90b-c2843281f2cc" Jul 10 15:55:40.232229 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:40.232206 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mjxwx"] Jul 10 15:55:40.232345 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:40.232291 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:40.232409 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:40.232393 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mjxwx" podUID="049e242d-f002-42b0-b116-e9baa5f8eb87" Jul 10 15:55:40.800272 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:40.800240 2560 generic.go:356] "Generic (PLEG): container finished" podID="92d76a94-65a8-457e-8a57-bc3729beeb80" containerID="98801ed6a35dd5885d66d2c658546777adc47adb185bc4bcc731ff9d6efa3fa7" exitCode=0 Jul 10 15:55:40.800707 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:40.800296 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cfcr7" event={"ID":"92d76a94-65a8-457e-8a57-bc3729beeb80","Type":"ContainerDied","Data":"98801ed6a35dd5885d66d2c658546777adc47adb185bc4bcc731ff9d6efa3fa7"} Jul 10 15:55:41.547040 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:41.547007 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:41.547219 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:41.547047 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:41.547219 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:41.547122 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mjxwx" podUID="049e242d-f002-42b0-b116-e9baa5f8eb87" Jul 10 15:55:41.547219 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:41.547141 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h276" podUID="ff0e4720-f5ab-4832-b118-eb5fd37dd8ab" Jul 10 15:55:41.547219 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:41.547209 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:41.547404 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:41.547319 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9z25" podUID="7511b558-f953-4c6b-a90b-c2843281f2cc" Jul 10 15:55:42.807480 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:42.807450 2560 generic.go:356] "Generic (PLEG): container finished" podID="92d76a94-65a8-457e-8a57-bc3729beeb80" containerID="f4f8c7be2735056f8043d66abf54aec022b8561db1417449df59d841e45f66c7" exitCode=0 Jul 10 15:55:42.808033 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:42.807513 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cfcr7" event={"ID":"92d76a94-65a8-457e-8a57-bc3729beeb80","Type":"ContainerDied","Data":"f4f8c7be2735056f8043d66abf54aec022b8561db1417449df59d841e45f66c7"} Jul 10 15:55:43.546924 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:43.546780 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:43.546924 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:43.546780 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:43.547176 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:43.546946 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:43.547176 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:43.546934 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h276" podUID="ff0e4720-f5ab-4832-b118-eb5fd37dd8ab" Jul 10 15:55:43.547176 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:43.547081 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mjxwx" podUID="049e242d-f002-42b0-b116-e9baa5f8eb87" Jul 10 15:55:43.547176 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:43.547164 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9z25" podUID="7511b558-f953-4c6b-a90b-c2843281f2cc" Jul 10 15:55:45.162825 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:45.162775 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs\") pod \"network-metrics-daemon-mjxwx\" (UID: \"049e242d-f002-42b0-b116-e9baa5f8eb87\") " pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:45.163442 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:45.162922 2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 15:55:45.163442 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:45.162996 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs podName:049e242d-f002-42b0-b116-e9baa5f8eb87 nodeName:}" failed. No retries permitted until 2026-07-10 15:56:17.162977464 +0000 UTC m=+66.135010833 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs") pod "network-metrics-daemon-mjxwx" (UID: "049e242d-f002-42b0-b116-e9baa5f8eb87") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 10 15:55:45.263459 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:45.263425 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret\") pod \"global-pull-secret-syncer-h9z25\" (UID: \"7511b558-f953-4c6b-a90b-c2843281f2cc\") " pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:45.263638 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:45.263572 2560 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 10 15:55:45.263681 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:45.263660 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret podName:7511b558-f953-4c6b-a90b-c2843281f2cc nodeName:}" failed. No retries permitted until 2026-07-10 15:56:17.263616843 +0000 UTC m=+66.235650196 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret") pod "global-pull-secret-syncer-h9z25" (UID: "7511b558-f953-4c6b-a90b-c2843281f2cc") : object "kube-system"/"original-pull-secret" not registered Jul 10 15:55:45.364084 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:45.364042 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z98ml\" (UniqueName: \"kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml\") pod \"network-check-target-2h276\" (UID: \"ff0e4720-f5ab-4832-b118-eb5fd37dd8ab\") " pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:45.364256 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:45.364237 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 10 15:55:45.364321 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:45.364263 2560 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 10 15:55:45.364321 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:45.364276 2560 projected.go:196] Error preparing data for projected volume kube-api-access-z98ml for pod openshift-network-diagnostics/network-check-target-2h276: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 15:55:45.364410 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:45.364344 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml podName:ff0e4720-f5ab-4832-b118-eb5fd37dd8ab nodeName:}" failed. No retries permitted until 2026-07-10 15:56:17.364324834 +0000 UTC m=+66.336358193 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-z98ml" (UniqueName: "kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml") pod "network-check-target-2h276" (UID: "ff0e4720-f5ab-4832-b118-eb5fd37dd8ab") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 10 15:55:45.550073 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:45.549979 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:45.550073 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:45.550011 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:45.550303 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:45.550104 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:45.550303 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:45.550102 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9z25" podUID="7511b558-f953-4c6b-a90b-c2843281f2cc" Jul 10 15:55:45.550303 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:45.550243 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mjxwx" podUID="049e242d-f002-42b0-b116-e9baa5f8eb87" Jul 10 15:55:45.550480 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:45.550329 2560 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2h276" podUID="ff0e4720-f5ab-4832-b118-eb5fd37dd8ab" Jul 10 15:55:46.333459 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.333286 2560 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-101.ec2.internal" event="NodeReady" Jul 10 15:55:46.333888 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.333567 2560 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 10 15:55:46.368613 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.368583 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6c4695cf5b-d9pnq"] Jul 10 15:55:46.397547 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.397498 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-22xxq"] Jul 10 15:55:46.397734 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.397642 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.401647 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.400928 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5drgf\"" Jul 10 15:55:46.401647 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.401194 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 10 15:55:46.401647 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.401223 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 10 15:55:46.401647 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.401454 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 10 15:55:46.407074 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.407048 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 10 15:55:46.413777 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.413753 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4g9ns"] Jul 10 15:55:46.413906 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.413881 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-22xxq" Jul 10 15:55:46.417134 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.417094 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 10 15:55:46.417274 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.417250 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 10 15:55:46.417407 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.417280 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-k6hng\"" Jul 10 15:55:46.417519 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.417503 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 10 15:55:46.429789 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.429770 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c4695cf5b-d9pnq"] Jul 10 15:55:46.429903 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.429807 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4g9ns"] Jul 10 15:55:46.429903 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.429837 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-22xxq"] Jul 10 15:55:46.430002 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.429916 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4g9ns" Jul 10 15:55:46.432576 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.432556 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tn9vq\"" Jul 10 15:55:46.432664 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.432596 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 10 15:55:46.432889 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.432864 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 10 15:55:46.471974 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.471929 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-trusted-ca\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.472148 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.471988 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.472148 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.472015 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-installation-pull-secrets\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.472148 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.472084 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-image-registry-private-configuration\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.472148 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.472106 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-bound-sa-token\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.472148 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.472130 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-certificates\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.472373 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.472156 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-ca-trust-extracted\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.472373 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.472178 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khtbj\" (UniqueName: \"kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-kube-api-access-khtbj\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.572462 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.572423 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0159b035-3359-4e09-bb64-9237da0c327c-tmp-dir\") pod \"dns-default-4g9ns\" (UID: \"0159b035-3359-4e09-bb64-9237da0c327c\") " pod="openshift-dns/dns-default-4g9ns" Jul 10 15:55:46.572627 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.572470 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26flr\" (UniqueName: \"kubernetes.io/projected/0159b035-3359-4e09-bb64-9237da0c327c-kube-api-access-26flr\") pod \"dns-default-4g9ns\" (UID: \"0159b035-3359-4e09-bb64-9237da0c327c\") " pod="openshift-dns/dns-default-4g9ns" Jul 10 15:55:46.572627 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.572498 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x44zf\" (UniqueName: \"kubernetes.io/projected/47620ce7-6020-406a-9b19-92c89d6fbcc2-kube-api-access-x44zf\") pod \"ingress-canary-22xxq\" (UID: \"47620ce7-6020-406a-9b19-92c89d6fbcc2\") " pod="openshift-ingress-canary/ingress-canary-22xxq" Jul 10 15:55:46.572627 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.572553 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47620ce7-6020-406a-9b19-92c89d6fbcc2-cert\") pod \"ingress-canary-22xxq\" (UID: \"47620ce7-6020-406a-9b19-92c89d6fbcc2\") " pod="openshift-ingress-canary/ingress-canary-22xxq" Jul 10 15:55:46.572627 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.572597 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.572627 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.572620 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-installation-pull-secrets\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.572931 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.572695 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-image-registry-private-configuration\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.572931 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.572714 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-bound-sa-token\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.572931 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:46.572722 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 15:55:46.572931 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:46.572741 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c4695cf5b-d9pnq: secret "image-registry-tls" not found Jul 10 15:55:46.572931 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.572744 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-certificates\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.572931 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.572772 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0159b035-3359-4e09-bb64-9237da0c327c-config-volume\") pod \"dns-default-4g9ns\" (UID: \"0159b035-3359-4e09-bb64-9237da0c327c\") " pod="openshift-dns/dns-default-4g9ns" Jul 10 15:55:46.572931 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:46.572811 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls podName:ebb71089-81ad-44b7-adbb-abc8cf40a9cb nodeName:}" failed. No retries permitted until 2026-07-10 15:55:47.072788853 +0000 UTC m=+36.044822230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls") pod "image-registry-6c4695cf5b-d9pnq" (UID: "ebb71089-81ad-44b7-adbb-abc8cf40a9cb") : secret "image-registry-tls" not found Jul 10 15:55:46.572931 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.572833 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-ca-trust-extracted\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.572931 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.572866 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khtbj\" (UniqueName: \"kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-kube-api-access-khtbj\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.572931 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.572915 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-trusted-ca\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.573417 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.572941 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0159b035-3359-4e09-bb64-9237da0c327c-metrics-tls\") pod \"dns-default-4g9ns\" (UID: \"0159b035-3359-4e09-bb64-9237da0c327c\") " pod="openshift-dns/dns-default-4g9ns" Jul 10 15:55:46.573417 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.573231 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-ca-trust-extracted\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.573417 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.573403 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-certificates\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.573870 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.573848 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-trusted-ca\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.577169 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.577140 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-image-registry-private-configuration\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.577270 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.577146 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-installation-pull-secrets\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.582694 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.582667 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-bound-sa-token\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.583252 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.583233 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khtbj\" (UniqueName: \"kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-kube-api-access-khtbj\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:46.673683 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.673591 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0159b035-3359-4e09-bb64-9237da0c327c-metrics-tls\") pod \"dns-default-4g9ns\" (UID: \"0159b035-3359-4e09-bb64-9237da0c327c\") " pod="openshift-dns/dns-default-4g9ns" Jul 10 15:55:46.673683 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.673639 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0159b035-3359-4e09-bb64-9237da0c327c-tmp-dir\") pod \"dns-default-4g9ns\" (UID: \"0159b035-3359-4e09-bb64-9237da0c327c\") " pod="openshift-dns/dns-default-4g9ns" Jul 10 15:55:46.673904 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.673698 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26flr\" (UniqueName: \"kubernetes.io/projected/0159b035-3359-4e09-bb64-9237da0c327c-kube-api-access-26flr\") pod \"dns-default-4g9ns\" (UID: \"0159b035-3359-4e09-bb64-9237da0c327c\") " pod="openshift-dns/dns-default-4g9ns" Jul 10 15:55:46.673904 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.673746 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x44zf\" (UniqueName: \"kubernetes.io/projected/47620ce7-6020-406a-9b19-92c89d6fbcc2-kube-api-access-x44zf\") pod \"ingress-canary-22xxq\" (UID: \"47620ce7-6020-406a-9b19-92c89d6fbcc2\") " pod="openshift-ingress-canary/ingress-canary-22xxq" Jul 10 15:55:46.673904 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:46.673822 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 15:55:46.673904 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:46.673896 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0159b035-3359-4e09-bb64-9237da0c327c-metrics-tls podName:0159b035-3359-4e09-bb64-9237da0c327c nodeName:}" failed. No retries permitted until 2026-07-10 15:55:47.173873509 +0000 UTC m=+36.145906864 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0159b035-3359-4e09-bb64-9237da0c327c-metrics-tls") pod "dns-default-4g9ns" (UID: "0159b035-3359-4e09-bb64-9237da0c327c") : secret "dns-default-metrics-tls" not found Jul 10 15:55:46.674138 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.673915 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47620ce7-6020-406a-9b19-92c89d6fbcc2-cert\") pod \"ingress-canary-22xxq\" (UID: \"47620ce7-6020-406a-9b19-92c89d6fbcc2\") " pod="openshift-ingress-canary/ingress-canary-22xxq" Jul 10 15:55:46.674138 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.674020 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0159b035-3359-4e09-bb64-9237da0c327c-config-volume\") pod \"dns-default-4g9ns\" (UID: \"0159b035-3359-4e09-bb64-9237da0c327c\") " pod="openshift-dns/dns-default-4g9ns" Jul 10 15:55:46.674227 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.674148 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0159b035-3359-4e09-bb64-9237da0c327c-tmp-dir\") pod \"dns-default-4g9ns\" (UID: \"0159b035-3359-4e09-bb64-9237da0c327c\") " pod="openshift-dns/dns-default-4g9ns" Jul 10 15:55:46.674569 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.674545 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0159b035-3359-4e09-bb64-9237da0c327c-config-volume\") pod \"dns-default-4g9ns\" (UID: \"0159b035-3359-4e09-bb64-9237da0c327c\") " pod="openshift-dns/dns-default-4g9ns" Jul 10 15:55:46.676765 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.676740 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/47620ce7-6020-406a-9b19-92c89d6fbcc2-cert\") pod \"ingress-canary-22xxq\" (UID: \"47620ce7-6020-406a-9b19-92c89d6fbcc2\") " pod="openshift-ingress-canary/ingress-canary-22xxq" Jul 10 15:55:46.683476 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.683454 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x44zf\" (UniqueName: \"kubernetes.io/projected/47620ce7-6020-406a-9b19-92c89d6fbcc2-kube-api-access-x44zf\") pod \"ingress-canary-22xxq\" (UID: \"47620ce7-6020-406a-9b19-92c89d6fbcc2\") " pod="openshift-ingress-canary/ingress-canary-22xxq" Jul 10 15:55:46.683577 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.683457 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26flr\" (UniqueName: \"kubernetes.io/projected/0159b035-3359-4e09-bb64-9237da0c327c-kube-api-access-26flr\") pod \"dns-default-4g9ns\" (UID: \"0159b035-3359-4e09-bb64-9237da0c327c\") " pod="openshift-dns/dns-default-4g9ns" Jul 10 15:55:46.723642 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.723592 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-22xxq" Jul 10 15:55:46.983975 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:46.983934 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-22xxq"] Jul 10 15:55:47.018821 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:47.018792 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47620ce7_6020_406a_9b19_92c89d6fbcc2.slice/crio-58b2f397bd53b38e0c7aab0822cf1da7b5f4545045b126080d15136b4c6085a4 WatchSource:0}: Error finding container 58b2f397bd53b38e0c7aab0822cf1da7b5f4545045b126080d15136b4c6085a4: Status 404 returned error can't find the container with id 58b2f397bd53b38e0c7aab0822cf1da7b5f4545045b126080d15136b4c6085a4 Jul 10 15:55:47.076925 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:47.076894 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:47.077081 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:47.077063 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 15:55:47.077143 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:47.077086 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c4695cf5b-d9pnq: secret "image-registry-tls" not found Jul 10 15:55:47.077192 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:47.077153 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls podName:ebb71089-81ad-44b7-adbb-abc8cf40a9cb nodeName:}" failed. No retries permitted until 2026-07-10 15:55:48.077131703 +0000 UTC m=+37.049165074 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls") pod "image-registry-6c4695cf5b-d9pnq" (UID: "ebb71089-81ad-44b7-adbb-abc8cf40a9cb") : secret "image-registry-tls" not found Jul 10 15:55:47.177683 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:47.177643 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0159b035-3359-4e09-bb64-9237da0c327c-metrics-tls\") pod \"dns-default-4g9ns\" (UID: \"0159b035-3359-4e09-bb64-9237da0c327c\") " pod="openshift-dns/dns-default-4g9ns" Jul 10 15:55:47.177837 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:47.177746 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 15:55:47.177837 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:47.177794 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0159b035-3359-4e09-bb64-9237da0c327c-metrics-tls podName:0159b035-3359-4e09-bb64-9237da0c327c nodeName:}" failed. No retries permitted until 2026-07-10 15:55:48.177781716 +0000 UTC m=+37.149815068 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0159b035-3359-4e09-bb64-9237da0c327c-metrics-tls") pod "dns-default-4g9ns" (UID: "0159b035-3359-4e09-bb64-9237da0c327c") : secret "dns-default-metrics-tls" not found Jul 10 15:55:47.546860 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:47.546772 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:55:47.547392 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:47.546772 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:55:47.547392 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:47.546788 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:55:47.550976 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:47.550823 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 15:55:47.550976 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:47.550835 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 15:55:47.550976 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:47.550857 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 15:55:47.551269 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:47.551155 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 15:55:47.551269 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:47.551220 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zp7tx\"" Jul 10 15:55:47.551343 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:47.551267 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9526x\"" Jul 10 15:55:47.821918 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:47.821829 2560 generic.go:356] "Generic (PLEG): container finished" podID="92d76a94-65a8-457e-8a57-bc3729beeb80" containerID="f7cdb3574d57c5b37e84cdf5b04f4ef57fe516ef905a6afbfa952e6c10f283c8" exitCode=0 Jul 10 15:55:47.822100 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:47.821907 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cfcr7" event={"ID":"92d76a94-65a8-457e-8a57-bc3729beeb80","Type":"ContainerDied","Data":"f7cdb3574d57c5b37e84cdf5b04f4ef57fe516ef905a6afbfa952e6c10f283c8"} Jul 10 15:55:47.823201 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:47.823176 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-22xxq" event={"ID":"47620ce7-6020-406a-9b19-92c89d6fbcc2","Type":"ContainerStarted","Data":"58b2f397bd53b38e0c7aab0822cf1da7b5f4545045b126080d15136b4c6085a4"} Jul 10 15:55:48.083816 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:48.083537 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:48.083997 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:48.083707 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 15:55:48.083997 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:48.083856 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c4695cf5b-d9pnq: secret "image-registry-tls" not found Jul 10 15:55:48.083997 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:48.083925 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls podName:ebb71089-81ad-44b7-adbb-abc8cf40a9cb nodeName:}" failed. No retries permitted until 2026-07-10 15:55:50.083908054 +0000 UTC m=+39.055941407 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls") pod "image-registry-6c4695cf5b-d9pnq" (UID: "ebb71089-81ad-44b7-adbb-abc8cf40a9cb") : secret "image-registry-tls" not found Jul 10 15:55:48.184725 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:48.184675 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0159b035-3359-4e09-bb64-9237da0c327c-metrics-tls\") pod \"dns-default-4g9ns\" (UID: \"0159b035-3359-4e09-bb64-9237da0c327c\") " pod="openshift-dns/dns-default-4g9ns" Jul 10 15:55:48.184912 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:48.184834 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 15:55:48.184974 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:48.184915 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0159b035-3359-4e09-bb64-9237da0c327c-metrics-tls podName:0159b035-3359-4e09-bb64-9237da0c327c nodeName:}" failed. No retries permitted until 2026-07-10 15:55:50.184894007 +0000 UTC m=+39.156927368 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0159b035-3359-4e09-bb64-9237da0c327c-metrics-tls") pod "dns-default-4g9ns" (UID: "0159b035-3359-4e09-bb64-9237da0c327c") : secret "dns-default-metrics-tls" not found Jul 10 15:55:48.830711 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:48.830674 2560 generic.go:356] "Generic (PLEG): container finished" podID="92d76a94-65a8-457e-8a57-bc3729beeb80" containerID="89801b5b6ceeff692c009d2b37bc9b76b668d43f4360359ba0a6bafb9cbf60db" exitCode=0 Jul 10 15:55:48.831187 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:48.830732 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cfcr7" event={"ID":"92d76a94-65a8-457e-8a57-bc3729beeb80","Type":"ContainerDied","Data":"89801b5b6ceeff692c009d2b37bc9b76b668d43f4360359ba0a6bafb9cbf60db"} Jul 10 15:55:49.836075 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:49.836034 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cfcr7" event={"ID":"92d76a94-65a8-457e-8a57-bc3729beeb80","Type":"ContainerStarted","Data":"f9a62c2c5bc16ec386021d2c9498fb45270c41ee2aad2e0a8dc7942277a82709"} Jul 10 15:55:49.837307 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:49.837285 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-22xxq" event={"ID":"47620ce7-6020-406a-9b19-92c89d6fbcc2","Type":"ContainerStarted","Data":"580021eff25a65f1cc4b4abd8f55195dc1d28edbbbd2e7800c17a702ec306ee8"} Jul 10 15:55:49.862780 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:49.862730 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cfcr7" podStartSLOduration=9.410422267 podStartE2EDuration="38.862715895s" totalImagesPullingTime="29.452293628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:11 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:49.860840096 +0000 UTC m=+38.832873469" watchObservedRunningTime="2026-07-10 15:55:49.862715895 +0000 UTC m=+38.834749269" Jul 10 15:55:50.100794 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:50.100710 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:50.100937 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:50.100845 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 15:55:50.100937 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:50.100858 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c4695cf5b-d9pnq: secret "image-registry-tls" not found Jul 10 15:55:50.100937 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:50.100908 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls podName:ebb71089-81ad-44b7-adbb-abc8cf40a9cb nodeName:}" failed. No retries permitted until 2026-07-10 15:55:54.100894619 +0000 UTC m=+43.072927974 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls") pod "image-registry-6c4695cf5b-d9pnq" (UID: "ebb71089-81ad-44b7-adbb-abc8cf40a9cb") : secret "image-registry-tls" not found Jul 10 15:55:50.201926 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:50.201891 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0159b035-3359-4e09-bb64-9237da0c327c-metrics-tls\") pod \"dns-default-4g9ns\" (UID: \"0159b035-3359-4e09-bb64-9237da0c327c\") " pod="openshift-dns/dns-default-4g9ns" Jul 10 15:55:50.202093 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:50.202071 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 15:55:50.202149 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:50.202136 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0159b035-3359-4e09-bb64-9237da0c327c-metrics-tls podName:0159b035-3359-4e09-bb64-9237da0c327c nodeName:}" failed. No retries permitted until 2026-07-10 15:55:54.202119717 +0000 UTC m=+43.174153070 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0159b035-3359-4e09-bb64-9237da0c327c-metrics-tls") pod "dns-default-4g9ns" (UID: "0159b035-3359-4e09-bb64-9237da0c327c") : secret "dns-default-metrics-tls" not found Jul 10 15:55:54.129231 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:54.129188 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:55:54.129628 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:54.129297 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 15:55:54.129628 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:54.129309 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c4695cf5b-d9pnq: secret "image-registry-tls" not found Jul 10 15:55:54.129628 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:54.129357 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls podName:ebb71089-81ad-44b7-adbb-abc8cf40a9cb nodeName:}" failed. No retries permitted until 2026-07-10 15:56:02.129343024 +0000 UTC m=+51.101376376 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls") pod "image-registry-6c4695cf5b-d9pnq" (UID: "ebb71089-81ad-44b7-adbb-abc8cf40a9cb") : secret "image-registry-tls" not found Jul 10 15:55:54.229529 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:54.229492 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0159b035-3359-4e09-bb64-9237da0c327c-metrics-tls\") pod \"dns-default-4g9ns\" (UID: \"0159b035-3359-4e09-bb64-9237da0c327c\") " pod="openshift-dns/dns-default-4g9ns" Jul 10 15:55:54.229627 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:54.229599 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 15:55:54.229661 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:54.229645 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0159b035-3359-4e09-bb64-9237da0c327c-metrics-tls podName:0159b035-3359-4e09-bb64-9237da0c327c nodeName:}" failed. No retries permitted until 2026-07-10 15:56:02.229633357 +0000 UTC m=+51.201666712 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0159b035-3359-4e09-bb64-9237da0c327c-metrics-tls") pod "dns-default-4g9ns" (UID: "0159b035-3359-4e09-bb64-9237da0c327c") : secret "dns-default-metrics-tls" not found Jul 10 15:55:55.779200 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:55.779145 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-22xxq" podStartSLOduration=7.409833954 podStartE2EDuration="9.779127464s" totalImagesPullingTime="2.36929351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:55:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:49.885086765 +0000 UTC m=+38.857120138" watchObservedRunningTime="2026-07-10 15:55:55.779127464 +0000 UTC m=+44.751160817" Jul 10 15:55:55.779580 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:55.779313 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fl48s"] Jul 10 15:55:55.798773 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:55.798748 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:55:55.801428 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:55.801399 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fl48s"] Jul 10 15:55:55.803089 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:55.803067 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 10 15:55:55.803194 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:55.803113 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 10 15:55:55.804127 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:55.804108 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 10 15:55:55.804236 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:55.804144 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 10 15:55:55.804236 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:55.804144 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-n6slx\"" Jul 10 15:55:55.943913 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:55.943869 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7vsf\" (UniqueName: \"kubernetes.io/projected/aaffad33-edc2-43c8-b65d-424739fd004e-kube-api-access-m7vsf\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:55:55.943913 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:55.943909 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aaffad33-edc2-43c8-b65d-424739fd004e-data-volume\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:55:55.944200 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:55.943934 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aaffad33-edc2-43c8-b65d-424739fd004e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:55:55.944200 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:55.944075 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aaffad33-edc2-43c8-b65d-424739fd004e-crio-socket\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:55:55.944200 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:55.944113 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aaffad33-edc2-43c8-b65d-424739fd004e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:55:56.045415 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:56.045331 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7vsf\" (UniqueName: \"kubernetes.io/projected/aaffad33-edc2-43c8-b65d-424739fd004e-kube-api-access-m7vsf\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:55:56.045415 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:56.045376 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aaffad33-edc2-43c8-b65d-424739fd004e-data-volume\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:55:56.045415 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:56.045397 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aaffad33-edc2-43c8-b65d-424739fd004e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:55:56.045684 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:56.045464 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aaffad33-edc2-43c8-b65d-424739fd004e-crio-socket\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:55:56.045684 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:56.045489 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aaffad33-edc2-43c8-b65d-424739fd004e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:55:56.045684 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:56.045544 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 15:55:56.045684 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:56.045589 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 15:55:56.045684 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:56.045652 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aaffad33-edc2-43c8-b65d-424739fd004e-insights-runtime-extractor-tls podName:aaffad33-edc2-43c8-b65d-424739fd004e nodeName:}" failed. No retries permitted until 2026-07-10 15:55:56.545631625 +0000 UTC m=+45.517664985 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aaffad33-edc2-43c8-b65d-424739fd004e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fl48s" (UID: "aaffad33-edc2-43c8-b65d-424739fd004e") : secret "insights-runtime-extractor-tls" not found Jul 10 15:55:56.045684 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:56.045663 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aaffad33-edc2-43c8-b65d-424739fd004e-crio-socket\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:55:56.045684 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:56.045676 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aaffad33-edc2-43c8-b65d-424739fd004e-kube-rbac-proxy-cm podName:aaffad33-edc2-43c8-b65d-424739fd004e nodeName:}" failed. No retries permitted until 2026-07-10 15:55:56.545657833 +0000 UTC m=+45.517691190 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aaffad33-edc2-43c8-b65d-424739fd004e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fl48s" (UID: "aaffad33-edc2-43c8-b65d-424739fd004e") : configmap "kube-rbac-proxy" not found Jul 10 15:55:56.045901 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:56.045718 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aaffad33-edc2-43c8-b65d-424739fd004e-data-volume\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:55:56.055152 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:56.055122 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7vsf\" (UniqueName: \"kubernetes.io/projected/aaffad33-edc2-43c8-b65d-424739fd004e-kube-api-access-m7vsf\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:55:56.294794 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:56.294753 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n5ppz"] Jul 10 15:55:56.318333 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:56.318259 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n5ppz"] Jul 10 15:55:56.318478 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:56.318380 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n5ppz" Jul 10 15:55:56.321234 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:56.321211 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 10 15:55:56.321356 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:56.321211 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-k2dcf\"" Jul 10 15:55:56.321356 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:56.321245 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 10 15:55:56.449425 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:56.449388 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgq6s\" (UniqueName: \"kubernetes.io/projected/436e585c-0f4c-49a0-a6b0-d9a7eed47fab-kube-api-access-jgq6s\") pod \"migrator-7f59b54f6c-n5ppz\" (UID: \"436e585c-0f4c-49a0-a6b0-d9a7eed47fab\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n5ppz" Jul 10 15:55:56.550696 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:56.550656 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aaffad33-edc2-43c8-b65d-424739fd004e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:55:56.550896 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:56.550747 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aaffad33-edc2-43c8-b65d-424739fd004e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:55:56.550896 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:56.550769 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 15:55:56.550896 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:56.550815 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgq6s\" (UniqueName: \"kubernetes.io/projected/436e585c-0f4c-49a0-a6b0-d9a7eed47fab-kube-api-access-jgq6s\") pod \"migrator-7f59b54f6c-n5ppz\" (UID: \"436e585c-0f4c-49a0-a6b0-d9a7eed47fab\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n5ppz" Jul 10 15:55:56.550896 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:56.550837 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aaffad33-edc2-43c8-b65d-424739fd004e-kube-rbac-proxy-cm podName:aaffad33-edc2-43c8-b65d-424739fd004e nodeName:}" failed. No retries permitted until 2026-07-10 15:55:57.550820567 +0000 UTC m=+46.522853924 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aaffad33-edc2-43c8-b65d-424739fd004e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fl48s" (UID: "aaffad33-edc2-43c8-b65d-424739fd004e") : configmap "kube-rbac-proxy" not found Jul 10 15:55:56.550896 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:56.550882 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 15:55:56.551180 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:56.550949 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aaffad33-edc2-43c8-b65d-424739fd004e-insights-runtime-extractor-tls podName:aaffad33-edc2-43c8-b65d-424739fd004e nodeName:}" failed. No retries permitted until 2026-07-10 15:55:57.550933943 +0000 UTC m=+46.522967312 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aaffad33-edc2-43c8-b65d-424739fd004e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fl48s" (UID: "aaffad33-edc2-43c8-b65d-424739fd004e") : secret "insights-runtime-extractor-tls" not found Jul 10 15:55:56.559253 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:56.559232 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgq6s\" (UniqueName: \"kubernetes.io/projected/436e585c-0f4c-49a0-a6b0-d9a7eed47fab-kube-api-access-jgq6s\") pod \"migrator-7f59b54f6c-n5ppz\" (UID: \"436e585c-0f4c-49a0-a6b0-d9a7eed47fab\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n5ppz" Jul 10 15:55:56.627741 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:56.627651 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n5ppz" Jul 10 15:55:56.761472 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:56.761441 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n5ppz"] Jul 10 15:55:56.764353 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:56.764318 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod436e585c_0f4c_49a0_a6b0_d9a7eed47fab.slice/crio-515e6931edc30bf7eb65cb2faa1412b7c30c4a7ed4a45bf7d23844ebaf5a90b9 WatchSource:0}: Error finding container 515e6931edc30bf7eb65cb2faa1412b7c30c4a7ed4a45bf7d23844ebaf5a90b9: Status 404 returned error can't find the container with id 515e6931edc30bf7eb65cb2faa1412b7c30c4a7ed4a45bf7d23844ebaf5a90b9 Jul 10 15:55:56.854055 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:56.854019 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n5ppz" event={"ID":"436e585c-0f4c-49a0-a6b0-d9a7eed47fab","Type":"ContainerStarted","Data":"515e6931edc30bf7eb65cb2faa1412b7c30c4a7ed4a45bf7d23844ebaf5a90b9"} Jul 10 15:55:57.557901 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:57.557871 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aaffad33-edc2-43c8-b65d-424739fd004e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:55:57.558089 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:57.557965 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aaffad33-edc2-43c8-b65d-424739fd004e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:55:57.558089 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:57.558028 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 15:55:57.558089 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:57.558077 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 15:55:57.558237 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:57.558104 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aaffad33-edc2-43c8-b65d-424739fd004e-kube-rbac-proxy-cm podName:aaffad33-edc2-43c8-b65d-424739fd004e nodeName:}" failed. No retries permitted until 2026-07-10 15:55:59.558083526 +0000 UTC m=+48.530116894 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aaffad33-edc2-43c8-b65d-424739fd004e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fl48s" (UID: "aaffad33-edc2-43c8-b65d-424739fd004e") : configmap "kube-rbac-proxy" not found Jul 10 15:55:57.558237 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:57.558127 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aaffad33-edc2-43c8-b65d-424739fd004e-insights-runtime-extractor-tls podName:aaffad33-edc2-43c8-b65d-424739fd004e nodeName:}" failed. No retries permitted until 2026-07-10 15:55:59.558111956 +0000 UTC m=+48.530145312 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aaffad33-edc2-43c8-b65d-424739fd004e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fl48s" (UID: "aaffad33-edc2-43c8-b65d-424739fd004e") : secret "insights-runtime-extractor-tls" not found Jul 10 15:55:57.671396 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:57.671367 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ngcrw_81e1bb75-b227-4f4a-81d4-6843cebccc61/dns-node-resolver/0.log" Jul 10 15:55:58.280730 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.280697 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-mlmtk"] Jul 10 15:55:58.295374 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.295346 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-mlmtk"] Jul 10 15:55:58.295522 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.295474 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-mlmtk" Jul 10 15:55:58.298461 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.298435 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 10 15:55:58.298589 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.298504 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 10 15:55:58.298589 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.298521 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 10 15:55:58.298589 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.298541 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 10 15:55:58.299682 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.299659 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-g2728\"" Jul 10 15:55:58.464811 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.464775 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsrfs\" (UniqueName: \"kubernetes.io/projected/3c110328-eed7-4945-a2bf-6933829d1bd5-kube-api-access-wsrfs\") pod \"service-ca-79f68b9f4f-mlmtk\" (UID: \"3c110328-eed7-4945-a2bf-6933829d1bd5\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mlmtk" Jul 10 15:55:58.465010 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.464831 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3c110328-eed7-4945-a2bf-6933829d1bd5-signing-cabundle\") pod \"service-ca-79f68b9f4f-mlmtk\" (UID: \"3c110328-eed7-4945-a2bf-6933829d1bd5\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mlmtk" Jul 10 15:55:58.465010 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.464873 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3c110328-eed7-4945-a2bf-6933829d1bd5-signing-key\") pod \"service-ca-79f68b9f4f-mlmtk\" (UID: \"3c110328-eed7-4945-a2bf-6933829d1bd5\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mlmtk" Jul 10 15:55:58.471205 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.471184 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8vfqs_18946076-1464-4aac-bbe1-3c4d2b4b6ae9/node-ca/0.log" Jul 10 15:55:58.566239 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.566204 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsrfs\" (UniqueName: \"kubernetes.io/projected/3c110328-eed7-4945-a2bf-6933829d1bd5-kube-api-access-wsrfs\") pod \"service-ca-79f68b9f4f-mlmtk\" (UID: \"3c110328-eed7-4945-a2bf-6933829d1bd5\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mlmtk" Jul 10 15:55:58.566395 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.566274 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3c110328-eed7-4945-a2bf-6933829d1bd5-signing-cabundle\") pod \"service-ca-79f68b9f4f-mlmtk\" (UID: \"3c110328-eed7-4945-a2bf-6933829d1bd5\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mlmtk" Jul 10 15:55:58.566395 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.566340 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3c110328-eed7-4945-a2bf-6933829d1bd5-signing-key\") pod \"service-ca-79f68b9f4f-mlmtk\" (UID: \"3c110328-eed7-4945-a2bf-6933829d1bd5\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mlmtk" Jul 10 15:55:58.566849 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.566830 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3c110328-eed7-4945-a2bf-6933829d1bd5-signing-cabundle\") pod \"service-ca-79f68b9f4f-mlmtk\" (UID: \"3c110328-eed7-4945-a2bf-6933829d1bd5\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mlmtk" Jul 10 15:55:58.568521 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.568501 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3c110328-eed7-4945-a2bf-6933829d1bd5-signing-key\") pod \"service-ca-79f68b9f4f-mlmtk\" (UID: \"3c110328-eed7-4945-a2bf-6933829d1bd5\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mlmtk" Jul 10 15:55:58.575008 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.574988 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsrfs\" (UniqueName: \"kubernetes.io/projected/3c110328-eed7-4945-a2bf-6933829d1bd5-kube-api-access-wsrfs\") pod \"service-ca-79f68b9f4f-mlmtk\" (UID: \"3c110328-eed7-4945-a2bf-6933829d1bd5\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mlmtk" Jul 10 15:55:58.604235 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.604204 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-mlmtk" Jul 10 15:55:58.759644 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.759621 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-mlmtk"] Jul 10 15:55:58.762403 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:55:58.762374 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c110328_eed7_4945_a2bf_6933829d1bd5.slice/crio-c633e904cc58bc972e67d04b37ac0b3f38a45855e0e6151ef42571db80c92dfa WatchSource:0}: Error finding container c633e904cc58bc972e67d04b37ac0b3f38a45855e0e6151ef42571db80c92dfa: Status 404 returned error can't find the container with id c633e904cc58bc972e67d04b37ac0b3f38a45855e0e6151ef42571db80c92dfa Jul 10 15:55:58.860710 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.860613 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n5ppz" event={"ID":"436e585c-0f4c-49a0-a6b0-d9a7eed47fab","Type":"ContainerStarted","Data":"7c56629ca9db66b07a27f5c1cd4e45cc22e62378e1ad06954bff43efe614cf66"} Jul 10 15:55:58.860710 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.860671 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n5ppz" event={"ID":"436e585c-0f4c-49a0-a6b0-d9a7eed47fab","Type":"ContainerStarted","Data":"a4ea0910b8a085cbed2ca5462e9337fe55eeff643c09687bbb47aef4b38ff029"} Jul 10 15:55:58.861709 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.861684 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-mlmtk" event={"ID":"3c110328-eed7-4945-a2bf-6933829d1bd5","Type":"ContainerStarted","Data":"c633e904cc58bc972e67d04b37ac0b3f38a45855e0e6151ef42571db80c92dfa"} Jul 10 15:55:58.881239 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:58.881189 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n5ppz" podStartSLOduration=1.162196377 podStartE2EDuration="2.881169662s" totalImagesPullingTime="1.718973285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:55:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:55:58.881111598 +0000 UTC m=+47.853144989" watchObservedRunningTime="2026-07-10 15:55:58.881169662 +0000 UTC m=+47.853203036" Jul 10 15:55:59.071596 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:59.071564 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-22xxq_47620ce7-6020-406a-9b19-92c89d6fbcc2/serve-healthcheck-canary/0.log" Jul 10 15:55:59.577331 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:59.577299 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aaffad33-edc2-43c8-b65d-424739fd004e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:55:59.577760 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:55:59.577375 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aaffad33-edc2-43c8-b65d-424739fd004e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:55:59.577760 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:59.577442 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 15:55:59.577760 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:59.577506 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 15:55:59.577760 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:59.577526 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aaffad33-edc2-43c8-b65d-424739fd004e-kube-rbac-proxy-cm podName:aaffad33-edc2-43c8-b65d-424739fd004e nodeName:}" failed. No retries permitted until 2026-07-10 15:56:03.577503422 +0000 UTC m=+52.549536785 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aaffad33-edc2-43c8-b65d-424739fd004e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fl48s" (UID: "aaffad33-edc2-43c8-b65d-424739fd004e") : configmap "kube-rbac-proxy" not found Jul 10 15:55:59.577760 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:55:59.577548 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aaffad33-edc2-43c8-b65d-424739fd004e-insights-runtime-extractor-tls podName:aaffad33-edc2-43c8-b65d-424739fd004e nodeName:}" failed. No retries permitted until 2026-07-10 15:56:03.57753747 +0000 UTC m=+52.549570858 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aaffad33-edc2-43c8-b65d-424739fd004e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fl48s" (UID: "aaffad33-edc2-43c8-b65d-424739fd004e") : secret "insights-runtime-extractor-tls" not found Jul 10 15:56:01.871538 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:01.871499 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-mlmtk" event={"ID":"3c110328-eed7-4945-a2bf-6933829d1bd5","Type":"ContainerStarted","Data":"3f31b8771e79b450e814dd8c6d103d4095c708d74167f6cb5339e5e8ffd4391e"} Jul 10 15:56:01.891825 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:01.891748 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-mlmtk" podStartSLOduration=1.6207491059999999 podStartE2EDuration="3.891729128s" totalImagesPullingTime="2.270980022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:55:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:01.89147649 +0000 UTC m=+50.863509864" watchObservedRunningTime="2026-07-10 15:56:01.891729128 +0000 UTC m=+50.863762505" Jul 10 15:56:02.199089 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:02.198993 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:56:02.199254 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:56:02.199144 2560 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 10 15:56:02.199254 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:56:02.199160 2560 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c4695cf5b-d9pnq: secret "image-registry-tls" not found Jul 10 15:56:02.199254 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:56:02.199211 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls podName:ebb71089-81ad-44b7-adbb-abc8cf40a9cb nodeName:}" failed. No retries permitted until 2026-07-10 15:56:18.199196447 +0000 UTC m=+67.171229800 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls") pod "image-registry-6c4695cf5b-d9pnq" (UID: "ebb71089-81ad-44b7-adbb-abc8cf40a9cb") : secret "image-registry-tls" not found Jul 10 15:56:02.299597 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:02.299555 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0159b035-3359-4e09-bb64-9237da0c327c-metrics-tls\") pod \"dns-default-4g9ns\" (UID: \"0159b035-3359-4e09-bb64-9237da0c327c\") " pod="openshift-dns/dns-default-4g9ns" Jul 10 15:56:02.299766 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:56:02.299669 2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 10 15:56:02.299766 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:56:02.299719 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0159b035-3359-4e09-bb64-9237da0c327c-metrics-tls podName:0159b035-3359-4e09-bb64-9237da0c327c nodeName:}" failed. No retries permitted until 2026-07-10 15:56:18.299705671 +0000 UTC m=+67.271739025 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0159b035-3359-4e09-bb64-9237da0c327c-metrics-tls") pod "dns-default-4g9ns" (UID: "0159b035-3359-4e09-bb64-9237da0c327c") : secret "dns-default-metrics-tls" not found Jul 10 15:56:03.610326 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:03.610289 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aaffad33-edc2-43c8-b65d-424739fd004e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:56:03.610721 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:03.610364 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aaffad33-edc2-43c8-b65d-424739fd004e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:56:03.610721 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:56:03.610448 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 15:56:03.610721 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:56:03.610466 2560 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 10 15:56:03.610721 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:56:03.610509 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aaffad33-edc2-43c8-b65d-424739fd004e-insights-runtime-extractor-tls podName:aaffad33-edc2-43c8-b65d-424739fd004e nodeName:}" failed. No retries permitted until 2026-07-10 15:56:11.610491997 +0000 UTC m=+60.582525352 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aaffad33-edc2-43c8-b65d-424739fd004e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fl48s" (UID: "aaffad33-edc2-43c8-b65d-424739fd004e") : secret "insights-runtime-extractor-tls" not found Jul 10 15:56:03.610721 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:56:03.610526 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aaffad33-edc2-43c8-b65d-424739fd004e-kube-rbac-proxy-cm podName:aaffad33-edc2-43c8-b65d-424739fd004e nodeName:}" failed. No retries permitted until 2026-07-10 15:56:11.610516125 +0000 UTC m=+60.582549477 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/aaffad33-edc2-43c8-b65d-424739fd004e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fl48s" (UID: "aaffad33-edc2-43c8-b65d-424739fd004e") : configmap "kube-rbac-proxy" not found Jul 10 15:56:11.676342 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:11.676304 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aaffad33-edc2-43c8-b65d-424739fd004e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:56:11.676908 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:11.676401 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aaffad33-edc2-43c8-b65d-424739fd004e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:56:11.676908 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:56:11.676573 2560 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 10 15:56:11.676908 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:56:11.676661 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aaffad33-edc2-43c8-b65d-424739fd004e-insights-runtime-extractor-tls podName:aaffad33-edc2-43c8-b65d-424739fd004e nodeName:}" failed. No retries permitted until 2026-07-10 15:56:27.676639282 +0000 UTC m=+76.648672670 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/aaffad33-edc2-43c8-b65d-424739fd004e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fl48s" (UID: "aaffad33-edc2-43c8-b65d-424739fd004e") : secret "insights-runtime-extractor-tls" not found Jul 10 15:56:11.676908 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:11.676797 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aaffad33-edc2-43c8-b65d-424739fd004e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:56:11.812536 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:11.812505 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6s2sn" Jul 10 15:56:17.219373 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:17.219322 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs\") pod \"network-metrics-daemon-mjxwx\" (UID: \"049e242d-f002-42b0-b116-e9baa5f8eb87\") " pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:56:17.222195 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:17.222174 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 10 15:56:17.232284 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:17.232259 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/049e242d-f002-42b0-b116-e9baa5f8eb87-metrics-certs\") pod \"network-metrics-daemon-mjxwx\" (UID: \"049e242d-f002-42b0-b116-e9baa5f8eb87\") " pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:56:17.275471 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:17.275440 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zp7tx\"" Jul 10 15:56:17.283727 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:17.283704 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mjxwx" Jul 10 15:56:17.320267 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:17.320227 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret\") pod \"global-pull-secret-syncer-h9z25\" (UID: \"7511b558-f953-4c6b-a90b-c2843281f2cc\") " pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:56:17.323179 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:17.323157 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 10 15:56:17.333617 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:17.333591 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7511b558-f953-4c6b-a90b-c2843281f2cc-original-pull-secret\") pod \"global-pull-secret-syncer-h9z25\" (UID: \"7511b558-f953-4c6b-a90b-c2843281f2cc\") " pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:56:17.404204 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:17.404170 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mjxwx"] Jul 10 15:56:17.407004 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:56:17.406973 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod049e242d_f002_42b0_b116_e9baa5f8eb87.slice/crio-69fe88b0022bf4c6271cbbea2a987f0cb79786028efe3c9c72b233eab1cb1084 WatchSource:0}: Error finding container 69fe88b0022bf4c6271cbbea2a987f0cb79786028efe3c9c72b233eab1cb1084: Status 404 returned error can't find the container with id 69fe88b0022bf4c6271cbbea2a987f0cb79786028efe3c9c72b233eab1cb1084 Jul 10 15:56:17.421318 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:17.421296 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z98ml\" (UniqueName: \"kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml\") pod \"network-check-target-2h276\" (UID: \"ff0e4720-f5ab-4832-b118-eb5fd37dd8ab\") " pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:56:17.424208 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:17.424191 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 10 15:56:17.434345 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:17.434326 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 10 15:56:17.444775 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:17.444753 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z98ml\" (UniqueName: \"kubernetes.io/projected/ff0e4720-f5ab-4832-b118-eb5fd37dd8ab-kube-api-access-z98ml\") pod \"network-check-target-2h276\" (UID: \"ff0e4720-f5ab-4832-b118-eb5fd37dd8ab\") " pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:56:17.558930 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:17.558854 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9z25" Jul 10 15:56:17.569605 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:17.569583 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9526x\"" Jul 10 15:56:17.577673 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:17.577645 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:56:17.689194 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:17.689162 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h9z25"] Jul 10 15:56:17.692028 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:56:17.692003 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7511b558_f953_4c6b_a90b_c2843281f2cc.slice/crio-63b0a785da149e23876e0bb787886c3bb2d91a9c11bdccda299419c503bfe9cc WatchSource:0}: Error finding container 63b0a785da149e23876e0bb787886c3bb2d91a9c11bdccda299419c503bfe9cc: Status 404 returned error can't find the container with id 63b0a785da149e23876e0bb787886c3bb2d91a9c11bdccda299419c503bfe9cc Jul 10 15:56:17.704639 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:17.704612 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2h276"] Jul 10 15:56:17.707799 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:56:17.707773 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff0e4720_f5ab_4832_b118_eb5fd37dd8ab.slice/crio-dac233e19bb1041f6359cf644589643d25a975ad2e93e512e0637e85e74f58a5 WatchSource:0}: Error finding container dac233e19bb1041f6359cf644589643d25a975ad2e93e512e0637e85e74f58a5: Status 404 returned error can't find the container with id dac233e19bb1041f6359cf644589643d25a975ad2e93e512e0637e85e74f58a5 Jul 10 15:56:17.914861 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:17.914761 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h9z25" event={"ID":"7511b558-f953-4c6b-a90b-c2843281f2cc","Type":"ContainerStarted","Data":"63b0a785da149e23876e0bb787886c3bb2d91a9c11bdccda299419c503bfe9cc"} Jul 10 15:56:17.915764 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:17.915737 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2h276" event={"ID":"ff0e4720-f5ab-4832-b118-eb5fd37dd8ab","Type":"ContainerStarted","Data":"dac233e19bb1041f6359cf644589643d25a975ad2e93e512e0637e85e74f58a5"} Jul 10 15:56:17.916858 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:17.916830 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mjxwx" event={"ID":"049e242d-f002-42b0-b116-e9baa5f8eb87","Type":"ContainerStarted","Data":"69fe88b0022bf4c6271cbbea2a987f0cb79786028efe3c9c72b233eab1cb1084"} Jul 10 15:56:18.226345 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:18.226303 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:56:18.229770 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:18.229732 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls\") pod \"image-registry-6c4695cf5b-d9pnq\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:56:18.327136 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:18.326986 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0159b035-3359-4e09-bb64-9237da0c327c-metrics-tls\") pod \"dns-default-4g9ns\" (UID: \"0159b035-3359-4e09-bb64-9237da0c327c\") " pod="openshift-dns/dns-default-4g9ns" Jul 10 15:56:18.329983 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:18.329915 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0159b035-3359-4e09-bb64-9237da0c327c-metrics-tls\") pod \"dns-default-4g9ns\" (UID: \"0159b035-3359-4e09-bb64-9237da0c327c\") " pod="openshift-dns/dns-default-4g9ns" Jul 10 15:56:18.516902 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:18.516818 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5drgf\"" Jul 10 15:56:18.524224 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:18.524198 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:56:18.543326 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:18.543298 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tn9vq\"" Jul 10 15:56:18.550829 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:18.550807 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4g9ns" Jul 10 15:56:18.773787 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:18.773708 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4g9ns"] Jul 10 15:56:18.780178 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:56:18.780146 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0159b035_3359_4e09_bb64_9237da0c327c.slice/crio-01b8c26c05ced9bb220604fe95ad86973c28b1dc3832f69c9585bfef2a436a5b WatchSource:0}: Error finding container 01b8c26c05ced9bb220604fe95ad86973c28b1dc3832f69c9585bfef2a436a5b: Status 404 returned error can't find the container with id 01b8c26c05ced9bb220604fe95ad86973c28b1dc3832f69c9585bfef2a436a5b Jul 10 15:56:18.802202 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:18.802122 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c4695cf5b-d9pnq"] Jul 10 15:56:18.807045 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:56:18.807010 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebb71089_81ad_44b7_adbb_abc8cf40a9cb.slice/crio-3a921d31bf8cac6301f38bc70c75de5ec7f0aa4482e3f991f0cc168ff491c6b4 WatchSource:0}: Error finding container 3a921d31bf8cac6301f38bc70c75de5ec7f0aa4482e3f991f0cc168ff491c6b4: Status 404 returned error can't find the container with id 3a921d31bf8cac6301f38bc70c75de5ec7f0aa4482e3f991f0cc168ff491c6b4 Jul 10 15:56:18.923926 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:18.923886 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mjxwx" event={"ID":"049e242d-f002-42b0-b116-e9baa5f8eb87","Type":"ContainerStarted","Data":"01ba492cca30a0c1643d7e33e98f07ca7fa3bf446b8a901d02b25d294c213bea"} Jul 10 15:56:18.924125 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:18.923976 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mjxwx" event={"ID":"049e242d-f002-42b0-b116-e9baa5f8eb87","Type":"ContainerStarted","Data":"f6060e152d695d7b16d0a8ea4a58accadffc06d74cfb41bfcc9d54c90edc8fbb"} Jul 10 15:56:18.926815 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:18.926784 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" event={"ID":"ebb71089-81ad-44b7-adbb-abc8cf40a9cb","Type":"ContainerStarted","Data":"442e36c776881a67182570ebc54b354c644efdb30970321abe1004e4b915061d"} Jul 10 15:56:18.926944 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:18.926835 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" event={"ID":"ebb71089-81ad-44b7-adbb-abc8cf40a9cb","Type":"ContainerStarted","Data":"3a921d31bf8cac6301f38bc70c75de5ec7f0aa4482e3f991f0cc168ff491c6b4"} Jul 10 15:56:18.927327 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:18.927305 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:56:18.929709 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:18.929685 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4g9ns" event={"ID":"0159b035-3359-4e09-bb64-9237da0c327c","Type":"ContainerStarted","Data":"01b8c26c05ced9bb220604fe95ad86973c28b1dc3832f69c9585bfef2a436a5b"} Jul 10 15:56:18.941541 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:18.941492 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mjxwx" podStartSLOduration=66.71171378 podStartE2EDuration="1m7.941467063s" totalImagesPullingTime="1.229753283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:55:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:18.941324732 +0000 UTC m=+67.913358108" watchObservedRunningTime="2026-07-10 15:56:18.941467063 +0000 UTC m=+67.913500439" Jul 10 15:56:18.962169 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:18.962114 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" podStartSLOduration=63.962093154 podStartE2EDuration="1m3.962093154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:55:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:18.959892142 +0000 UTC m=+67.931925523" watchObservedRunningTime="2026-07-10 15:56:18.962093154 +0000 UTC m=+67.934126531" Jul 10 15:56:20.339770 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.339731 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-586875c878-6tk75"] Jul 10 15:56:20.357895 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.357831 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fbf97d5fd-zj684"] Jul 10 15:56:20.359166 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.358185 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-586875c878-6tk75" Jul 10 15:56:20.361175 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.360975 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 10 15:56:20.361175 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.361069 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 10 15:56:20.361490 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.361467 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 10 15:56:20.362928 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.362876 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 10 15:56:20.376079 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.376042 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-586875c878-6tk75"] Jul 10 15:56:20.376198 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.376156 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fbf97d5fd-zj684" Jul 10 15:56:20.376245 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.376206 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fbf97d5fd-zj684"] Jul 10 15:56:20.379394 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.379376 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-5svjh\"" Jul 10 15:56:20.379697 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.379678 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 10 15:56:20.447862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.447693 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ca82a3b5-f8eb-4bee-8bf9-6964fe736668-klusterlet-config\") pod \"klusterlet-addon-workmgr-586875c878-6tk75\" (UID: \"ca82a3b5-f8eb-4bee-8bf9-6964fe736668\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-586875c878-6tk75" Jul 10 15:56:20.447862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.447766 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b7b5\" (UniqueName: \"kubernetes.io/projected/ca82a3b5-f8eb-4bee-8bf9-6964fe736668-kube-api-access-6b7b5\") pod \"klusterlet-addon-workmgr-586875c878-6tk75\" (UID: \"ca82a3b5-f8eb-4bee-8bf9-6964fe736668\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-586875c878-6tk75" Jul 10 15:56:20.447862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.447805 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa8112cf-6615-4a74-82c1-b4167240fc1c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5fbf97d5fd-zj684\" (UID: \"fa8112cf-6615-4a74-82c1-b4167240fc1c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fbf97d5fd-zj684" Jul 10 15:56:20.447862 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.447846 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcb9n\" (UniqueName: \"kubernetes.io/projected/fa8112cf-6615-4a74-82c1-b4167240fc1c-kube-api-access-gcb9n\") pod \"managed-serviceaccount-addon-agent-5fbf97d5fd-zj684\" (UID: \"fa8112cf-6615-4a74-82c1-b4167240fc1c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fbf97d5fd-zj684" Jul 10 15:56:20.448325 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.447883 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca82a3b5-f8eb-4bee-8bf9-6964fe736668-tmp\") pod \"klusterlet-addon-workmgr-586875c878-6tk75\" (UID: \"ca82a3b5-f8eb-4bee-8bf9-6964fe736668\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-586875c878-6tk75" Jul 10 15:56:20.449363 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.449335 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6c4695cf5b-d9pnq"] Jul 10 15:56:20.499213 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.499171 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b954bbc8f-7tjtk"] Jul 10 15:56:20.509905 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.509872 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.548938 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.548896 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8w8r\" (UniqueName: \"kubernetes.io/projected/60bc3007-ed90-4dff-8a98-4dfd79ab3965-kube-api-access-h8w8r\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.549151 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.548972 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/60bc3007-ed90-4dff-8a98-4dfd79ab3965-ca-trust-extracted\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.549151 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.549005 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcb9n\" (UniqueName: \"kubernetes.io/projected/fa8112cf-6615-4a74-82c1-b4167240fc1c-kube-api-access-gcb9n\") pod \"managed-serviceaccount-addon-agent-5fbf97d5fd-zj684\" (UID: \"fa8112cf-6615-4a74-82c1-b4167240fc1c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fbf97d5fd-zj684" Jul 10 15:56:20.549151 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.549023 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/60bc3007-ed90-4dff-8a98-4dfd79ab3965-image-registry-private-configuration\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.549151 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.549052 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/60bc3007-ed90-4dff-8a98-4dfd79ab3965-trusted-ca\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.549151 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.549085 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca82a3b5-f8eb-4bee-8bf9-6964fe736668-tmp\") pod \"klusterlet-addon-workmgr-586875c878-6tk75\" (UID: \"ca82a3b5-f8eb-4bee-8bf9-6964fe736668\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-586875c878-6tk75" Jul 10 15:56:20.549151 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.549116 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/60bc3007-ed90-4dff-8a98-4dfd79ab3965-registry-certificates\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.549151 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.549144 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/60bc3007-ed90-4dff-8a98-4dfd79ab3965-installation-pull-secrets\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.549497 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.549174 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/60bc3007-ed90-4dff-8a98-4dfd79ab3965-registry-tls\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.549497 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.549199 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/60bc3007-ed90-4dff-8a98-4dfd79ab3965-bound-sa-token\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.549497 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.549223 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ca82a3b5-f8eb-4bee-8bf9-6964fe736668-klusterlet-config\") pod \"klusterlet-addon-workmgr-586875c878-6tk75\" (UID: \"ca82a3b5-f8eb-4bee-8bf9-6964fe736668\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-586875c878-6tk75" Jul 10 15:56:20.549497 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.549280 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b7b5\" (UniqueName: \"kubernetes.io/projected/ca82a3b5-f8eb-4bee-8bf9-6964fe736668-kube-api-access-6b7b5\") pod \"klusterlet-addon-workmgr-586875c878-6tk75\" (UID: \"ca82a3b5-f8eb-4bee-8bf9-6964fe736668\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-586875c878-6tk75" Jul 10 15:56:20.549497 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.549319 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa8112cf-6615-4a74-82c1-b4167240fc1c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5fbf97d5fd-zj684\" (UID: \"fa8112cf-6615-4a74-82c1-b4167240fc1c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fbf97d5fd-zj684" Jul 10 15:56:20.549497 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.549495 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ca82a3b5-f8eb-4bee-8bf9-6964fe736668-tmp\") pod \"klusterlet-addon-workmgr-586875c878-6tk75\" (UID: \"ca82a3b5-f8eb-4bee-8bf9-6964fe736668\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-586875c878-6tk75" Jul 10 15:56:20.552496 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.552467 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fa8112cf-6615-4a74-82c1-b4167240fc1c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5fbf97d5fd-zj684\" (UID: \"fa8112cf-6615-4a74-82c1-b4167240fc1c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fbf97d5fd-zj684" Jul 10 15:56:20.552640 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.552467 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ca82a3b5-f8eb-4bee-8bf9-6964fe736668-klusterlet-config\") pod \"klusterlet-addon-workmgr-586875c878-6tk75\" (UID: \"ca82a3b5-f8eb-4bee-8bf9-6964fe736668\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-586875c878-6tk75" Jul 10 15:56:20.556017 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.555948 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b954bbc8f-7tjtk"] Jul 10 15:56:20.571538 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.571504 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b7b5\" (UniqueName: \"kubernetes.io/projected/ca82a3b5-f8eb-4bee-8bf9-6964fe736668-kube-api-access-6b7b5\") pod \"klusterlet-addon-workmgr-586875c878-6tk75\" (UID: \"ca82a3b5-f8eb-4bee-8bf9-6964fe736668\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-586875c878-6tk75" Jul 10 15:56:20.580811 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.580764 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcb9n\" (UniqueName: \"kubernetes.io/projected/fa8112cf-6615-4a74-82c1-b4167240fc1c-kube-api-access-gcb9n\") pod \"managed-serviceaccount-addon-agent-5fbf97d5fd-zj684\" (UID: \"fa8112cf-6615-4a74-82c1-b4167240fc1c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fbf97d5fd-zj684" Jul 10 15:56:20.650816 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.650728 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/60bc3007-ed90-4dff-8a98-4dfd79ab3965-trusted-ca\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.650816 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.650786 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/60bc3007-ed90-4dff-8a98-4dfd79ab3965-registry-certificates\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.651055 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.650921 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/60bc3007-ed90-4dff-8a98-4dfd79ab3965-installation-pull-secrets\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.651055 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.651016 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/60bc3007-ed90-4dff-8a98-4dfd79ab3965-registry-tls\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.651055 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.651045 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/60bc3007-ed90-4dff-8a98-4dfd79ab3965-bound-sa-token\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.651684 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.651241 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8w8r\" (UniqueName: \"kubernetes.io/projected/60bc3007-ed90-4dff-8a98-4dfd79ab3965-kube-api-access-h8w8r\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.651684 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.651283 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/60bc3007-ed90-4dff-8a98-4dfd79ab3965-ca-trust-extracted\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.651684 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.651333 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/60bc3007-ed90-4dff-8a98-4dfd79ab3965-image-registry-private-configuration\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.651684 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.651664 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/60bc3007-ed90-4dff-8a98-4dfd79ab3965-registry-certificates\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.651684 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.651676 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/60bc3007-ed90-4dff-8a98-4dfd79ab3965-ca-trust-extracted\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.652065 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.651796 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/60bc3007-ed90-4dff-8a98-4dfd79ab3965-trusted-ca\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.653789 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.653765 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/60bc3007-ed90-4dff-8a98-4dfd79ab3965-installation-pull-secrets\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.653939 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.653916 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/60bc3007-ed90-4dff-8a98-4dfd79ab3965-registry-tls\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.653939 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.653931 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/60bc3007-ed90-4dff-8a98-4dfd79ab3965-image-registry-private-configuration\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.663042 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.663018 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8w8r\" (UniqueName: \"kubernetes.io/projected/60bc3007-ed90-4dff-8a98-4dfd79ab3965-kube-api-access-h8w8r\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.663274 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.663251 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/60bc3007-ed90-4dff-8a98-4dfd79ab3965-bound-sa-token\") pod \"image-registry-6b954bbc8f-7tjtk\" (UID: \"60bc3007-ed90-4dff-8a98-4dfd79ab3965\") " pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:20.673155 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.673121 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-586875c878-6tk75" Jul 10 15:56:20.699449 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.699424 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fbf97d5fd-zj684" Jul 10 15:56:20.821799 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:20.821763 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:23.525924 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:23.525846 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fbf97d5fd-zj684"] Jul 10 15:56:23.544112 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:56:23.544061 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa8112cf_6615_4a74_82c1_b4167240fc1c.slice/crio-423e31ccfb866b89aecd4868dec338cdc66867fa99833dd0c5f1893b3f7a9159 WatchSource:0}: Error finding container 423e31ccfb866b89aecd4868dec338cdc66867fa99833dd0c5f1893b3f7a9159: Status 404 returned error can't find the container with id 423e31ccfb866b89aecd4868dec338cdc66867fa99833dd0c5f1893b3f7a9159 Jul 10 15:56:23.553074 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:23.553036 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-586875c878-6tk75"] Jul 10 15:56:23.557998 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:56:23.557916 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca82a3b5_f8eb_4bee_8bf9_6964fe736668.slice/crio-39e3e9b1f7f900630da67b91bafa1bfd7cfd09f7b4dd1f2942160009c7eaa196 WatchSource:0}: Error finding container 39e3e9b1f7f900630da67b91bafa1bfd7cfd09f7b4dd1f2942160009c7eaa196: Status 404 returned error can't find the container with id 39e3e9b1f7f900630da67b91bafa1bfd7cfd09f7b4dd1f2942160009c7eaa196 Jul 10 15:56:23.573025 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:23.572997 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b954bbc8f-7tjtk"] Jul 10 15:56:23.575798 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:56:23.575739 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60bc3007_ed90_4dff_8a98_4dfd79ab3965.slice/crio-6c0654ef616e65a8509cffb4d8301a4eefac73286aa8ae43f53a2cf6acd6debf WatchSource:0}: Error finding container 6c0654ef616e65a8509cffb4d8301a4eefac73286aa8ae43f53a2cf6acd6debf: Status 404 returned error can't find the container with id 6c0654ef616e65a8509cffb4d8301a4eefac73286aa8ae43f53a2cf6acd6debf Jul 10 15:56:23.950112 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:23.950031 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4g9ns" event={"ID":"0159b035-3359-4e09-bb64-9237da0c327c","Type":"ContainerStarted","Data":"5223c107280c96384c837d5f447e22c3fca82a1e95dfa77b9e9813633efeaa40"} Jul 10 15:56:23.950112 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:23.950095 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4g9ns" event={"ID":"0159b035-3359-4e09-bb64-9237da0c327c","Type":"ContainerStarted","Data":"3b708374e0fec83fb0f4d0019f6202ce7f3877d4a570f365ed33a35b76573644"} Jul 10 15:56:23.950326 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:23.950154 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4g9ns" Jul 10 15:56:23.952191 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:23.952165 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" event={"ID":"60bc3007-ed90-4dff-8a98-4dfd79ab3965","Type":"ContainerStarted","Data":"d7c122e8c966e60bfae345a4bbb7fdf26574994b8bb3afa399d6e46d29ee19a0"} Jul 10 15:56:23.952324 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:23.952205 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" event={"ID":"60bc3007-ed90-4dff-8a98-4dfd79ab3965","Type":"ContainerStarted","Data":"6c0654ef616e65a8509cffb4d8301a4eefac73286aa8ae43f53a2cf6acd6debf"} Jul 10 15:56:23.952324 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:23.952228 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:23.954051 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:23.953993 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fbf97d5fd-zj684" event={"ID":"fa8112cf-6615-4a74-82c1-b4167240fc1c","Type":"ContainerStarted","Data":"423e31ccfb866b89aecd4868dec338cdc66867fa99833dd0c5f1893b3f7a9159"} Jul 10 15:56:23.960008 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:23.959972 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h9z25" event={"ID":"7511b558-f953-4c6b-a90b-c2843281f2cc","Type":"ContainerStarted","Data":"992bc922ea58e5ff56d9a2f7ceb507274d5a9fefcd7d31a9c4cab19422ccb540"} Jul 10 15:56:23.961590 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:23.961567 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2h276" event={"ID":"ff0e4720-f5ab-4832-b118-eb5fd37dd8ab","Type":"ContainerStarted","Data":"9f5568bfb38fc35fa5b7f219f1045358c1345304862f098a52831323d5a34699"} Jul 10 15:56:23.961692 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:23.961637 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:56:23.962808 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:23.962763 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-586875c878-6tk75" event={"ID":"ca82a3b5-f8eb-4bee-8bf9-6964fe736668","Type":"ContainerStarted","Data":"39e3e9b1f7f900630da67b91bafa1bfd7cfd09f7b4dd1f2942160009c7eaa196"} Jul 10 15:56:23.968721 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:23.968632 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4g9ns" podStartSLOduration=33.408957529 podStartE2EDuration="37.968616208s" totalImagesPullingTime="4.559658679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:55:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:23.968335199 +0000 UTC m=+72.940368573" watchObservedRunningTime="2026-07-10 15:56:23.968616208 +0000 UTC m=+72.940649582" Jul 10 15:56:23.984064 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:23.984013 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-h9z25" podStartSLOduration=66.320429673 podStartE2EDuration="1m11.983998845s" totalImagesPullingTime="5.663569172s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:23.9830971 +0000 UTC m=+72.955130476" watchObservedRunningTime="2026-07-10 15:56:23.983998845 +0000 UTC m=+72.956032197" Jul 10 15:56:24.002640 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:24.002583 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" podStartSLOduration=4.002548122 podStartE2EDuration="4.002548122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:56:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:24.002287458 +0000 UTC m=+72.974320830" watchObservedRunningTime="2026-07-10 15:56:24.002548122 +0000 UTC m=+72.974581498" Jul 10 15:56:24.018641 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:24.018588 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2h276" podStartSLOduration=67.379341845 podStartE2EDuration="1m13.018566844s" totalImagesPullingTime="5.639224999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:55:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:24.017886368 +0000 UTC m=+72.989919756" watchObservedRunningTime="2026-07-10 15:56:24.018566844 +0000 UTC m=+72.990600220" Jul 10 15:56:27.711620 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:27.711581 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aaffad33-edc2-43c8-b65d-424739fd004e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:56:27.713805 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:27.713784 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aaffad33-edc2-43c8-b65d-424739fd004e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fl48s\" (UID: \"aaffad33-edc2-43c8-b65d-424739fd004e\") " pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:56:27.911327 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:27.911296 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-n6slx\"" Jul 10 15:56:27.918583 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:27.918551 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fl48s" Jul 10 15:56:27.982069 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:27.981903 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-586875c878-6tk75" event={"ID":"ca82a3b5-f8eb-4bee-8bf9-6964fe736668","Type":"ContainerStarted","Data":"0b03b5f2024813c4db456db864465328d53ac2feafa4c9671d29fb703ea3961e"} Jul 10 15:56:27.982543 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:27.982513 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-586875c878-6tk75" Jul 10 15:56:27.984204 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:27.984184 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-586875c878-6tk75" Jul 10 15:56:27.984517 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:27.984497 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fbf97d5fd-zj684" event={"ID":"fa8112cf-6615-4a74-82c1-b4167240fc1c","Type":"ContainerStarted","Data":"453927d77b80bf4016232fdc01a5a0a4cdc98457d571ff6be04d281cde8d8246"} Jul 10 15:56:28.000362 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:28.000072 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-586875c878-6tk75" podStartSLOduration=4.280208811 podStartE2EDuration="8.000053421s" totalImagesPullingTime="3.71984461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:56:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:27.999113179 +0000 UTC m=+76.971146556" watchObservedRunningTime="2026-07-10 15:56:28.000053421 +0000 UTC m=+76.972086797" Jul 10 15:56:28.038927 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:28.038870 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fbf97d5fd-zj684" podStartSLOduration=4.319515442 podStartE2EDuration="8.038850882s" totalImagesPullingTime="3.71933544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 15:56:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:28.038593504 +0000 UTC m=+77.010626895" watchObservedRunningTime="2026-07-10 15:56:28.038850882 +0000 UTC m=+77.010884257" Jul 10 15:56:28.054628 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:28.054598 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fl48s"] Jul 10 15:56:28.057932 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:56:28.057906 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaffad33_edc2_43c8_b65d_424739fd004e.slice/crio-ae42168357920ebefce4f7f566e5b4da66ee4d2f4d2f69f24b83b0cd4b742c5a WatchSource:0}: Error finding container ae42168357920ebefce4f7f566e5b4da66ee4d2f4d2f69f24b83b0cd4b742c5a: Status 404 returned error can't find the container with id ae42168357920ebefce4f7f566e5b4da66ee4d2f4d2f69f24b83b0cd4b742c5a Jul 10 15:56:28.991151 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:28.990984 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fl48s" event={"ID":"aaffad33-edc2-43c8-b65d-424739fd004e","Type":"ContainerStarted","Data":"ae42168357920ebefce4f7f566e5b4da66ee4d2f4d2f69f24b83b0cd4b742c5a"} Jul 10 15:56:32.005731 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:32.005692 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fl48s" event={"ID":"aaffad33-edc2-43c8-b65d-424739fd004e","Type":"ContainerStarted","Data":"4b670d35b4d411310ff8a508314fa48ef568875f8f2efe22d8abcdbc041de956"} Jul 10 15:56:33.011509 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:33.011469 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fl48s" event={"ID":"aaffad33-edc2-43c8-b65d-424739fd004e","Type":"ContainerStarted","Data":"69acd934c697aec98c29774641dd269fed98cb0af365634fbe87dab6b96638de"} Jul 10 15:56:33.971030 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:33.970999 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4g9ns" Jul 10 15:56:35.020821 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.020777 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fl48s" event={"ID":"aaffad33-edc2-43c8-b65d-424739fd004e","Type":"ContainerStarted","Data":"1f8b6f56c5c03e51d1a26b7c96588408132ec864ce33da9bf010be9e59b1d373"} Jul 10 15:56:35.050385 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.049996 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fl48s" podStartSLOduration=33.292270955 podStartE2EDuration="40.049972957s" totalImagesPullingTime="6.757702002s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 15:55:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:35.049198962 +0000 UTC m=+84.021232353" watchObservedRunningTime="2026-07-10 15:56:35.049972957 +0000 UTC m=+84.022006325" Jul 10 15:56:35.323025 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.322989 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hdxsb"] Jul 10 15:56:35.332815 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.332786 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.336377 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.336349 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 10 15:56:35.336513 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.336353 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 10 15:56:35.338102 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.338075 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 10 15:56:35.338376 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.338352 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-r4hpk\"" Jul 10 15:56:35.338586 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.338569 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 10 15:56:35.338758 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.338742 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 10 15:56:35.338923 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.338908 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 10 15:56:35.471688 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.471646 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1347d2e6-ddae-479d-92d5-603b85a892c3-node-exporter-tls\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.471688 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.471696 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1347d2e6-ddae-479d-92d5-603b85a892c3-metrics-client-ca\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.471967 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.471727 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1347d2e6-ddae-479d-92d5-603b85a892c3-root\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.471967 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.471753 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1347d2e6-ddae-479d-92d5-603b85a892c3-node-exporter-textfile\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.471967 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.471784 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1347d2e6-ddae-479d-92d5-603b85a892c3-node-exporter-wtmp\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.471967 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.471819 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1347d2e6-ddae-479d-92d5-603b85a892c3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.471967 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.471842 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdfd4\" (UniqueName: \"kubernetes.io/projected/1347d2e6-ddae-479d-92d5-603b85a892c3-kube-api-access-pdfd4\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.471967 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.471882 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1347d2e6-ddae-479d-92d5-603b85a892c3-sys\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.471967 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.471938 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1347d2e6-ddae-479d-92d5-603b85a892c3-node-exporter-accelerators-collector-config\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.572630 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.572593 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1347d2e6-ddae-479d-92d5-603b85a892c3-node-exporter-tls\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.572630 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.572626 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1347d2e6-ddae-479d-92d5-603b85a892c3-metrics-client-ca\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.572870 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.572649 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1347d2e6-ddae-479d-92d5-603b85a892c3-root\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.572870 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.572666 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1347d2e6-ddae-479d-92d5-603b85a892c3-node-exporter-textfile\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.572870 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.572696 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1347d2e6-ddae-479d-92d5-603b85a892c3-node-exporter-wtmp\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.572870 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.572730 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1347d2e6-ddae-479d-92d5-603b85a892c3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.572870 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.572736 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1347d2e6-ddae-479d-92d5-603b85a892c3-root\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.572870 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.572754 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdfd4\" (UniqueName: \"kubernetes.io/projected/1347d2e6-ddae-479d-92d5-603b85a892c3-kube-api-access-pdfd4\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.572870 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.572783 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1347d2e6-ddae-479d-92d5-603b85a892c3-sys\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.572870 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.572812 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1347d2e6-ddae-479d-92d5-603b85a892c3-node-exporter-accelerators-collector-config\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.572870 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.572865 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1347d2e6-ddae-479d-92d5-603b85a892c3-node-exporter-wtmp\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.573331 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.573086 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1347d2e6-ddae-479d-92d5-603b85a892c3-node-exporter-textfile\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.573331 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.573157 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1347d2e6-ddae-479d-92d5-603b85a892c3-sys\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.573414 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.573326 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1347d2e6-ddae-479d-92d5-603b85a892c3-metrics-client-ca\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.573414 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.573401 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1347d2e6-ddae-479d-92d5-603b85a892c3-node-exporter-accelerators-collector-config\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.575199 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.575172 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1347d2e6-ddae-479d-92d5-603b85a892c3-node-exporter-tls\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.575303 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.575221 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1347d2e6-ddae-479d-92d5-603b85a892c3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.581247 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.581227 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdfd4\" (UniqueName: \"kubernetes.io/projected/1347d2e6-ddae-479d-92d5-603b85a892c3-kube-api-access-pdfd4\") pod \"node-exporter-hdxsb\" (UID: \"1347d2e6-ddae-479d-92d5-603b85a892c3\") " pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.644707 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:35.644680 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hdxsb" Jul 10 15:56:35.654624 ip-10-0-129-101 kubenswrapper[2560]: W0710 15:56:35.654596 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1347d2e6_ddae_479d_92d5_603b85a892c3.slice/crio-1d5aeb227efc5aa269757523dd8d47eeff53fe647f4521cb1ae48eef46b46165 WatchSource:0}: Error finding container 1d5aeb227efc5aa269757523dd8d47eeff53fe647f4521cb1ae48eef46b46165: Status 404 returned error can't find the container with id 1d5aeb227efc5aa269757523dd8d47eeff53fe647f4521cb1ae48eef46b46165 Jul 10 15:56:36.025287 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:36.025250 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hdxsb" event={"ID":"1347d2e6-ddae-479d-92d5-603b85a892c3","Type":"ContainerStarted","Data":"1d5aeb227efc5aa269757523dd8d47eeff53fe647f4521cb1ae48eef46b46165"} Jul 10 15:56:37.033019 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:37.032976 2560 generic.go:356] "Generic (PLEG): container finished" podID="1347d2e6-ddae-479d-92d5-603b85a892c3" containerID="434e47d5b252891af4e21744d4f853315d5a4761bdbd7088a38f7514b0e38bd0" exitCode=0 Jul 10 15:56:37.033462 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:37.033030 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hdxsb" event={"ID":"1347d2e6-ddae-479d-92d5-603b85a892c3","Type":"ContainerDied","Data":"434e47d5b252891af4e21744d4f853315d5a4761bdbd7088a38f7514b0e38bd0"} Jul 10 15:56:38.038341 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:38.038306 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hdxsb" event={"ID":"1347d2e6-ddae-479d-92d5-603b85a892c3","Type":"ContainerStarted","Data":"2774c4b0798ea70d284f999414b5887263953e095f585387e8cd3b94d2301d9e"} Jul 10 15:56:38.038710 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:38.038368 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hdxsb" event={"ID":"1347d2e6-ddae-479d-92d5-603b85a892c3","Type":"ContainerStarted","Data":"ebe5210e8ae69e7b14623ef2be11b3c7a5aca12140e72f1cc370f4be6f23e9e8"} Jul 10 15:56:38.061106 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:38.061051 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hdxsb" podStartSLOduration=1.9935341420000001 podStartE2EDuration="3.061035216s" totalImagesPullingTime="1.004165607s" totalInitContainerRuntime="63.335467ms" isStatefulPod=true podCreationTimestamp="2026-07-10 15:56:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 15:56:38.059506773 +0000 UTC m=+87.031540146" watchObservedRunningTime="2026-07-10 15:56:38.061035216 +0000 UTC m=+87.033068647" Jul 10 15:56:41.946910 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:41.946877 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:56:44.973163 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:44.973131 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6b954bbc8f-7tjtk" Jul 10 15:56:46.960626 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:46.960554 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" podUID="ebb71089-81ad-44b7-adbb-abc8cf40a9cb" containerName="registry" containerID="cri-o://442e36c776881a67182570ebc54b354c644efdb30970321abe1004e4b915061d" gracePeriod=30 Jul 10 15:56:47.200806 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.200778 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:56:47.363530 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.363434 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khtbj\" (UniqueName: \"kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-kube-api-access-khtbj\") pod \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " Jul 10 15:56:47.363530 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.363479 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-image-registry-private-configuration\") pod \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " Jul 10 15:56:47.363530 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.363525 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-ca-trust-extracted\") pod \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " Jul 10 15:56:47.363780 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.363580 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-trusted-ca\") pod \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " Jul 10 15:56:47.363780 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.363609 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls\") pod \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " Jul 10 15:56:47.363780 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.363633 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-bound-sa-token\") pod \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " Jul 10 15:56:47.363780 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.363660 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-installation-pull-secrets\") pod \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " Jul 10 15:56:47.363780 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.363690 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-certificates\") pod \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\" (UID: \"ebb71089-81ad-44b7-adbb-abc8cf40a9cb\") " Jul 10 15:56:47.364258 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.364197 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ebb71089-81ad-44b7-adbb-abc8cf40a9cb" (UID: "ebb71089-81ad-44b7-adbb-abc8cf40a9cb"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 15:56:47.364374 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.364257 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ebb71089-81ad-44b7-adbb-abc8cf40a9cb" (UID: "ebb71089-81ad-44b7-adbb-abc8cf40a9cb"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 15:56:47.366458 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.366377 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ebb71089-81ad-44b7-adbb-abc8cf40a9cb" (UID: "ebb71089-81ad-44b7-adbb-abc8cf40a9cb"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 15:56:47.366458 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.366412 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ebb71089-81ad-44b7-adbb-abc8cf40a9cb" (UID: "ebb71089-81ad-44b7-adbb-abc8cf40a9cb"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 15:56:47.366610 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.366462 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ebb71089-81ad-44b7-adbb-abc8cf40a9cb" (UID: "ebb71089-81ad-44b7-adbb-abc8cf40a9cb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 15:56:47.366610 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.366518 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ebb71089-81ad-44b7-adbb-abc8cf40a9cb" (UID: "ebb71089-81ad-44b7-adbb-abc8cf40a9cb"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 15:56:47.366610 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.366540 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-kube-api-access-khtbj" (OuterVolumeSpecName: "kube-api-access-khtbj") pod "ebb71089-81ad-44b7-adbb-abc8cf40a9cb" (UID: "ebb71089-81ad-44b7-adbb-abc8cf40a9cb"). InnerVolumeSpecName "kube-api-access-khtbj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 15:56:47.373011 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.372941 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ebb71089-81ad-44b7-adbb-abc8cf40a9cb" (UID: "ebb71089-81ad-44b7-adbb-abc8cf40a9cb"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 15:56:47.464969 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.464927 2560 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-tls\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 15:56:47.465127 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.464989 2560 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-bound-sa-token\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 15:56:47.465127 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.465002 2560 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-installation-pull-secrets\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 15:56:47.465127 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.465015 2560 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-registry-certificates\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 15:56:47.465127 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.465027 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khtbj\" (UniqueName: \"kubernetes.io/projected/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-kube-api-access-khtbj\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 15:56:47.465127 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.465040 2560 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-image-registry-private-configuration\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 15:56:47.465127 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.465052 2560 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-ca-trust-extracted\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 15:56:47.465127 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:47.465063 2560 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebb71089-81ad-44b7-adbb-abc8cf40a9cb-trusted-ca\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 15:56:48.072326 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:48.072289 2560 generic.go:356] "Generic (PLEG): container finished" podID="ebb71089-81ad-44b7-adbb-abc8cf40a9cb" containerID="442e36c776881a67182570ebc54b354c644efdb30970321abe1004e4b915061d" exitCode=0 Jul 10 15:56:48.072843 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:48.072339 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" event={"ID":"ebb71089-81ad-44b7-adbb-abc8cf40a9cb","Type":"ContainerDied","Data":"442e36c776881a67182570ebc54b354c644efdb30970321abe1004e4b915061d"} Jul 10 15:56:48.072843 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:48.072370 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" Jul 10 15:56:48.072843 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:48.072391 2560 scope.go:117] "RemoveContainer" containerID="442e36c776881a67182570ebc54b354c644efdb30970321abe1004e4b915061d" Jul 10 15:56:48.072843 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:48.072380 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c4695cf5b-d9pnq" event={"ID":"ebb71089-81ad-44b7-adbb-abc8cf40a9cb","Type":"ContainerDied","Data":"3a921d31bf8cac6301f38bc70c75de5ec7f0aa4482e3f991f0cc168ff491c6b4"} Jul 10 15:56:48.080802 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:48.080781 2560 scope.go:117] "RemoveContainer" containerID="442e36c776881a67182570ebc54b354c644efdb30970321abe1004e4b915061d" Jul 10 15:56:48.081203 ip-10-0-129-101 kubenswrapper[2560]: E0710 15:56:48.081179 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"442e36c776881a67182570ebc54b354c644efdb30970321abe1004e4b915061d\": container with ID starting with 442e36c776881a67182570ebc54b354c644efdb30970321abe1004e4b915061d not found: ID does not exist" containerID="442e36c776881a67182570ebc54b354c644efdb30970321abe1004e4b915061d" Jul 10 15:56:48.081262 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:48.081212 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"442e36c776881a67182570ebc54b354c644efdb30970321abe1004e4b915061d"} err="failed to get container status \"442e36c776881a67182570ebc54b354c644efdb30970321abe1004e4b915061d\": rpc error: code = NotFound desc = could not find container \"442e36c776881a67182570ebc54b354c644efdb30970321abe1004e4b915061d\": container with ID starting with 442e36c776881a67182570ebc54b354c644efdb30970321abe1004e4b915061d not found: ID does not exist" Jul 10 15:56:48.093088 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:48.093063 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6c4695cf5b-d9pnq"] Jul 10 15:56:48.095614 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:48.095584 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6c4695cf5b-d9pnq"] Jul 10 15:56:49.551350 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:49.551314 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebb71089-81ad-44b7-adbb-abc8cf40a9cb" path="/var/lib/kubelet/pods/ebb71089-81ad-44b7-adbb-abc8cf40a9cb/volumes" Jul 10 15:56:54.971569 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:56:54.971540 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2h276" Jul 10 15:57:22.301508 ip-10-0-129-101 kubenswrapper[2560]: I0710 15:57:22.301478 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-22xxq_47620ce7-6020-406a-9b19-92c89d6fbcc2/serve-healthcheck-canary/0.log" Jul 10 16:00:11.406827 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:11.406788 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6s2sn_a7a2569d-7313-49a4-b283-aa416f5fdd9e/ovn-acl-logging/0.log" Jul 10 16:00:11.408414 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:11.408389 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6s2sn_a7a2569d-7313-49a4-b283-aa416f5fdd9e/ovn-acl-logging/0.log" Jul 10 16:00:11.413923 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:11.413903 2560 kubelet.go:1648] "Image garbage collection succeeded" Jul 10 16:00:18.041512 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:18.041476 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7"] Jul 10 16:00:18.043047 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:18.041799 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebb71089-81ad-44b7-adbb-abc8cf40a9cb" containerName="registry" Jul 10 16:00:18.043047 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:18.041815 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebb71089-81ad-44b7-adbb-abc8cf40a9cb" containerName="registry" Jul 10 16:00:18.043047 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:18.041887 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebb71089-81ad-44b7-adbb-abc8cf40a9cb" containerName="registry" Jul 10 16:00:18.043663 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:18.043647 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7" Jul 10 16:00:18.046181 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:18.046159 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 16:00:18.046279 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:18.046190 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 16:00:18.047162 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:18.047141 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sltc4\"" Jul 10 16:00:18.053228 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:18.053203 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7"] Jul 10 16:00:18.084675 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:18.084652 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e091f567-6ebc-4bd8-a450-138af6e68986-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7\" (UID: \"e091f567-6ebc-4bd8-a450-138af6e68986\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7" Jul 10 16:00:18.084792 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:18.084686 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e091f567-6ebc-4bd8-a450-138af6e68986-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7\" (UID: \"e091f567-6ebc-4bd8-a450-138af6e68986\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7" Jul 10 16:00:18.084792 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:18.084718 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znss7\" (UniqueName: \"kubernetes.io/projected/e091f567-6ebc-4bd8-a450-138af6e68986-kube-api-access-znss7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7\" (UID: \"e091f567-6ebc-4bd8-a450-138af6e68986\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7" Jul 10 16:00:18.185316 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:18.185272 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znss7\" (UniqueName: \"kubernetes.io/projected/e091f567-6ebc-4bd8-a450-138af6e68986-kube-api-access-znss7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7\" (UID: \"e091f567-6ebc-4bd8-a450-138af6e68986\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7" Jul 10 16:00:18.185474 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:18.185339 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e091f567-6ebc-4bd8-a450-138af6e68986-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7\" (UID: \"e091f567-6ebc-4bd8-a450-138af6e68986\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7" Jul 10 16:00:18.185474 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:18.185366 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e091f567-6ebc-4bd8-a450-138af6e68986-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7\" (UID: \"e091f567-6ebc-4bd8-a450-138af6e68986\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7" Jul 10 16:00:18.185686 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:18.185671 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e091f567-6ebc-4bd8-a450-138af6e68986-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7\" (UID: \"e091f567-6ebc-4bd8-a450-138af6e68986\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7" Jul 10 16:00:18.185721 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:18.185697 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e091f567-6ebc-4bd8-a450-138af6e68986-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7\" (UID: \"e091f567-6ebc-4bd8-a450-138af6e68986\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7" Jul 10 16:00:18.193715 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:18.193686 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znss7\" (UniqueName: \"kubernetes.io/projected/e091f567-6ebc-4bd8-a450-138af6e68986-kube-api-access-znss7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7\" (UID: \"e091f567-6ebc-4bd8-a450-138af6e68986\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7" Jul 10 16:00:18.352884 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:18.352799 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7" Jul 10 16:00:18.472686 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:18.472658 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7"] Jul 10 16:00:18.475545 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:00:18.475516 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode091f567_6ebc_4bd8_a450_138af6e68986.slice/crio-61e1d5b29f0bd67c67478171cacc8fd995b94f48304f39f9ffea22b69502a5d6 WatchSource:0}: Error finding container 61e1d5b29f0bd67c67478171cacc8fd995b94f48304f39f9ffea22b69502a5d6: Status 404 returned error can't find the container with id 61e1d5b29f0bd67c67478171cacc8fd995b94f48304f39f9ffea22b69502a5d6 Jul 10 16:00:18.477376 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:18.477358 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 16:00:18.755325 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:18.755291 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7" event={"ID":"e091f567-6ebc-4bd8-a450-138af6e68986","Type":"ContainerStarted","Data":"61e1d5b29f0bd67c67478171cacc8fd995b94f48304f39f9ffea22b69502a5d6"} Jul 10 16:00:27.786660 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:27.786626 2560 generic.go:356] "Generic (PLEG): container finished" podID="e091f567-6ebc-4bd8-a450-138af6e68986" containerID="1b30e008abd6d639bee323ad225908dd6cdbb22ea60336d4953a8463502906d9" exitCode=0 Jul 10 16:00:27.787179 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:27.786671 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7" event={"ID":"e091f567-6ebc-4bd8-a450-138af6e68986","Type":"ContainerDied","Data":"1b30e008abd6d639bee323ad225908dd6cdbb22ea60336d4953a8463502906d9"} Jul 10 16:00:30.799469 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:30.799433 2560 generic.go:356] "Generic (PLEG): container finished" podID="e091f567-6ebc-4bd8-a450-138af6e68986" containerID="dba9001e0660ed256a88bef22e7a80b93ba37cdc0c2c1e2698d30bbb57edb922" exitCode=0 Jul 10 16:00:30.799829 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:30.799490 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7" event={"ID":"e091f567-6ebc-4bd8-a450-138af6e68986","Type":"ContainerDied","Data":"dba9001e0660ed256a88bef22e7a80b93ba37cdc0c2c1e2698d30bbb57edb922"} Jul 10 16:00:39.832740 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:39.832641 2560 generic.go:356] "Generic (PLEG): container finished" podID="e091f567-6ebc-4bd8-a450-138af6e68986" containerID="93517739a6b60b70755e7d860be68cbefada0738ce5f273a2c3f4aac5a5d6706" exitCode=0 Jul 10 16:00:39.832740 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:39.832708 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7" event={"ID":"e091f567-6ebc-4bd8-a450-138af6e68986","Type":"ContainerDied","Data":"93517739a6b60b70755e7d860be68cbefada0738ce5f273a2c3f4aac5a5d6706"} Jul 10 16:00:40.971986 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:40.971939 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7" Jul 10 16:00:41.057287 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:41.057250 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znss7\" (UniqueName: \"kubernetes.io/projected/e091f567-6ebc-4bd8-a450-138af6e68986-kube-api-access-znss7\") pod \"e091f567-6ebc-4bd8-a450-138af6e68986\" (UID: \"e091f567-6ebc-4bd8-a450-138af6e68986\") " Jul 10 16:00:41.057467 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:41.057323 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e091f567-6ebc-4bd8-a450-138af6e68986-util\") pod \"e091f567-6ebc-4bd8-a450-138af6e68986\" (UID: \"e091f567-6ebc-4bd8-a450-138af6e68986\") " Jul 10 16:00:41.057467 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:41.057370 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e091f567-6ebc-4bd8-a450-138af6e68986-bundle\") pod \"e091f567-6ebc-4bd8-a450-138af6e68986\" (UID: \"e091f567-6ebc-4bd8-a450-138af6e68986\") " Jul 10 16:00:41.058083 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:41.058057 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e091f567-6ebc-4bd8-a450-138af6e68986-bundle" (OuterVolumeSpecName: "bundle") pod "e091f567-6ebc-4bd8-a450-138af6e68986" (UID: "e091f567-6ebc-4bd8-a450-138af6e68986"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:00:41.059493 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:41.059468 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e091f567-6ebc-4bd8-a450-138af6e68986-kube-api-access-znss7" (OuterVolumeSpecName: "kube-api-access-znss7") pod "e091f567-6ebc-4bd8-a450-138af6e68986" (UID: "e091f567-6ebc-4bd8-a450-138af6e68986"). InnerVolumeSpecName "kube-api-access-znss7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:00:41.061357 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:41.061334 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e091f567-6ebc-4bd8-a450-138af6e68986-util" (OuterVolumeSpecName: "util") pod "e091f567-6ebc-4bd8-a450-138af6e68986" (UID: "e091f567-6ebc-4bd8-a450-138af6e68986"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:00:41.158060 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:41.157971 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e091f567-6ebc-4bd8-a450-138af6e68986-util\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:00:41.158060 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:41.158008 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e091f567-6ebc-4bd8-a450-138af6e68986-bundle\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:00:41.158060 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:41.158021 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znss7\" (UniqueName: \"kubernetes.io/projected/e091f567-6ebc-4bd8-a450-138af6e68986-kube-api-access-znss7\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:00:41.841615 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:41.841537 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7" event={"ID":"e091f567-6ebc-4bd8-a450-138af6e68986","Type":"ContainerDied","Data":"61e1d5b29f0bd67c67478171cacc8fd995b94f48304f39f9ffea22b69502a5d6"} Jul 10 16:00:41.841615 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:41.841572 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540rtvs7" Jul 10 16:00:41.841615 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:41.841588 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61e1d5b29f0bd67c67478171cacc8fd995b94f48304f39f9ffea22b69502a5d6" Jul 10 16:00:46.062233 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:46.062201 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s7flf"] Jul 10 16:00:46.062635 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:46.062455 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e091f567-6ebc-4bd8-a450-138af6e68986" containerName="pull" Jul 10 16:00:46.062635 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:46.062466 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e091f567-6ebc-4bd8-a450-138af6e68986" containerName="pull" Jul 10 16:00:46.062635 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:46.062477 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e091f567-6ebc-4bd8-a450-138af6e68986" containerName="util" Jul 10 16:00:46.062635 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:46.062482 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e091f567-6ebc-4bd8-a450-138af6e68986" containerName="util" Jul 10 16:00:46.062635 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:46.062490 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e091f567-6ebc-4bd8-a450-138af6e68986" containerName="extract" Jul 10 16:00:46.062635 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:46.062500 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="e091f567-6ebc-4bd8-a450-138af6e68986" containerName="extract" Jul 10 16:00:46.062635 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:46.062552 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="e091f567-6ebc-4bd8-a450-138af6e68986" containerName="extract" Jul 10 16:00:46.066076 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:46.066057 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s7flf" Jul 10 16:00:46.068766 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:46.068744 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 10 16:00:46.068875 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:46.068809 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 10 16:00:46.068875 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:46.068810 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-qwp9q\"" Jul 10 16:00:46.082485 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:46.082462 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s7flf"] Jul 10 16:00:46.196427 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:46.196394 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe4a0f79-8f57-4680-92ff-882a9a761220-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-s7flf\" (UID: \"fe4a0f79-8f57-4680-92ff-882a9a761220\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s7flf" Jul 10 16:00:46.196594 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:46.196434 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbkbf\" (UniqueName: \"kubernetes.io/projected/fe4a0f79-8f57-4680-92ff-882a9a761220-kube-api-access-gbkbf\") pod \"cert-manager-operator-controller-manager-55c444bf86-s7flf\" (UID: \"fe4a0f79-8f57-4680-92ff-882a9a761220\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s7flf" Jul 10 16:00:46.297312 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:46.297280 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbkbf\" (UniqueName: \"kubernetes.io/projected/fe4a0f79-8f57-4680-92ff-882a9a761220-kube-api-access-gbkbf\") pod \"cert-manager-operator-controller-manager-55c444bf86-s7flf\" (UID: \"fe4a0f79-8f57-4680-92ff-882a9a761220\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s7flf" Jul 10 16:00:46.297459 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:46.297342 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe4a0f79-8f57-4680-92ff-882a9a761220-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-s7flf\" (UID: \"fe4a0f79-8f57-4680-92ff-882a9a761220\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s7flf" Jul 10 16:00:46.297749 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:46.297732 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe4a0f79-8f57-4680-92ff-882a9a761220-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-s7flf\" (UID: \"fe4a0f79-8f57-4680-92ff-882a9a761220\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s7flf" Jul 10 16:00:46.309152 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:46.309130 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbkbf\" (UniqueName: \"kubernetes.io/projected/fe4a0f79-8f57-4680-92ff-882a9a761220-kube-api-access-gbkbf\") pod \"cert-manager-operator-controller-manager-55c444bf86-s7flf\" (UID: \"fe4a0f79-8f57-4680-92ff-882a9a761220\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s7flf" Jul 10 16:00:46.374375 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:46.374305 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s7flf" Jul 10 16:00:46.501640 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:46.501618 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s7flf"] Jul 10 16:00:46.504411 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:00:46.504386 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe4a0f79_8f57_4680_92ff_882a9a761220.slice/crio-9438670a3a83ac93ba0877fda64e3810042809683a0e44aab13b49bd235cb5db WatchSource:0}: Error finding container 9438670a3a83ac93ba0877fda64e3810042809683a0e44aab13b49bd235cb5db: Status 404 returned error can't find the container with id 9438670a3a83ac93ba0877fda64e3810042809683a0e44aab13b49bd235cb5db Jul 10 16:00:46.858025 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:46.857989 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s7flf" event={"ID":"fe4a0f79-8f57-4680-92ff-882a9a761220","Type":"ContainerStarted","Data":"9438670a3a83ac93ba0877fda64e3810042809683a0e44aab13b49bd235cb5db"} Jul 10 16:00:48.868591 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:48.868551 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s7flf" event={"ID":"fe4a0f79-8f57-4680-92ff-882a9a761220","Type":"ContainerStarted","Data":"3646c6872b9576060a58c7912a7631e9b89224431f5257667eef2b70c6692f04"} Jul 10 16:00:48.906965 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:48.906904 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-s7flf" podStartSLOduration=1.342550592 podStartE2EDuration="2.90688756s" totalImagesPullingTime="1.564336968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:00:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:00:48.905878951 +0000 UTC m=+337.877912326" watchObservedRunningTime="2026-07-10 16:00:48.90688756 +0000 UTC m=+337.878920934" Jul 10 16:00:49.970612 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:49.970578 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg"] Jul 10 16:00:49.972652 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:49.972634 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg" Jul 10 16:00:49.975993 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:49.975936 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 16:00:49.977561 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:49.977542 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 16:00:49.977675 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:49.977551 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sltc4\"" Jul 10 16:00:49.984474 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:49.984455 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg"] Jul 10 16:00:50.099145 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.099053 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8jwrg"] Jul 10 16:00:50.101086 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.101072 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8jwrg" Jul 10 16:00:50.104745 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.104728 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-dgncq\"" Jul 10 16:00:50.105346 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.105332 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 10 16:00:50.105599 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.105586 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 10 16:00:50.114426 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.114402 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8jwrg"] Jul 10 16:00:50.123457 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.123435 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a0b86a74-04a0-4c7f-bc11-a15015426b9f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg\" (UID: \"a0b86a74-04a0-4c7f-bc11-a15015426b9f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg" Jul 10 16:00:50.123556 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.123463 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a0b86a74-04a0-4c7f-bc11-a15015426b9f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg\" (UID: \"a0b86a74-04a0-4c7f-bc11-a15015426b9f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg" Jul 10 16:00:50.123556 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.123488 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6kx2\" (UniqueName: \"kubernetes.io/projected/a0b86a74-04a0-4c7f-bc11-a15015426b9f-kube-api-access-j6kx2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg\" (UID: \"a0b86a74-04a0-4c7f-bc11-a15015426b9f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg" Jul 10 16:00:50.224449 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.224419 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvxpd\" (UniqueName: \"kubernetes.io/projected/9c359674-b066-4b29-ae66-82fc1052705d-kube-api-access-vvxpd\") pod \"cert-manager-webhook-6486ffd997-8jwrg\" (UID: \"9c359674-b066-4b29-ae66-82fc1052705d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8jwrg" Jul 10 16:00:50.224636 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.224480 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a0b86a74-04a0-4c7f-bc11-a15015426b9f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg\" (UID: \"a0b86a74-04a0-4c7f-bc11-a15015426b9f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg" Jul 10 16:00:50.224636 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.224498 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a0b86a74-04a0-4c7f-bc11-a15015426b9f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg\" (UID: \"a0b86a74-04a0-4c7f-bc11-a15015426b9f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg" Jul 10 16:00:50.224636 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.224522 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9c359674-b066-4b29-ae66-82fc1052705d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8jwrg\" (UID: \"9c359674-b066-4b29-ae66-82fc1052705d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8jwrg" Jul 10 16:00:50.224636 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.224540 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6kx2\" (UniqueName: \"kubernetes.io/projected/a0b86a74-04a0-4c7f-bc11-a15015426b9f-kube-api-access-j6kx2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg\" (UID: \"a0b86a74-04a0-4c7f-bc11-a15015426b9f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg" Jul 10 16:00:50.224848 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.224809 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a0b86a74-04a0-4c7f-bc11-a15015426b9f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg\" (UID: \"a0b86a74-04a0-4c7f-bc11-a15015426b9f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg" Jul 10 16:00:50.224901 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.224886 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a0b86a74-04a0-4c7f-bc11-a15015426b9f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg\" (UID: \"a0b86a74-04a0-4c7f-bc11-a15015426b9f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg" Jul 10 16:00:50.233281 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.233254 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6kx2\" (UniqueName: \"kubernetes.io/projected/a0b86a74-04a0-4c7f-bc11-a15015426b9f-kube-api-access-j6kx2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg\" (UID: \"a0b86a74-04a0-4c7f-bc11-a15015426b9f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg" Jul 10 16:00:50.281990 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.281942 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg" Jul 10 16:00:50.325259 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.325225 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9c359674-b066-4b29-ae66-82fc1052705d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8jwrg\" (UID: \"9c359674-b066-4b29-ae66-82fc1052705d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8jwrg" Jul 10 16:00:50.325445 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.325275 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvxpd\" (UniqueName: \"kubernetes.io/projected/9c359674-b066-4b29-ae66-82fc1052705d-kube-api-access-vvxpd\") pod \"cert-manager-webhook-6486ffd997-8jwrg\" (UID: \"9c359674-b066-4b29-ae66-82fc1052705d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8jwrg" Jul 10 16:00:50.334857 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.334793 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9c359674-b066-4b29-ae66-82fc1052705d-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-8jwrg\" (UID: \"9c359674-b066-4b29-ae66-82fc1052705d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8jwrg" Jul 10 16:00:50.335154 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.335131 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvxpd\" (UniqueName: \"kubernetes.io/projected/9c359674-b066-4b29-ae66-82fc1052705d-kube-api-access-vvxpd\") pod \"cert-manager-webhook-6486ffd997-8jwrg\" (UID: \"9c359674-b066-4b29-ae66-82fc1052705d\") " pod="cert-manager/cert-manager-webhook-6486ffd997-8jwrg" Jul 10 16:00:50.409741 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.409699 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-8jwrg" Jul 10 16:00:50.410332 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.410304 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg"] Jul 10 16:00:50.412573 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:00:50.412550 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0b86a74_04a0_4c7f_bc11_a15015426b9f.slice/crio-244993d0266f78f5c9e25360ce998fdde32fadf6216c1d810533c4866885a52d WatchSource:0}: Error finding container 244993d0266f78f5c9e25360ce998fdde32fadf6216c1d810533c4866885a52d: Status 404 returned error can't find the container with id 244993d0266f78f5c9e25360ce998fdde32fadf6216c1d810533c4866885a52d Jul 10 16:00:50.538790 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.538772 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-8jwrg"] Jul 10 16:00:50.566694 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:00:50.566666 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c359674_b066_4b29_ae66_82fc1052705d.slice/crio-93fd9fa46db8ae4796dcc33a14d77498be0d222d43c704bc05af1a4e4d045b1d WatchSource:0}: Error finding container 93fd9fa46db8ae4796dcc33a14d77498be0d222d43c704bc05af1a4e4d045b1d: Status 404 returned error can't find the container with id 93fd9fa46db8ae4796dcc33a14d77498be0d222d43c704bc05af1a4e4d045b1d Jul 10 16:00:50.877318 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.877285 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8jwrg" event={"ID":"9c359674-b066-4b29-ae66-82fc1052705d","Type":"ContainerStarted","Data":"93fd9fa46db8ae4796dcc33a14d77498be0d222d43c704bc05af1a4e4d045b1d"} Jul 10 16:00:50.878719 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.878687 2560 generic.go:356] "Generic (PLEG): container finished" podID="a0b86a74-04a0-4c7f-bc11-a15015426b9f" containerID="82b67f21e393dc3e4945b8611f031b9b52171e45f0b27acbe2bee50a914160c1" exitCode=0 Jul 10 16:00:50.878719 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.878718 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg" event={"ID":"a0b86a74-04a0-4c7f-bc11-a15015426b9f","Type":"ContainerDied","Data":"82b67f21e393dc3e4945b8611f031b9b52171e45f0b27acbe2bee50a914160c1"} Jul 10 16:00:50.878869 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:50.878747 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg" event={"ID":"a0b86a74-04a0-4c7f-bc11-a15015426b9f","Type":"ContainerStarted","Data":"244993d0266f78f5c9e25360ce998fdde32fadf6216c1d810533c4866885a52d"} Jul 10 16:00:53.894628 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:53.894557 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-8jwrg" event={"ID":"9c359674-b066-4b29-ae66-82fc1052705d","Type":"ContainerStarted","Data":"c234e5ea6d8b7a2ded62b1258e5f7623bfa9b0bc59ca9af40fa359fab55494bc"} Jul 10 16:00:53.895002 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:53.894685 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8jwrg" Jul 10 16:00:53.896345 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:53.896322 2560 generic.go:356] "Generic (PLEG): container finished" podID="a0b86a74-04a0-4c7f-bc11-a15015426b9f" containerID="ea10b192f83bee7288f9814494b2692dd8b5a9e8414b57ce4c87430b25e88856" exitCode=0 Jul 10 16:00:53.896423 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:53.896353 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg" event={"ID":"a0b86a74-04a0-4c7f-bc11-a15015426b9f","Type":"ContainerDied","Data":"ea10b192f83bee7288f9814494b2692dd8b5a9e8414b57ce4c87430b25e88856"} Jul 10 16:00:53.911524 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:53.911463 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-8jwrg" podStartSLOduration=0.966143854 podStartE2EDuration="3.911449866s" totalImagesPullingTime="2.945306012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:00:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:00:53.910155807 +0000 UTC m=+342.882189178" watchObservedRunningTime="2026-07-10 16:00:53.911449866 +0000 UTC m=+342.883483239" Jul 10 16:00:54.902181 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:54.902146 2560 generic.go:356] "Generic (PLEG): container finished" podID="a0b86a74-04a0-4c7f-bc11-a15015426b9f" containerID="349d53341a376bebd56b6b638ca54835a6a63178fb67c684147905b38fff3052" exitCode=0 Jul 10 16:00:54.902581 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:54.902234 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg" event={"ID":"a0b86a74-04a0-4c7f-bc11-a15015426b9f","Type":"ContainerDied","Data":"349d53341a376bebd56b6b638ca54835a6a63178fb67c684147905b38fff3052"} Jul 10 16:00:56.026986 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:56.026940 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg" Jul 10 16:00:56.172623 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:56.172544 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6kx2\" (UniqueName: \"kubernetes.io/projected/a0b86a74-04a0-4c7f-bc11-a15015426b9f-kube-api-access-j6kx2\") pod \"a0b86a74-04a0-4c7f-bc11-a15015426b9f\" (UID: \"a0b86a74-04a0-4c7f-bc11-a15015426b9f\") " Jul 10 16:00:56.172623 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:56.172582 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a0b86a74-04a0-4c7f-bc11-a15015426b9f-util\") pod \"a0b86a74-04a0-4c7f-bc11-a15015426b9f\" (UID: \"a0b86a74-04a0-4c7f-bc11-a15015426b9f\") " Jul 10 16:00:56.172845 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:56.172719 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a0b86a74-04a0-4c7f-bc11-a15015426b9f-bundle\") pod \"a0b86a74-04a0-4c7f-bc11-a15015426b9f\" (UID: \"a0b86a74-04a0-4c7f-bc11-a15015426b9f\") " Jul 10 16:00:56.173047 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:56.173026 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0b86a74-04a0-4c7f-bc11-a15015426b9f-bundle" (OuterVolumeSpecName: "bundle") pod "a0b86a74-04a0-4c7f-bc11-a15015426b9f" (UID: "a0b86a74-04a0-4c7f-bc11-a15015426b9f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:00:56.174539 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:56.174520 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0b86a74-04a0-4c7f-bc11-a15015426b9f-kube-api-access-j6kx2" (OuterVolumeSpecName: "kube-api-access-j6kx2") pod "a0b86a74-04a0-4c7f-bc11-a15015426b9f" (UID: "a0b86a74-04a0-4c7f-bc11-a15015426b9f"). InnerVolumeSpecName "kube-api-access-j6kx2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:00:56.179253 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:56.179205 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0b86a74-04a0-4c7f-bc11-a15015426b9f-util" (OuterVolumeSpecName: "util") pod "a0b86a74-04a0-4c7f-bc11-a15015426b9f" (UID: "a0b86a74-04a0-4c7f-bc11-a15015426b9f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:00:56.273504 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:56.273469 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a0b86a74-04a0-4c7f-bc11-a15015426b9f-bundle\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:00:56.273504 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:56.273498 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6kx2\" (UniqueName: \"kubernetes.io/projected/a0b86a74-04a0-4c7f-bc11-a15015426b9f-kube-api-access-j6kx2\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:00:56.273504 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:56.273510 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a0b86a74-04a0-4c7f-bc11-a15015426b9f-util\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:00:56.913243 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:56.913208 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg" event={"ID":"a0b86a74-04a0-4c7f-bc11-a15015426b9f","Type":"ContainerDied","Data":"244993d0266f78f5c9e25360ce998fdde32fadf6216c1d810533c4866885a52d"} Jul 10 16:00:56.913243 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:56.913226 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft6rrg" Jul 10 16:00:56.913438 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:56.913256 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="244993d0266f78f5c9e25360ce998fdde32fadf6216c1d810533c4866885a52d" Jul 10 16:00:58.416934 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:58.416899 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ntswq"] Jul 10 16:00:58.417446 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:58.417154 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0b86a74-04a0-4c7f-bc11-a15015426b9f" containerName="extract" Jul 10 16:00:58.417446 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:58.417165 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0b86a74-04a0-4c7f-bc11-a15015426b9f" containerName="extract" Jul 10 16:00:58.417446 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:58.417176 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0b86a74-04a0-4c7f-bc11-a15015426b9f" containerName="util" Jul 10 16:00:58.417446 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:58.417181 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0b86a74-04a0-4c7f-bc11-a15015426b9f" containerName="util" Jul 10 16:00:58.417446 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:58.417188 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0b86a74-04a0-4c7f-bc11-a15015426b9f" containerName="pull" Jul 10 16:00:58.417446 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:58.417193 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0b86a74-04a0-4c7f-bc11-a15015426b9f" containerName="pull" Jul 10 16:00:58.417446 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:58.417235 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0b86a74-04a0-4c7f-bc11-a15015426b9f" containerName="extract" Jul 10 16:00:58.418876 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:58.418854 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ntswq" Jul 10 16:00:58.421425 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:58.421408 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-sgq2r\"" Jul 10 16:00:58.428668 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:58.428648 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ntswq"] Jul 10 16:00:58.592672 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:58.592638 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a798ee8c-e2d8-4f21-8ba2-c1ed985db738-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ntswq\" (UID: \"a798ee8c-e2d8-4f21-8ba2-c1ed985db738\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ntswq" Jul 10 16:00:58.592824 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:58.592702 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbcww\" (UniqueName: \"kubernetes.io/projected/a798ee8c-e2d8-4f21-8ba2-c1ed985db738-kube-api-access-hbcww\") pod \"cert-manager-cainjector-559cdd8c87-ntswq\" (UID: \"a798ee8c-e2d8-4f21-8ba2-c1ed985db738\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ntswq" Jul 10 16:00:58.693671 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:58.693593 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbcww\" (UniqueName: \"kubernetes.io/projected/a798ee8c-e2d8-4f21-8ba2-c1ed985db738-kube-api-access-hbcww\") pod \"cert-manager-cainjector-559cdd8c87-ntswq\" (UID: \"a798ee8c-e2d8-4f21-8ba2-c1ed985db738\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ntswq" Jul 10 16:00:58.693671 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:58.693633 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a798ee8c-e2d8-4f21-8ba2-c1ed985db738-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ntswq\" (UID: \"a798ee8c-e2d8-4f21-8ba2-c1ed985db738\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ntswq" Jul 10 16:00:58.702476 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:58.702439 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a798ee8c-e2d8-4f21-8ba2-c1ed985db738-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-ntswq\" (UID: \"a798ee8c-e2d8-4f21-8ba2-c1ed985db738\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ntswq" Jul 10 16:00:58.702592 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:58.702543 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbcww\" (UniqueName: \"kubernetes.io/projected/a798ee8c-e2d8-4f21-8ba2-c1ed985db738-kube-api-access-hbcww\") pod \"cert-manager-cainjector-559cdd8c87-ntswq\" (UID: \"a798ee8c-e2d8-4f21-8ba2-c1ed985db738\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-ntswq" Jul 10 16:00:58.727545 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:58.727518 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ntswq" Jul 10 16:00:58.847807 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:58.847777 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-ntswq"] Jul 10 16:00:58.850134 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:00:58.850107 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda798ee8c_e2d8_4f21_8ba2_c1ed985db738.slice/crio-65a3e2a01e849ef99645ca290186a90af8c396dd0bcebdebd27102f698878124 WatchSource:0}: Error finding container 65a3e2a01e849ef99645ca290186a90af8c396dd0bcebdebd27102f698878124: Status 404 returned error can't find the container with id 65a3e2a01e849ef99645ca290186a90af8c396dd0bcebdebd27102f698878124 Jul 10 16:00:58.927751 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:58.927717 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ntswq" event={"ID":"a798ee8c-e2d8-4f21-8ba2-c1ed985db738","Type":"ContainerStarted","Data":"2a4875438c03efe03516a68fcfee3f1de0116fbf45561164c0f611fa0b501740"} Jul 10 16:00:58.927898 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:58.927764 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ntswq" event={"ID":"a798ee8c-e2d8-4f21-8ba2-c1ed985db738","Type":"ContainerStarted","Data":"65a3e2a01e849ef99645ca290186a90af8c396dd0bcebdebd27102f698878124"} Jul 10 16:00:58.945554 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:58.945467 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-ntswq" podStartSLOduration=0.945453472 podStartE2EDuration="945.453472ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:00:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:00:58.945166285 +0000 UTC m=+347.917199663" watchObservedRunningTime="2026-07-10 16:00:58.945453472 +0000 UTC m=+347.917486847" Jul 10 16:00:59.904413 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:00:59.904383 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-8jwrg" Jul 10 16:01:10.471469 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:10.471386 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx"] Jul 10 16:01:10.475891 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:10.475870 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx" Jul 10 16:01:10.479613 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:10.479593 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-sltc4\"" Jul 10 16:01:10.479918 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:10.479900 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 10 16:01:10.480719 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:10.480704 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 10 16:01:10.485221 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:10.485199 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb16d839-e37b-4983-ae00-7ee2002e42e4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx\" (UID: \"eb16d839-e37b-4983-ae00-7ee2002e42e4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx" Jul 10 16:01:10.485318 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:10.485236 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxdvl\" (UniqueName: \"kubernetes.io/projected/eb16d839-e37b-4983-ae00-7ee2002e42e4-kube-api-access-zxdvl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx\" (UID: \"eb16d839-e37b-4983-ae00-7ee2002e42e4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx" Jul 10 16:01:10.485368 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:10.485351 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb16d839-e37b-4983-ae00-7ee2002e42e4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx\" (UID: \"eb16d839-e37b-4983-ae00-7ee2002e42e4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx" Jul 10 16:01:10.488160 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:10.488141 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx"] Jul 10 16:01:10.585672 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:10.585631 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb16d839-e37b-4983-ae00-7ee2002e42e4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx\" (UID: \"eb16d839-e37b-4983-ae00-7ee2002e42e4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx" Jul 10 16:01:10.585855 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:10.585698 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb16d839-e37b-4983-ae00-7ee2002e42e4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx\" (UID: \"eb16d839-e37b-4983-ae00-7ee2002e42e4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx" Jul 10 16:01:10.585855 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:10.585719 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxdvl\" (UniqueName: \"kubernetes.io/projected/eb16d839-e37b-4983-ae00-7ee2002e42e4-kube-api-access-zxdvl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx\" (UID: \"eb16d839-e37b-4983-ae00-7ee2002e42e4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx" Jul 10 16:01:10.586069 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:10.586048 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb16d839-e37b-4983-ae00-7ee2002e42e4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx\" (UID: \"eb16d839-e37b-4983-ae00-7ee2002e42e4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx" Jul 10 16:01:10.586145 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:10.586081 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb16d839-e37b-4983-ae00-7ee2002e42e4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx\" (UID: \"eb16d839-e37b-4983-ae00-7ee2002e42e4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx" Jul 10 16:01:10.596300 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:10.596276 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxdvl\" (UniqueName: \"kubernetes.io/projected/eb16d839-e37b-4983-ae00-7ee2002e42e4-kube-api-access-zxdvl\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx\" (UID: \"eb16d839-e37b-4983-ae00-7ee2002e42e4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx" Jul 10 16:01:10.784945 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:10.784862 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx" Jul 10 16:01:10.908043 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:10.908023 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx"] Jul 10 16:01:10.910230 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:01:10.910190 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb16d839_e37b_4983_ae00_7ee2002e42e4.slice/crio-d385c87c9c39f1e228c21fc2dd338fe4d8a8c1c94faf9bdcdec962200d789660 WatchSource:0}: Error finding container d385c87c9c39f1e228c21fc2dd338fe4d8a8c1c94faf9bdcdec962200d789660: Status 404 returned error can't find the container with id d385c87c9c39f1e228c21fc2dd338fe4d8a8c1c94faf9bdcdec962200d789660 Jul 10 16:01:10.975625 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:10.975590 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx" event={"ID":"eb16d839-e37b-4983-ae00-7ee2002e42e4","Type":"ContainerStarted","Data":"1a38f94588721d0aeb9b58b2013c87e8d1a82904d02c264b7f215cb84c310f97"} Jul 10 16:01:10.975761 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:10.975644 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx" event={"ID":"eb16d839-e37b-4983-ae00-7ee2002e42e4","Type":"ContainerStarted","Data":"d385c87c9c39f1e228c21fc2dd338fe4d8a8c1c94faf9bdcdec962200d789660"} Jul 10 16:01:11.980697 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:11.980662 2560 generic.go:356] "Generic (PLEG): container finished" podID="eb16d839-e37b-4983-ae00-7ee2002e42e4" containerID="1a38f94588721d0aeb9b58b2013c87e8d1a82904d02c264b7f215cb84c310f97" exitCode=0 Jul 10 16:01:11.981105 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:11.980751 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx" event={"ID":"eb16d839-e37b-4983-ae00-7ee2002e42e4","Type":"ContainerDied","Data":"1a38f94588721d0aeb9b58b2013c87e8d1a82904d02c264b7f215cb84c310f97"} Jul 10 16:01:12.986794 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:12.986758 2560 generic.go:356] "Generic (PLEG): container finished" podID="eb16d839-e37b-4983-ae00-7ee2002e42e4" containerID="36a9adbc66a340ac8f49d00678d11593af8b81891957c8747a6e4d2b8af05f61" exitCode=0 Jul 10 16:01:12.987173 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:12.986824 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx" event={"ID":"eb16d839-e37b-4983-ae00-7ee2002e42e4","Type":"ContainerDied","Data":"36a9adbc66a340ac8f49d00678d11593af8b81891957c8747a6e4d2b8af05f61"} Jul 10 16:01:13.992811 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:13.992767 2560 generic.go:356] "Generic (PLEG): container finished" podID="eb16d839-e37b-4983-ae00-7ee2002e42e4" containerID="f4ccf859b6e1db8290202f0fbfd5dd698e54aa0d3b6f5bf5cc88e520abcba83f" exitCode=0 Jul 10 16:01:13.993385 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:13.993347 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx" event={"ID":"eb16d839-e37b-4983-ae00-7ee2002e42e4","Type":"ContainerDied","Data":"f4ccf859b6e1db8290202f0fbfd5dd698e54aa0d3b6f5bf5cc88e520abcba83f"} Jul 10 16:01:15.117023 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:15.117002 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx" Jul 10 16:01:15.223061 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:15.223033 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb16d839-e37b-4983-ae00-7ee2002e42e4-util\") pod \"eb16d839-e37b-4983-ae00-7ee2002e42e4\" (UID: \"eb16d839-e37b-4983-ae00-7ee2002e42e4\") " Jul 10 16:01:15.223061 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:15.223066 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb16d839-e37b-4983-ae00-7ee2002e42e4-bundle\") pod \"eb16d839-e37b-4983-ae00-7ee2002e42e4\" (UID: \"eb16d839-e37b-4983-ae00-7ee2002e42e4\") " Jul 10 16:01:15.223289 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:15.223109 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxdvl\" (UniqueName: \"kubernetes.io/projected/eb16d839-e37b-4983-ae00-7ee2002e42e4-kube-api-access-zxdvl\") pod \"eb16d839-e37b-4983-ae00-7ee2002e42e4\" (UID: \"eb16d839-e37b-4983-ae00-7ee2002e42e4\") " Jul 10 16:01:15.223763 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:15.223732 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb16d839-e37b-4983-ae00-7ee2002e42e4-bundle" (OuterVolumeSpecName: "bundle") pod "eb16d839-e37b-4983-ae00-7ee2002e42e4" (UID: "eb16d839-e37b-4983-ae00-7ee2002e42e4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:01:15.225185 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:15.225161 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb16d839-e37b-4983-ae00-7ee2002e42e4-kube-api-access-zxdvl" (OuterVolumeSpecName: "kube-api-access-zxdvl") pod "eb16d839-e37b-4983-ae00-7ee2002e42e4" (UID: "eb16d839-e37b-4983-ae00-7ee2002e42e4"). InnerVolumeSpecName "kube-api-access-zxdvl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:01:15.229020 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:15.228985 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb16d839-e37b-4983-ae00-7ee2002e42e4-util" (OuterVolumeSpecName: "util") pod "eb16d839-e37b-4983-ae00-7ee2002e42e4" (UID: "eb16d839-e37b-4983-ae00-7ee2002e42e4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:01:15.323720 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:15.323652 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/eb16d839-e37b-4983-ae00-7ee2002e42e4-util\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:01:15.323720 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:15.323677 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/eb16d839-e37b-4983-ae00-7ee2002e42e4-bundle\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:01:15.323720 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:15.323687 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxdvl\" (UniqueName: \"kubernetes.io/projected/eb16d839-e37b-4983-ae00-7ee2002e42e4-kube-api-access-zxdvl\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:01:16.002297 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:16.002274 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx" Jul 10 16:01:16.002297 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:16.002279 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796aft96hx" event={"ID":"eb16d839-e37b-4983-ae00-7ee2002e42e4","Type":"ContainerDied","Data":"d385c87c9c39f1e228c21fc2dd338fe4d8a8c1c94faf9bdcdec962200d789660"} Jul 10 16:01:16.002510 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:16.002317 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d385c87c9c39f1e228c21fc2dd338fe4d8a8c1c94faf9bdcdec962200d789660" Jul 10 16:01:27.631628 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.631596 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5"] Jul 10 16:01:27.632100 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.631935 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb16d839-e37b-4983-ae00-7ee2002e42e4" containerName="extract" Jul 10 16:01:27.632100 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.631970 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb16d839-e37b-4983-ae00-7ee2002e42e4" containerName="extract" Jul 10 16:01:27.632100 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.631987 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb16d839-e37b-4983-ae00-7ee2002e42e4" containerName="util" Jul 10 16:01:27.632100 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.631995 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb16d839-e37b-4983-ae00-7ee2002e42e4" containerName="util" Jul 10 16:01:27.632100 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.632005 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb16d839-e37b-4983-ae00-7ee2002e42e4" containerName="pull" Jul 10 16:01:27.632100 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.632013 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb16d839-e37b-4983-ae00-7ee2002e42e4" containerName="pull" Jul 10 16:01:27.632100 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.632080 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb16d839-e37b-4983-ae00-7ee2002e42e4" containerName="extract" Jul 10 16:01:27.634718 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.634695 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" Jul 10 16:01:27.639862 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.639842 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 10 16:01:27.640224 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.640204 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 10 16:01:27.640551 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.640531 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 10 16:01:27.640633 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.640610 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-sl2l5\"" Jul 10 16:01:27.640948 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.640650 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 10 16:01:27.654338 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.654318 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5"] Jul 10 16:01:27.710330 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.710304 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/02f79ace-8bc2-4499-9551-5218dad2e06a-webhook-cert\") pod \"opendatahub-operator-controller-manager-765dfd8876-858s5\" (UID: \"02f79ace-8bc2-4499-9551-5218dad2e06a\") " pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" Jul 10 16:01:27.710453 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.710332 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/02f79ace-8bc2-4499-9551-5218dad2e06a-manifests\") pod \"opendatahub-operator-controller-manager-765dfd8876-858s5\" (UID: \"02f79ace-8bc2-4499-9551-5218dad2e06a\") " pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" Jul 10 16:01:27.710453 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.710365 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02f79ace-8bc2-4499-9551-5218dad2e06a-tmp\") pod \"opendatahub-operator-controller-manager-765dfd8876-858s5\" (UID: \"02f79ace-8bc2-4499-9551-5218dad2e06a\") " pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" Jul 10 16:01:27.710453 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.710398 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwr96\" (UniqueName: \"kubernetes.io/projected/02f79ace-8bc2-4499-9551-5218dad2e06a-kube-api-access-qwr96\") pod \"opendatahub-operator-controller-manager-765dfd8876-858s5\" (UID: \"02f79ace-8bc2-4499-9551-5218dad2e06a\") " pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" Jul 10 16:01:27.710453 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.710437 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/02f79ace-8bc2-4499-9551-5218dad2e06a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-765dfd8876-858s5\" (UID: \"02f79ace-8bc2-4499-9551-5218dad2e06a\") " pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" Jul 10 16:01:27.811642 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.811611 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/02f79ace-8bc2-4499-9551-5218dad2e06a-manifests\") pod \"opendatahub-operator-controller-manager-765dfd8876-858s5\" (UID: \"02f79ace-8bc2-4499-9551-5218dad2e06a\") " pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" Jul 10 16:01:27.811795 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.811657 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02f79ace-8bc2-4499-9551-5218dad2e06a-tmp\") pod \"opendatahub-operator-controller-manager-765dfd8876-858s5\" (UID: \"02f79ace-8bc2-4499-9551-5218dad2e06a\") " pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" Jul 10 16:01:27.811795 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.811687 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwr96\" (UniqueName: \"kubernetes.io/projected/02f79ace-8bc2-4499-9551-5218dad2e06a-kube-api-access-qwr96\") pod \"opendatahub-operator-controller-manager-765dfd8876-858s5\" (UID: \"02f79ace-8bc2-4499-9551-5218dad2e06a\") " pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" Jul 10 16:01:27.811795 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.811730 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/02f79ace-8bc2-4499-9551-5218dad2e06a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-765dfd8876-858s5\" (UID: \"02f79ace-8bc2-4499-9551-5218dad2e06a\") " pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" Jul 10 16:01:27.811795 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.811782 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/02f79ace-8bc2-4499-9551-5218dad2e06a-webhook-cert\") pod \"opendatahub-operator-controller-manager-765dfd8876-858s5\" (UID: \"02f79ace-8bc2-4499-9551-5218dad2e06a\") " pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" Jul 10 16:01:27.812023 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.811985 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/02f79ace-8bc2-4499-9551-5218dad2e06a-manifests\") pod \"opendatahub-operator-controller-manager-765dfd8876-858s5\" (UID: \"02f79ace-8bc2-4499-9551-5218dad2e06a\") " pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" Jul 10 16:01:27.814012 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.813988 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02f79ace-8bc2-4499-9551-5218dad2e06a-tmp\") pod \"opendatahub-operator-controller-manager-765dfd8876-858s5\" (UID: \"02f79ace-8bc2-4499-9551-5218dad2e06a\") " pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" Jul 10 16:01:27.814117 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.814072 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/02f79ace-8bc2-4499-9551-5218dad2e06a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-765dfd8876-858s5\" (UID: \"02f79ace-8bc2-4499-9551-5218dad2e06a\") " pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" Jul 10 16:01:27.814168 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.814117 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/02f79ace-8bc2-4499-9551-5218dad2e06a-webhook-cert\") pod \"opendatahub-operator-controller-manager-765dfd8876-858s5\" (UID: \"02f79ace-8bc2-4499-9551-5218dad2e06a\") " pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" Jul 10 16:01:27.826517 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.826493 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwr96\" (UniqueName: \"kubernetes.io/projected/02f79ace-8bc2-4499-9551-5218dad2e06a-kube-api-access-qwr96\") pod \"opendatahub-operator-controller-manager-765dfd8876-858s5\" (UID: \"02f79ace-8bc2-4499-9551-5218dad2e06a\") " pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" Jul 10 16:01:27.943872 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:27.943800 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" Jul 10 16:01:28.083521 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:28.083448 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5"] Jul 10 16:01:28.087996 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:01:28.087969 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02f79ace_8bc2_4499_9551_5218dad2e06a.slice/crio-d7ec60f43d179c3563b1ca83b06f8a2e99c9fea5e698d99b3809544a109b93ca WatchSource:0}: Error finding container d7ec60f43d179c3563b1ca83b06f8a2e99c9fea5e698d99b3809544a109b93ca: Status 404 returned error can't find the container with id d7ec60f43d179c3563b1ca83b06f8a2e99c9fea5e698d99b3809544a109b93ca Jul 10 16:01:29.055440 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:29.055404 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" event={"ID":"02f79ace-8bc2-4499-9551-5218dad2e06a","Type":"ContainerStarted","Data":"d7ec60f43d179c3563b1ca83b06f8a2e99c9fea5e698d99b3809544a109b93ca"} Jul 10 16:01:32.068722 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:32.068688 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" event={"ID":"02f79ace-8bc2-4499-9551-5218dad2e06a","Type":"ContainerStarted","Data":"932a4db1adc844c08371c6e6d834d07e32b309930f5c026a8f784e0528a5c93c"} Jul 10 16:01:33.073987 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:33.073937 2560 generic.go:356] "Generic (PLEG): container finished" podID="02f79ace-8bc2-4499-9551-5218dad2e06a" containerID="932a4db1adc844c08371c6e6d834d07e32b309930f5c026a8f784e0528a5c93c" exitCode=0 Jul 10 16:01:33.073987 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:33.073982 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" event={"ID":"02f79ace-8bc2-4499-9551-5218dad2e06a","Type":"ContainerDied","Data":"932a4db1adc844c08371c6e6d834d07e32b309930f5c026a8f784e0528a5c93c"} Jul 10 16:01:34.082308 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:34.082262 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" event={"ID":"02f79ace-8bc2-4499-9551-5218dad2e06a","Type":"ContainerStarted","Data":"540281a8cbde4c5f2e53500912f3a4e19d9e3bd499dc09c1787fc818c4a1427b"} Jul 10 16:01:34.082724 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:34.082419 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" Jul 10 16:01:34.119845 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:34.119797 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" podStartSLOduration=2.825932351 podStartE2EDuration="7.119781407s" totalImagesPullingTime="3.274148773s" totalInitContainerRuntime="1.019700283s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:01:27 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:01:34.119113032 +0000 UTC m=+383.091146423" watchObservedRunningTime="2026-07-10 16:01:34.119781407 +0000 UTC m=+383.091814781" Jul 10 16:01:45.088491 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:45.088462 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-765dfd8876-858s5" Jul 10 16:01:47.429195 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:47.429161 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6b74489749-cvbvn"] Jul 10 16:01:47.526852 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:47.526819 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6b74489749-cvbvn"] Jul 10 16:01:47.527038 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:47.526937 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6b74489749-cvbvn" Jul 10 16:01:47.530640 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:47.530622 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 10 16:01:47.530793 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:47.530777 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 10 16:01:47.531303 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:47.531282 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 10 16:01:47.531403 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:47.531316 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 10 16:01:47.565895 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:47.565872 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8-tmp\") pod \"kube-auth-proxy-6b74489749-cvbvn\" (UID: \"4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8\") " pod="openshift-ingress/kube-auth-proxy-6b74489749-cvbvn" Jul 10 16:01:47.566017 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:47.565910 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbldx\" (UniqueName: \"kubernetes.io/projected/4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8-kube-api-access-dbldx\") pod \"kube-auth-proxy-6b74489749-cvbvn\" (UID: \"4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8\") " pod="openshift-ingress/kube-auth-proxy-6b74489749-cvbvn" Jul 10 16:01:47.566017 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:47.565975 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8-tls-certs\") pod \"kube-auth-proxy-6b74489749-cvbvn\" (UID: \"4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8\") " pod="openshift-ingress/kube-auth-proxy-6b74489749-cvbvn" Jul 10 16:01:47.666679 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:47.666648 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8-tls-certs\") pod \"kube-auth-proxy-6b74489749-cvbvn\" (UID: \"4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8\") " pod="openshift-ingress/kube-auth-proxy-6b74489749-cvbvn" Jul 10 16:01:47.666861 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:47.666687 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8-tmp\") pod \"kube-auth-proxy-6b74489749-cvbvn\" (UID: \"4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8\") " pod="openshift-ingress/kube-auth-proxy-6b74489749-cvbvn" Jul 10 16:01:47.666861 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:47.666716 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbldx\" (UniqueName: \"kubernetes.io/projected/4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8-kube-api-access-dbldx\") pod \"kube-auth-proxy-6b74489749-cvbvn\" (UID: \"4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8\") " pod="openshift-ingress/kube-auth-proxy-6b74489749-cvbvn" Jul 10 16:01:47.668934 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:47.668914 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8-tmp\") pod \"kube-auth-proxy-6b74489749-cvbvn\" (UID: \"4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8\") " pod="openshift-ingress/kube-auth-proxy-6b74489749-cvbvn" Jul 10 16:01:47.669088 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:47.669069 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8-tls-certs\") pod \"kube-auth-proxy-6b74489749-cvbvn\" (UID: \"4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8\") " pod="openshift-ingress/kube-auth-proxy-6b74489749-cvbvn" Jul 10 16:01:47.723024 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:47.723000 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbldx\" (UniqueName: \"kubernetes.io/projected/4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8-kube-api-access-dbldx\") pod \"kube-auth-proxy-6b74489749-cvbvn\" (UID: \"4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8\") " pod="openshift-ingress/kube-auth-proxy-6b74489749-cvbvn" Jul 10 16:01:47.836097 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:47.836067 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6b74489749-cvbvn" Jul 10 16:01:47.989735 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:47.989653 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-f9w9q"] Jul 10 16:01:47.992222 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:47.992202 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-f9w9q" Jul 10 16:01:47.995832 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:47.995814 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 10 16:01:47.998182 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:47.998165 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-j6qwv\"" Jul 10 16:01:48.015167 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:48.015151 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6b74489749-cvbvn"] Jul 10 16:01:48.017654 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:01:48.017613 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4381cadb_1ee8_4ef5_8e83_4b7a5bebb1f8.slice/crio-7ed9664937178317cd9eafaf8d1ef2d540c89061fff8744657315f5d3abde1cf WatchSource:0}: Error finding container 7ed9664937178317cd9eafaf8d1ef2d540c89061fff8744657315f5d3abde1cf: Status 404 returned error can't find the container with id 7ed9664937178317cd9eafaf8d1ef2d540c89061fff8744657315f5d3abde1cf Jul 10 16:01:48.018991 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:48.018972 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-f9w9q"] Jul 10 16:01:48.069570 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:48.069536 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86b97add-9066-4d71-bdac-4f9f923d55f5-cert\") pod \"odh-model-controller-7998c67dc4-f9w9q\" (UID: \"86b97add-9066-4d71-bdac-4f9f923d55f5\") " pod="opendatahub/odh-model-controller-7998c67dc4-f9w9q" Jul 10 16:01:48.069735 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:48.069593 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf7qm\" (UniqueName: \"kubernetes.io/projected/86b97add-9066-4d71-bdac-4f9f923d55f5-kube-api-access-kf7qm\") pod \"odh-model-controller-7998c67dc4-f9w9q\" (UID: \"86b97add-9066-4d71-bdac-4f9f923d55f5\") " pod="opendatahub/odh-model-controller-7998c67dc4-f9w9q" Jul 10 16:01:48.137523 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:48.137494 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6b74489749-cvbvn" event={"ID":"4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8","Type":"ContainerStarted","Data":"7ed9664937178317cd9eafaf8d1ef2d540c89061fff8744657315f5d3abde1cf"} Jul 10 16:01:48.170541 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:48.170503 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86b97add-9066-4d71-bdac-4f9f923d55f5-cert\") pod \"odh-model-controller-7998c67dc4-f9w9q\" (UID: \"86b97add-9066-4d71-bdac-4f9f923d55f5\") " pod="opendatahub/odh-model-controller-7998c67dc4-f9w9q" Jul 10 16:01:48.170682 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:48.170584 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf7qm\" (UniqueName: \"kubernetes.io/projected/86b97add-9066-4d71-bdac-4f9f923d55f5-kube-api-access-kf7qm\") pod \"odh-model-controller-7998c67dc4-f9w9q\" (UID: \"86b97add-9066-4d71-bdac-4f9f923d55f5\") " pod="opendatahub/odh-model-controller-7998c67dc4-f9w9q" Jul 10 16:01:48.170682 ip-10-0-129-101 kubenswrapper[2560]: E0710 16:01:48.170638 2560 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 10 16:01:48.170751 ip-10-0-129-101 kubenswrapper[2560]: E0710 16:01:48.170695 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86b97add-9066-4d71-bdac-4f9f923d55f5-cert podName:86b97add-9066-4d71-bdac-4f9f923d55f5 nodeName:}" failed. No retries permitted until 2026-07-10 16:01:48.670678598 +0000 UTC m=+397.642711951 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/86b97add-9066-4d71-bdac-4f9f923d55f5-cert") pod "odh-model-controller-7998c67dc4-f9w9q" (UID: "86b97add-9066-4d71-bdac-4f9f923d55f5") : secret "odh-model-controller-webhook-cert" not found Jul 10 16:01:48.182049 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:48.182020 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf7qm\" (UniqueName: \"kubernetes.io/projected/86b97add-9066-4d71-bdac-4f9f923d55f5-kube-api-access-kf7qm\") pod \"odh-model-controller-7998c67dc4-f9w9q\" (UID: \"86b97add-9066-4d71-bdac-4f9f923d55f5\") " pod="opendatahub/odh-model-controller-7998c67dc4-f9w9q" Jul 10 16:01:48.673852 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:48.673813 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86b97add-9066-4d71-bdac-4f9f923d55f5-cert\") pod \"odh-model-controller-7998c67dc4-f9w9q\" (UID: \"86b97add-9066-4d71-bdac-4f9f923d55f5\") " pod="opendatahub/odh-model-controller-7998c67dc4-f9w9q" Jul 10 16:01:48.676765 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:48.676731 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/86b97add-9066-4d71-bdac-4f9f923d55f5-cert\") pod \"odh-model-controller-7998c67dc4-f9w9q\" (UID: \"86b97add-9066-4d71-bdac-4f9f923d55f5\") " pod="opendatahub/odh-model-controller-7998c67dc4-f9w9q" Jul 10 16:01:48.902230 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:48.902186 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-f9w9q" Jul 10 16:01:49.033734 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.033711 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-f9w9q"] Jul 10 16:01:49.036090 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:01:49.036061 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86b97add_9066_4d71_bdac_4f9f923d55f5.slice/crio-65a78c2aacfcf4d6f7f9b220c74796295bbb142b1345acfe28fa2f4b835edcf1 WatchSource:0}: Error finding container 65a78c2aacfcf4d6f7f9b220c74796295bbb142b1345acfe28fa2f4b835edcf1: Status 404 returned error can't find the container with id 65a78c2aacfcf4d6f7f9b220c74796295bbb142b1345acfe28fa2f4b835edcf1 Jul 10 16:01:49.143919 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.143883 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-f9w9q" event={"ID":"86b97add-9066-4d71-bdac-4f9f923d55f5","Type":"ContainerStarted","Data":"65a78c2aacfcf4d6f7f9b220c74796295bbb142b1345acfe28fa2f4b835edcf1"} Jul 10 16:01:49.226476 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.226442 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4"] Jul 10 16:01:49.229475 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.229452 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:49.233066 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.232688 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 10 16:01:49.233066 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.232766 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-vqnfb\"" Jul 10 16:01:49.233230 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.233119 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 10 16:01:49.233403 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.233384 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 10 16:01:49.233476 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.233434 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 10 16:01:49.247736 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.247710 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4"] Jul 10 16:01:49.279315 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.279286 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-nwpc4\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:49.279315 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.279324 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9c51051d-a8b7-401b-8115-ce2dc90d9762-cacerts\") pod \"istiod-openshift-gateway-7574459656-nwpc4\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:49.279534 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.279369 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-nwpc4\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:49.279534 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.279402 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-token\") pod \"istiod-openshift-gateway-7574459656-nwpc4\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:49.279616 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.279525 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-nwpc4\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:49.279616 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.279573 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c51051d-a8b7-401b-8115-ce2dc90d9762-local-certs\") pod \"istiod-openshift-gateway-7574459656-nwpc4\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:49.279703 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.279612 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lv4g\" (UniqueName: \"kubernetes.io/projected/9c51051d-a8b7-401b-8115-ce2dc90d9762-kube-api-access-2lv4g\") pod \"istiod-openshift-gateway-7574459656-nwpc4\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:49.380513 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.380482 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-nwpc4\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:49.380673 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.380519 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9c51051d-a8b7-401b-8115-ce2dc90d9762-cacerts\") pod \"istiod-openshift-gateway-7574459656-nwpc4\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:49.380673 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.380545 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-nwpc4\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:49.380673 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.380569 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-token\") pod \"istiod-openshift-gateway-7574459656-nwpc4\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:49.380833 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.380705 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-nwpc4\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:49.380833 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.380768 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c51051d-a8b7-401b-8115-ce2dc90d9762-local-certs\") pod \"istiod-openshift-gateway-7574459656-nwpc4\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:49.380833 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.380812 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lv4g\" (UniqueName: \"kubernetes.io/projected/9c51051d-a8b7-401b-8115-ce2dc90d9762-kube-api-access-2lv4g\") pod \"istiod-openshift-gateway-7574459656-nwpc4\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:49.381257 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.381229 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-nwpc4\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:49.382986 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.382937 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9c51051d-a8b7-401b-8115-ce2dc90d9762-cacerts\") pod \"istiod-openshift-gateway-7574459656-nwpc4\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:49.383100 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.383051 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-nwpc4\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:49.383228 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.383212 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-nwpc4\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:49.383284 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.383229 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c51051d-a8b7-401b-8115-ce2dc90d9762-local-certs\") pod \"istiod-openshift-gateway-7574459656-nwpc4\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:49.401424 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.401405 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-token\") pod \"istiod-openshift-gateway-7574459656-nwpc4\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:49.411776 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.411752 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lv4g\" (UniqueName: \"kubernetes.io/projected/9c51051d-a8b7-401b-8115-ce2dc90d9762-kube-api-access-2lv4g\") pod \"istiod-openshift-gateway-7574459656-nwpc4\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:49.545322 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.545227 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:49.698678 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:49.698609 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4"] Jul 10 16:01:49.701078 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:01:49.701044 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c51051d_a8b7_401b_8115_ce2dc90d9762.slice/crio-6d344755432dcfb8a991314f9b15e108bfb89629770d2829198f711818fa8ed7 WatchSource:0}: Error finding container 6d344755432dcfb8a991314f9b15e108bfb89629770d2829198f711818fa8ed7: Status 404 returned error can't find the container with id 6d344755432dcfb8a991314f9b15e108bfb89629770d2829198f711818fa8ed7 Jul 10 16:01:50.150971 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:50.150921 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" event={"ID":"9c51051d-a8b7-401b-8115-ce2dc90d9762","Type":"ContainerStarted","Data":"6d344755432dcfb8a991314f9b15e108bfb89629770d2829198f711818fa8ed7"} Jul 10 16:01:53.433248 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:53.433204 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:01:53.433599 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:53.433295 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:01:54.171002 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.170939 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-f9w9q" event={"ID":"86b97add-9066-4d71-bdac-4f9f923d55f5","Type":"ContainerStarted","Data":"62fbde44b83530f9cfc4e98c28f6cdd7473b048c50021d3ad28cb1d4e23260d7"} Jul 10 16:01:54.171203 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.171058 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-f9w9q" Jul 10 16:01:54.172484 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.172459 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6b74489749-cvbvn" event={"ID":"4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8","Type":"ContainerStarted","Data":"fcc83caa572cc4d2ac0138f040abd2b92682089d5ebf7753aa724220f0a074fb"} Jul 10 16:01:54.173932 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.173912 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" event={"ID":"9c51051d-a8b7-401b-8115-ce2dc90d9762","Type":"ContainerStarted","Data":"06bec6ba7841815c192e8d8b7a40ca169fad9c4b23bd470b1fa1a121ce5fbef0"} Jul 10 16:01:54.174109 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.174096 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:54.175768 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.175745 2560 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-nwpc4 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 10 16:01:54.175865 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.175793 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" podUID="9c51051d-a8b7-401b-8115-ce2dc90d9762" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 10 16:01:54.194145 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.194102 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-f9w9q" podStartSLOduration=2.849000678 podStartE2EDuration="7.194088884s" totalImagesPullingTime="4.345088206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:01:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:01:54.192696748 +0000 UTC m=+403.164730143" watchObservedRunningTime="2026-07-10 16:01:54.194088884 +0000 UTC m=+403.166122259" Jul 10 16:01:54.213706 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.213642 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6b74489749-cvbvn" podStartSLOduration=1.860468133 podStartE2EDuration="7.213623142s" totalImagesPullingTime="5.353155009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:01:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:01:54.212845388 +0000 UTC m=+403.184878763" watchObservedRunningTime="2026-07-10 16:01:54.213623142 +0000 UTC m=+403.185656519" Jul 10 16:01:54.237915 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.237867 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" podStartSLOduration=1.508739116 podStartE2EDuration="5.237852724s" totalImagesPullingTime="3.729113608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:01:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:01:54.237801657 +0000 UTC m=+403.209835067" watchObservedRunningTime="2026-07-10 16:01:54.237852724 +0000 UTC m=+403.209886100" Jul 10 16:01:54.679073 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.679045 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf"] Jul 10 16:01:54.682276 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.682256 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.684855 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.684834 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-xv9vh\"" Jul 10 16:01:54.701302 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.701278 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf"] Jul 10 16:01:54.733424 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.733375 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e361be6-accf-41f9-870f-ee44f6664edd-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.733605 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.733452 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e361be6-accf-41f9-870f-ee44f6664edd-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.733605 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.733486 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e361be6-accf-41f9-870f-ee44f6664edd-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.733605 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.733515 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.733605 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.733556 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.733605 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.733588 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.733844 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.733668 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.733844 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.733708 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.733844 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.733743 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46lkp\" (UniqueName: \"kubernetes.io/projected/3e361be6-accf-41f9-870f-ee44f6664edd-kube-api-access-46lkp\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.834562 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.834520 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.834776 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.834581 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46lkp\" (UniqueName: \"kubernetes.io/projected/3e361be6-accf-41f9-870f-ee44f6664edd-kube-api-access-46lkp\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.834776 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.834626 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e361be6-accf-41f9-870f-ee44f6664edd-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.834776 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.834679 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e361be6-accf-41f9-870f-ee44f6664edd-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.834776 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.834718 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e361be6-accf-41f9-870f-ee44f6664edd-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.834776 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.834748 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.835084 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.834794 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.835084 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.834835 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.835084 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.834880 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.835248 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.835226 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.835470 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.835445 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.835637 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.835610 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.835637 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.835464 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.835811 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.835533 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e361be6-accf-41f9-870f-ee44f6664edd-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.837628 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.837605 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.837714 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.837638 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e361be6-accf-41f9-870f-ee44f6664edd-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.846039 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.846015 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46lkp\" (UniqueName: \"kubernetes.io/projected/3e361be6-accf-41f9-870f-ee44f6664edd-kube-api-access-46lkp\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.846150 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.846131 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e361be6-accf-41f9-870f-ee44f6664edd-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:54.991698 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:54.991658 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:55.150561 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.150537 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf"] Jul 10 16:01:55.152582 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:01:55.152543 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e361be6_accf_41f9_870f_ee44f6664edd.slice/crio-f971fd001ae884213a38b063ae3068248ce5c8630a9ddf9a351ffecd3b8ddda7 WatchSource:0}: Error finding container f971fd001ae884213a38b063ae3068248ce5c8630a9ddf9a351ffecd3b8ddda7: Status 404 returned error can't find the container with id f971fd001ae884213a38b063ae3068248ce5c8630a9ddf9a351ffecd3b8ddda7 Jul 10 16:01:55.179445 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.179410 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" event={"ID":"3e361be6-accf-41f9-870f-ee44f6664edd","Type":"ContainerStarted","Data":"f971fd001ae884213a38b063ae3068248ce5c8630a9ddf9a351ffecd3b8ddda7"} Jul 10 16:01:55.180254 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.180237 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:55.634269 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.634235 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m"] Jul 10 16:01:55.638424 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.638407 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:55.683684 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.683652 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m"] Jul 10 16:01:55.743607 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.743574 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6205ec9a-afd1-4e7d-97b6-56b60c00847a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-5f67m\" (UID: \"6205ec9a-afd1-4e7d-97b6-56b60c00847a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:55.743607 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.743613 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6205ec9a-afd1-4e7d-97b6-56b60c00847a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-5f67m\" (UID: \"6205ec9a-afd1-4e7d-97b6-56b60c00847a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:55.743833 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.743637 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6205ec9a-afd1-4e7d-97b6-56b60c00847a-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-5f67m\" (UID: \"6205ec9a-afd1-4e7d-97b6-56b60c00847a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:55.743833 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.743709 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6205ec9a-afd1-4e7d-97b6-56b60c00847a-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-5f67m\" (UID: \"6205ec9a-afd1-4e7d-97b6-56b60c00847a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:55.743833 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.743743 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ll22\" (UniqueName: \"kubernetes.io/projected/6205ec9a-afd1-4e7d-97b6-56b60c00847a-kube-api-access-5ll22\") pod \"istiod-openshift-gateway-94bb8fbfd-5f67m\" (UID: \"6205ec9a-afd1-4e7d-97b6-56b60c00847a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:55.743833 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.743774 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6205ec9a-afd1-4e7d-97b6-56b60c00847a-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-5f67m\" (UID: \"6205ec9a-afd1-4e7d-97b6-56b60c00847a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:55.744000 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.743850 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6205ec9a-afd1-4e7d-97b6-56b60c00847a-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-5f67m\" (UID: \"6205ec9a-afd1-4e7d-97b6-56b60c00847a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:55.844552 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.844517 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6205ec9a-afd1-4e7d-97b6-56b60c00847a-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-5f67m\" (UID: \"6205ec9a-afd1-4e7d-97b6-56b60c00847a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:55.844722 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.844575 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6205ec9a-afd1-4e7d-97b6-56b60c00847a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-5f67m\" (UID: \"6205ec9a-afd1-4e7d-97b6-56b60c00847a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:55.844722 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.844609 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6205ec9a-afd1-4e7d-97b6-56b60c00847a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-5f67m\" (UID: \"6205ec9a-afd1-4e7d-97b6-56b60c00847a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:55.844722 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.844643 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6205ec9a-afd1-4e7d-97b6-56b60c00847a-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-5f67m\" (UID: \"6205ec9a-afd1-4e7d-97b6-56b60c00847a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:55.844722 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.844674 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6205ec9a-afd1-4e7d-97b6-56b60c00847a-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-5f67m\" (UID: \"6205ec9a-afd1-4e7d-97b6-56b60c00847a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:55.844722 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.844695 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ll22\" (UniqueName: \"kubernetes.io/projected/6205ec9a-afd1-4e7d-97b6-56b60c00847a-kube-api-access-5ll22\") pod \"istiod-openshift-gateway-94bb8fbfd-5f67m\" (UID: \"6205ec9a-afd1-4e7d-97b6-56b60c00847a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:55.844722 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.844721 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6205ec9a-afd1-4e7d-97b6-56b60c00847a-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-5f67m\" (UID: \"6205ec9a-afd1-4e7d-97b6-56b60c00847a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:55.847448 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.845522 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6205ec9a-afd1-4e7d-97b6-56b60c00847a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-5f67m\" (UID: \"6205ec9a-afd1-4e7d-97b6-56b60c00847a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:55.847448 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.847395 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6205ec9a-afd1-4e7d-97b6-56b60c00847a-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-5f67m\" (UID: \"6205ec9a-afd1-4e7d-97b6-56b60c00847a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:55.847685 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.847639 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6205ec9a-afd1-4e7d-97b6-56b60c00847a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-5f67m\" (UID: \"6205ec9a-afd1-4e7d-97b6-56b60c00847a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:55.847927 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.847902 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6205ec9a-afd1-4e7d-97b6-56b60c00847a-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-5f67m\" (UID: \"6205ec9a-afd1-4e7d-97b6-56b60c00847a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:55.847927 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.847912 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6205ec9a-afd1-4e7d-97b6-56b60c00847a-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-5f67m\" (UID: \"6205ec9a-afd1-4e7d-97b6-56b60c00847a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:55.855728 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.855684 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6205ec9a-afd1-4e7d-97b6-56b60c00847a-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-5f67m\" (UID: \"6205ec9a-afd1-4e7d-97b6-56b60c00847a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:55.855825 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.855748 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ll22\" (UniqueName: \"kubernetes.io/projected/6205ec9a-afd1-4e7d-97b6-56b60c00847a-kube-api-access-5ll22\") pod \"istiod-openshift-gateway-94bb8fbfd-5f67m\" (UID: \"6205ec9a-afd1-4e7d-97b6-56b60c00847a\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:55.947123 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:55.947036 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:56.100795 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:56.100765 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m"] Jul 10 16:01:56.103262 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:01:56.103231 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6205ec9a_afd1_4e7d_97b6_56b60c00847a.slice/crio-c9b4ffd7ce9d498af60e95ec16b7073aa0136a9a46ee5415e74292310480d7ee WatchSource:0}: Error finding container c9b4ffd7ce9d498af60e95ec16b7073aa0136a9a46ee5415e74292310480d7ee: Status 404 returned error can't find the container with id c9b4ffd7ce9d498af60e95ec16b7073aa0136a9a46ee5415e74292310480d7ee Jul 10 16:01:56.106745 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:56.106678 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:01:56.106836 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:56.106819 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:01:56.186614 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:56.186582 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" event={"ID":"6205ec9a-afd1-4e7d-97b6-56b60c00847a","Type":"ContainerStarted","Data":"c9b4ffd7ce9d498af60e95ec16b7073aa0136a9a46ee5415e74292310480d7ee"} Jul 10 16:01:57.193140 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.193102 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" event={"ID":"6205ec9a-afd1-4e7d-97b6-56b60c00847a","Type":"ContainerStarted","Data":"adb8ceff11431afd4e54d273ac3bb040d8f96bc559f73a2c8d1c35dac46dcd3f"} Jul 10 16:01:57.193624 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.193304 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:57.195346 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.195319 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" Jul 10 16:01:57.240867 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.240820 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-5f67m" podStartSLOduration=2.240799056 podStartE2EDuration="2.240799056s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:01:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:01:57.240238611 +0000 UTC m=+406.212272020" watchObservedRunningTime="2026-07-10 16:01:57.240799056 +0000 UTC m=+406.212832430" Jul 10 16:01:57.328516 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.328484 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4"] Jul 10 16:01:57.328737 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.328685 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" podUID="9c51051d-a8b7-401b-8115-ce2dc90d9762" containerName="discovery" containerID="cri-o://06bec6ba7841815c192e8d8b7a40ca169fad9c4b23bd470b1fa1a121ce5fbef0" gracePeriod=30 Jul 10 16:01:57.781235 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.781200 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:01:57.781320 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.781267 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:01:57.781320 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.781299 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:01:57.878059 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.878023 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:57.964941 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.964908 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2lv4g\" (UniqueName: \"kubernetes.io/projected/9c51051d-a8b7-401b-8115-ce2dc90d9762-kube-api-access-2lv4g\") pod \"9c51051d-a8b7-401b-8115-ce2dc90d9762\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " Jul 10 16:01:57.965136 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.965002 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-csr-ca-configmap\") pod \"9c51051d-a8b7-401b-8115-ce2dc90d9762\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " Jul 10 16:01:57.965136 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.965038 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-csr-dns-cert\") pod \"9c51051d-a8b7-401b-8115-ce2dc90d9762\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " Jul 10 16:01:57.965393 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.965355 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c51051d-a8b7-401b-8115-ce2dc90d9762-local-certs\") pod \"9c51051d-a8b7-401b-8115-ce2dc90d9762\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " Jul 10 16:01:57.965526 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.965428 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-kubeconfig\") pod \"9c51051d-a8b7-401b-8115-ce2dc90d9762\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " Jul 10 16:01:57.965526 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.965461 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-token\") pod \"9c51051d-a8b7-401b-8115-ce2dc90d9762\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " Jul 10 16:01:57.965526 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.965486 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9c51051d-a8b7-401b-8115-ce2dc90d9762-cacerts\") pod \"9c51051d-a8b7-401b-8115-ce2dc90d9762\" (UID: \"9c51051d-a8b7-401b-8115-ce2dc90d9762\") " Jul 10 16:01:57.965694 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.965580 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "9c51051d-a8b7-401b-8115-ce2dc90d9762" (UID: "9c51051d-a8b7-401b-8115-ce2dc90d9762"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 16:01:57.966042 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.965752 2560 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-csr-ca-configmap\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:01:57.968281 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.968254 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "9c51051d-a8b7-401b-8115-ce2dc90d9762" (UID: "9c51051d-a8b7-401b-8115-ce2dc90d9762"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 16:01:57.968384 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.968275 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c51051d-a8b7-401b-8115-ce2dc90d9762-cacerts" (OuterVolumeSpecName: "cacerts") pod "9c51051d-a8b7-401b-8115-ce2dc90d9762" (UID: "9c51051d-a8b7-401b-8115-ce2dc90d9762"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 16:01:57.968448 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.968378 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c51051d-a8b7-401b-8115-ce2dc90d9762-kube-api-access-2lv4g" (OuterVolumeSpecName: "kube-api-access-2lv4g") pod "9c51051d-a8b7-401b-8115-ce2dc90d9762" (UID: "9c51051d-a8b7-401b-8115-ce2dc90d9762"). InnerVolumeSpecName "kube-api-access-2lv4g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:01:57.968697 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.968664 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "9c51051d-a8b7-401b-8115-ce2dc90d9762" (UID: "9c51051d-a8b7-401b-8115-ce2dc90d9762"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 10 16:01:57.968895 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.968866 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c51051d-a8b7-401b-8115-ce2dc90d9762-local-certs" (OuterVolumeSpecName: "local-certs") pod "9c51051d-a8b7-401b-8115-ce2dc90d9762" (UID: "9c51051d-a8b7-401b-8115-ce2dc90d9762"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:01:57.969025 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:57.969003 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-token" (OuterVolumeSpecName: "istio-token") pod "9c51051d-a8b7-401b-8115-ce2dc90d9762" (UID: "9c51051d-a8b7-401b-8115-ce2dc90d9762"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:01:58.066831 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.066803 2560 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9c51051d-a8b7-401b-8115-ce2dc90d9762-local-certs\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:01:58.066831 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.066830 2560 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-kubeconfig\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:01:58.066831 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.066839 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-token\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:01:58.067069 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.066851 2560 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9c51051d-a8b7-401b-8115-ce2dc90d9762-cacerts\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:01:58.067069 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.066863 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2lv4g\" (UniqueName: \"kubernetes.io/projected/9c51051d-a8b7-401b-8115-ce2dc90d9762-kube-api-access-2lv4g\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:01:58.067069 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.066873 2560 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9c51051d-a8b7-401b-8115-ce2dc90d9762-istio-csr-dns-cert\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:01:58.199470 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.199428 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" event={"ID":"3e361be6-accf-41f9-870f-ee44f6664edd","Type":"ContainerStarted","Data":"74577e9888864be050c88ae0215ad21c8e49952c66eeb12bdc2d736a040fa13d"} Jul 10 16:01:58.200901 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.200873 2560 generic.go:356] "Generic (PLEG): container finished" podID="9c51051d-a8b7-401b-8115-ce2dc90d9762" containerID="06bec6ba7841815c192e8d8b7a40ca169fad9c4b23bd470b1fa1a121ce5fbef0" exitCode=0 Jul 10 16:01:58.201091 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.200924 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" Jul 10 16:01:58.201091 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.200968 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" event={"ID":"9c51051d-a8b7-401b-8115-ce2dc90d9762","Type":"ContainerDied","Data":"06bec6ba7841815c192e8d8b7a40ca169fad9c4b23bd470b1fa1a121ce5fbef0"} Jul 10 16:01:58.201091 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.201017 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4" event={"ID":"9c51051d-a8b7-401b-8115-ce2dc90d9762","Type":"ContainerDied","Data":"6d344755432dcfb8a991314f9b15e108bfb89629770d2829198f711818fa8ed7"} Jul 10 16:01:58.201091 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.201053 2560 scope.go:117] "RemoveContainer" containerID="06bec6ba7841815c192e8d8b7a40ca169fad9c4b23bd470b1fa1a121ce5fbef0" Jul 10 16:01:58.210615 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.210597 2560 scope.go:117] "RemoveContainer" containerID="06bec6ba7841815c192e8d8b7a40ca169fad9c4b23bd470b1fa1a121ce5fbef0" Jul 10 16:01:58.210922 ip-10-0-129-101 kubenswrapper[2560]: E0710 16:01:58.210897 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06bec6ba7841815c192e8d8b7a40ca169fad9c4b23bd470b1fa1a121ce5fbef0\": container with ID starting with 06bec6ba7841815c192e8d8b7a40ca169fad9c4b23bd470b1fa1a121ce5fbef0 not found: ID does not exist" containerID="06bec6ba7841815c192e8d8b7a40ca169fad9c4b23bd470b1fa1a121ce5fbef0" Jul 10 16:01:58.211057 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.210930 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06bec6ba7841815c192e8d8b7a40ca169fad9c4b23bd470b1fa1a121ce5fbef0"} err="failed to get container status \"06bec6ba7841815c192e8d8b7a40ca169fad9c4b23bd470b1fa1a121ce5fbef0\": rpc error: code = NotFound desc = could not find container \"06bec6ba7841815c192e8d8b7a40ca169fad9c4b23bd470b1fa1a121ce5fbef0\": container with ID starting with 06bec6ba7841815c192e8d8b7a40ca169fad9c4b23bd470b1fa1a121ce5fbef0 not found: ID does not exist" Jul 10 16:01:58.243561 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.243471 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" podStartSLOduration=1.617077909 podStartE2EDuration="4.243452724s" totalImagesPullingTime="2.626374815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:01:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:01:58.242802795 +0000 UTC m=+407.214836175" watchObservedRunningTime="2026-07-10 16:01:58.243452724 +0000 UTC m=+407.215486098" Jul 10 16:01:58.287256 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.287213 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4"] Jul 10 16:01:58.292822 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.292787 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-nwpc4"] Jul 10 16:01:58.899787 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.899749 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-65stt"] Jul 10 16:01:58.900085 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.900071 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c51051d-a8b7-401b-8115-ce2dc90d9762" containerName="discovery" Jul 10 16:01:58.900144 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.900086 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c51051d-a8b7-401b-8115-ce2dc90d9762" containerName="discovery" Jul 10 16:01:58.900214 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.900147 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c51051d-a8b7-401b-8115-ce2dc90d9762" containerName="discovery" Jul 10 16:01:58.904248 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.904231 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-65stt" Jul 10 16:01:58.907090 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.907068 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 10 16:01:58.907198 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.907145 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-9fhr9\"" Jul 10 16:01:58.925105 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.925082 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-65stt"] Jul 10 16:01:58.974756 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.974723 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2rq7\" (UniqueName: \"kubernetes.io/projected/447098e2-a5fe-4c36-833b-59631e845e53-kube-api-access-w2rq7\") pod \"kserve-controller-manager-d6f4b65b8-65stt\" (UID: \"447098e2-a5fe-4c36-833b-59631e845e53\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-65stt" Jul 10 16:01:58.974895 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.974764 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/447098e2-a5fe-4c36-833b-59631e845e53-cert\") pod \"kserve-controller-manager-d6f4b65b8-65stt\" (UID: \"447098e2-a5fe-4c36-833b-59631e845e53\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-65stt" Jul 10 16:01:58.992665 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.992638 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:01:58.993691 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.993667 2560 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Jul 10 16:01:58.993801 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:58.993720 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" podUID="3e361be6-accf-41f9-870f-ee44f6664edd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Jul 10 16:01:59.075483 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:59.075454 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2rq7\" (UniqueName: \"kubernetes.io/projected/447098e2-a5fe-4c36-833b-59631e845e53-kube-api-access-w2rq7\") pod \"kserve-controller-manager-d6f4b65b8-65stt\" (UID: \"447098e2-a5fe-4c36-833b-59631e845e53\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-65stt" Jul 10 16:01:59.075483 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:59.075487 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/447098e2-a5fe-4c36-833b-59631e845e53-cert\") pod \"kserve-controller-manager-d6f4b65b8-65stt\" (UID: \"447098e2-a5fe-4c36-833b-59631e845e53\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-65stt" Jul 10 16:01:59.075784 ip-10-0-129-101 kubenswrapper[2560]: E0710 16:01:59.075664 2560 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 10 16:01:59.075784 ip-10-0-129-101 kubenswrapper[2560]: E0710 16:01:59.075740 2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/447098e2-a5fe-4c36-833b-59631e845e53-cert podName:447098e2-a5fe-4c36-833b-59631e845e53 nodeName:}" failed. No retries permitted until 2026-07-10 16:01:59.57571813 +0000 UTC m=+408.547751496 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/447098e2-a5fe-4c36-833b-59631e845e53-cert") pod "kserve-controller-manager-d6f4b65b8-65stt" (UID: "447098e2-a5fe-4c36-833b-59631e845e53") : secret "kserve-webhook-server-cert" not found Jul 10 16:01:59.084823 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:59.084796 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2rq7\" (UniqueName: \"kubernetes.io/projected/447098e2-a5fe-4c36-833b-59631e845e53-kube-api-access-w2rq7\") pod \"kserve-controller-manager-d6f4b65b8-65stt\" (UID: \"447098e2-a5fe-4c36-833b-59631e845e53\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-65stt" Jul 10 16:01:59.483767 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:59.483733 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf"] Jul 10 16:01:59.551538 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:59.551503 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c51051d-a8b7-401b-8115-ce2dc90d9762" path="/var/lib/kubelet/pods/9c51051d-a8b7-401b-8115-ce2dc90d9762/volumes" Jul 10 16:01:59.580870 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:59.580836 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/447098e2-a5fe-4c36-833b-59631e845e53-cert\") pod \"kserve-controller-manager-d6f4b65b8-65stt\" (UID: \"447098e2-a5fe-4c36-833b-59631e845e53\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-65stt" Jul 10 16:01:59.583140 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:59.583114 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/447098e2-a5fe-4c36-833b-59631e845e53-cert\") pod \"kserve-controller-manager-d6f4b65b8-65stt\" (UID: \"447098e2-a5fe-4c36-833b-59631e845e53\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-65stt" Jul 10 16:01:59.816404 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:59.816319 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-65stt" Jul 10 16:01:59.960745 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:59.960723 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-65stt"] Jul 10 16:01:59.962766 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:01:59.962743 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod447098e2_a5fe_4c36_833b_59631e845e53.slice/crio-9b2a417f3fe79496ff5b8ef22fe224d900c26f0ade3efd84277f9fbff8531aa2 WatchSource:0}: Error finding container 9b2a417f3fe79496ff5b8ef22fe224d900c26f0ade3efd84277f9fbff8531aa2: Status 404 returned error can't find the container with id 9b2a417f3fe79496ff5b8ef22fe224d900c26f0ade3efd84277f9fbff8531aa2 Jul 10 16:01:59.992323 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:59.992297 2560 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Jul 10 16:01:59.992465 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:01:59.992336 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" podUID="3e361be6-accf-41f9-870f-ee44f6664edd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Jul 10 16:02:00.214733 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:00.214695 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-65stt" event={"ID":"447098e2-a5fe-4c36-833b-59631e845e53","Type":"ContainerStarted","Data":"9b2a417f3fe79496ff5b8ef22fe224d900c26f0ade3efd84277f9fbff8531aa2"} Jul 10 16:02:00.992940 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:00.992898 2560 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Jul 10 16:02:00.993320 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:00.992993 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" podUID="3e361be6-accf-41f9-870f-ee44f6664edd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Jul 10 16:02:01.218820 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:01.218775 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" podUID="3e361be6-accf-41f9-870f-ee44f6664edd" containerName="istio-proxy" containerID="cri-o://74577e9888864be050c88ae0215ad21c8e49952c66eeb12bdc2d736a040fa13d" gracePeriod=30 Jul 10 16:02:03.229076 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:03.229045 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-65stt" event={"ID":"447098e2-a5fe-4c36-833b-59631e845e53","Type":"ContainerStarted","Data":"dcf7cb0dbcabb96f741ac75a61c62c2aae43b3fcdeae1707efebb4f8fafdad94"} Jul 10 16:02:03.229514 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:03.229174 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-65stt" Jul 10 16:02:03.278378 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:03.278328 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-65stt" podStartSLOduration=2.423185298 podStartE2EDuration="5.278312018s" totalImagesPullingTime="2.85512672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:01:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:02:03.276234775 +0000 UTC m=+412.248268168" watchObservedRunningTime="2026-07-10 16:02:03.278312018 +0000 UTC m=+412.250345394" Jul 10 16:02:05.181894 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:05.181862 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-f9w9q" Jul 10 16:02:06.463870 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.463849 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:02:06.540420 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.540345 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-istio-data\") pod \"3e361be6-accf-41f9-870f-ee44f6664edd\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " Jul 10 16:02:06.540420 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.540417 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e361be6-accf-41f9-870f-ee44f6664edd-istiod-ca-cert\") pod \"3e361be6-accf-41f9-870f-ee44f6664edd\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " Jul 10 16:02:06.540631 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.540436 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-credential-socket\") pod \"3e361be6-accf-41f9-870f-ee44f6664edd\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " Jul 10 16:02:06.540631 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.540459 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e361be6-accf-41f9-870f-ee44f6664edd-istio-token\") pod \"3e361be6-accf-41f9-870f-ee44f6664edd\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " Jul 10 16:02:06.540631 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.540484 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-istio-envoy\") pod \"3e361be6-accf-41f9-870f-ee44f6664edd\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " Jul 10 16:02:06.540631 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.540500 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-workload-certs\") pod \"3e361be6-accf-41f9-870f-ee44f6664edd\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " Jul 10 16:02:06.540631 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.540525 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-workload-socket\") pod \"3e361be6-accf-41f9-870f-ee44f6664edd\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " Jul 10 16:02:06.540631 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.540561 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-istio-data" (OuterVolumeSpecName: "istio-data") pod "3e361be6-accf-41f9-870f-ee44f6664edd" (UID: "3e361be6-accf-41f9-870f-ee44f6664edd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:02:06.540631 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.540616 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e361be6-accf-41f9-870f-ee44f6664edd-istio-podinfo\") pod \"3e361be6-accf-41f9-870f-ee44f6664edd\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " Jul 10 16:02:06.540986 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.540660 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46lkp\" (UniqueName: \"kubernetes.io/projected/3e361be6-accf-41f9-870f-ee44f6664edd-kube-api-access-46lkp\") pod \"3e361be6-accf-41f9-870f-ee44f6664edd\" (UID: \"3e361be6-accf-41f9-870f-ee44f6664edd\") " Jul 10 16:02:06.540986 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.540667 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3e361be6-accf-41f9-870f-ee44f6664edd" (UID: "3e361be6-accf-41f9-870f-ee44f6664edd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:02:06.540986 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.540892 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-istio-data\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:02:06.540986 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.540912 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-credential-socket\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:02:06.541182 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.541126 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3e361be6-accf-41f9-870f-ee44f6664edd" (UID: "3e361be6-accf-41f9-870f-ee44f6664edd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:02:06.541237 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.541178 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3e361be6-accf-41f9-870f-ee44f6664edd" (UID: "3e361be6-accf-41f9-870f-ee44f6664edd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:02:06.541237 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.541218 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e361be6-accf-41f9-870f-ee44f6664edd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3e361be6-accf-41f9-870f-ee44f6664edd" (UID: "3e361be6-accf-41f9-870f-ee44f6664edd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 16:02:06.542869 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.542843 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e361be6-accf-41f9-870f-ee44f6664edd-kube-api-access-46lkp" (OuterVolumeSpecName: "kube-api-access-46lkp") pod "3e361be6-accf-41f9-870f-ee44f6664edd" (UID: "3e361be6-accf-41f9-870f-ee44f6664edd"). InnerVolumeSpecName "kube-api-access-46lkp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:02:06.542869 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.542862 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3e361be6-accf-41f9-870f-ee44f6664edd" (UID: "3e361be6-accf-41f9-870f-ee44f6664edd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:02:06.543154 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.543130 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3e361be6-accf-41f9-870f-ee44f6664edd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3e361be6-accf-41f9-870f-ee44f6664edd" (UID: "3e361be6-accf-41f9-870f-ee44f6664edd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 16:02:06.543154 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.543137 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e361be6-accf-41f9-870f-ee44f6664edd-istio-token" (OuterVolumeSpecName: "istio-token") pod "3e361be6-accf-41f9-870f-ee44f6664edd" (UID: "3e361be6-accf-41f9-870f-ee44f6664edd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:02:06.642162 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.642127 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e361be6-accf-41f9-870f-ee44f6664edd-istiod-ca-cert\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:02:06.642162 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.642153 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e361be6-accf-41f9-870f-ee44f6664edd-istio-token\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:02:06.642162 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.642162 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-istio-envoy\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:02:06.642162 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.642171 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-workload-certs\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:02:06.642414 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.642179 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e361be6-accf-41f9-870f-ee44f6664edd-workload-socket\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:02:06.642414 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.642188 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e361be6-accf-41f9-870f-ee44f6664edd-istio-podinfo\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:02:06.642414 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:06.642196 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46lkp\" (UniqueName: \"kubernetes.io/projected/3e361be6-accf-41f9-870f-ee44f6664edd-kube-api-access-46lkp\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:02:07.248340 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:07.248309 2560 generic.go:356] "Generic (PLEG): container finished" podID="3e361be6-accf-41f9-870f-ee44f6664edd" containerID="74577e9888864be050c88ae0215ad21c8e49952c66eeb12bdc2d736a040fa13d" exitCode=0 Jul 10 16:02:07.248511 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:07.248356 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" Jul 10 16:02:07.248511 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:07.248359 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" event={"ID":"3e361be6-accf-41f9-870f-ee44f6664edd","Type":"ContainerDied","Data":"74577e9888864be050c88ae0215ad21c8e49952c66eeb12bdc2d736a040fa13d"} Jul 10 16:02:07.248511 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:07.248401 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf" event={"ID":"3e361be6-accf-41f9-870f-ee44f6664edd","Type":"ContainerDied","Data":"f971fd001ae884213a38b063ae3068248ce5c8630a9ddf9a351ffecd3b8ddda7"} Jul 10 16:02:07.248511 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:07.248422 2560 scope.go:117] "RemoveContainer" containerID="74577e9888864be050c88ae0215ad21c8e49952c66eeb12bdc2d736a040fa13d" Jul 10 16:02:07.256907 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:07.256888 2560 scope.go:117] "RemoveContainer" containerID="74577e9888864be050c88ae0215ad21c8e49952c66eeb12bdc2d736a040fa13d" Jul 10 16:02:07.257166 ip-10-0-129-101 kubenswrapper[2560]: E0710 16:02:07.257149 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74577e9888864be050c88ae0215ad21c8e49952c66eeb12bdc2d736a040fa13d\": container with ID starting with 74577e9888864be050c88ae0215ad21c8e49952c66eeb12bdc2d736a040fa13d not found: ID does not exist" containerID="74577e9888864be050c88ae0215ad21c8e49952c66eeb12bdc2d736a040fa13d" Jul 10 16:02:07.257211 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:07.257175 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74577e9888864be050c88ae0215ad21c8e49952c66eeb12bdc2d736a040fa13d"} err="failed to get container status \"74577e9888864be050c88ae0215ad21c8e49952c66eeb12bdc2d736a040fa13d\": rpc error: code = NotFound desc = could not find container \"74577e9888864be050c88ae0215ad21c8e49952c66eeb12bdc2d736a040fa13d\": container with ID starting with 74577e9888864be050c88ae0215ad21c8e49952c66eeb12bdc2d736a040fa13d not found: ID does not exist" Jul 10 16:02:07.270043 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:07.270012 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf"] Jul 10 16:02:07.275250 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:07.275230 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-vl9zf"] Jul 10 16:02:07.551764 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:07.551691 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e361be6-accf-41f9-870f-ee44f6664edd" path="/var/lib/kubelet/pods/3e361be6-accf-41f9-870f-ee44f6664edd/volumes" Jul 10 16:02:34.239577 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:34.239497 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-65stt" Jul 10 16:02:48.650457 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:48.650421 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p7bpq"] Jul 10 16:02:48.650836 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:48.650758 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e361be6-accf-41f9-870f-ee44f6664edd" containerName="istio-proxy" Jul 10 16:02:48.650836 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:48.650771 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e361be6-accf-41f9-870f-ee44f6664edd" containerName="istio-proxy" Jul 10 16:02:48.650836 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:48.650818 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e361be6-accf-41f9-870f-ee44f6664edd" containerName="istio-proxy" Jul 10 16:02:48.653943 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:48.653926 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-p7bpq" Jul 10 16:02:48.657324 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:48.657297 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-jtqcn\"" Jul 10 16:02:48.658382 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:48.658351 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 10 16:02:48.658382 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:48.658368 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 10 16:02:48.671244 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:48.671220 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p7bpq"] Jul 10 16:02:48.762203 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:48.762168 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97t2q\" (UniqueName: \"kubernetes.io/projected/91fca889-5351-4390-b556-d1e2ee0e9f10-kube-api-access-97t2q\") pod \"kuadrant-operator-catalog-p7bpq\" (UID: \"91fca889-5351-4390-b556-d1e2ee0e9f10\") " pod="kuadrant-system/kuadrant-operator-catalog-p7bpq" Jul 10 16:02:48.862811 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:48.862781 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97t2q\" (UniqueName: \"kubernetes.io/projected/91fca889-5351-4390-b556-d1e2ee0e9f10-kube-api-access-97t2q\") pod \"kuadrant-operator-catalog-p7bpq\" (UID: \"91fca889-5351-4390-b556-d1e2ee0e9f10\") " pod="kuadrant-system/kuadrant-operator-catalog-p7bpq" Jul 10 16:02:48.871876 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:48.871843 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97t2q\" (UniqueName: \"kubernetes.io/projected/91fca889-5351-4390-b556-d1e2ee0e9f10-kube-api-access-97t2q\") pod \"kuadrant-operator-catalog-p7bpq\" (UID: \"91fca889-5351-4390-b556-d1e2ee0e9f10\") " pod="kuadrant-system/kuadrant-operator-catalog-p7bpq" Jul 10 16:02:48.965313 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:48.965262 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-p7bpq" Jul 10 16:02:48.967493 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:48.967303 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p7bpq"] Jul 10 16:02:49.094696 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:49.094671 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p7bpq"] Jul 10 16:02:49.096603 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:02:49.096574 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91fca889_5351_4390_b556_d1e2ee0e9f10.slice/crio-c1ca74b9b06e5e9064d3d438a2d1125f9c3265ca3e602120dfd69222a89aa7b6 WatchSource:0}: Error finding container c1ca74b9b06e5e9064d3d438a2d1125f9c3265ca3e602120dfd69222a89aa7b6: Status 404 returned error can't find the container with id c1ca74b9b06e5e9064d3d438a2d1125f9c3265ca3e602120dfd69222a89aa7b6 Jul 10 16:02:49.176792 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:49.176768 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-w9vd9"] Jul 10 16:02:49.179624 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:49.179609 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-w9vd9" Jul 10 16:02:49.189842 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:49.189819 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-w9vd9"] Jul 10 16:02:49.266288 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:49.266211 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th6r5\" (UniqueName: \"kubernetes.io/projected/a004768d-ac8e-4ef3-9d38-a1547c29c362-kube-api-access-th6r5\") pod \"kuadrant-operator-catalog-w9vd9\" (UID: \"a004768d-ac8e-4ef3-9d38-a1547c29c362\") " pod="kuadrant-system/kuadrant-operator-catalog-w9vd9" Jul 10 16:02:49.367187 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:49.367152 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th6r5\" (UniqueName: \"kubernetes.io/projected/a004768d-ac8e-4ef3-9d38-a1547c29c362-kube-api-access-th6r5\") pod \"kuadrant-operator-catalog-w9vd9\" (UID: \"a004768d-ac8e-4ef3-9d38-a1547c29c362\") " pod="kuadrant-system/kuadrant-operator-catalog-w9vd9" Jul 10 16:02:49.376884 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:49.376848 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-th6r5\" (UniqueName: \"kubernetes.io/projected/a004768d-ac8e-4ef3-9d38-a1547c29c362-kube-api-access-th6r5\") pod \"kuadrant-operator-catalog-w9vd9\" (UID: \"a004768d-ac8e-4ef3-9d38-a1547c29c362\") " pod="kuadrant-system/kuadrant-operator-catalog-w9vd9" Jul 10 16:02:49.427566 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:49.427531 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-p7bpq" event={"ID":"91fca889-5351-4390-b556-d1e2ee0e9f10","Type":"ContainerStarted","Data":"c1ca74b9b06e5e9064d3d438a2d1125f9c3265ca3e602120dfd69222a89aa7b6"} Jul 10 16:02:49.489070 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:49.489041 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-w9vd9" Jul 10 16:02:49.618515 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:49.618492 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-w9vd9"] Jul 10 16:02:49.620460 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:02:49.620432 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda004768d_ac8e_4ef3_9d38_a1547c29c362.slice/crio-b923278e5c578ad9a87d1dde50feeb49347d9f412601ea4226273b380ba794c5 WatchSource:0}: Error finding container b923278e5c578ad9a87d1dde50feeb49347d9f412601ea4226273b380ba794c5: Status 404 returned error can't find the container with id b923278e5c578ad9a87d1dde50feeb49347d9f412601ea4226273b380ba794c5 Jul 10 16:02:50.433742 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:50.433708 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-w9vd9" event={"ID":"a004768d-ac8e-4ef3-9d38-a1547c29c362","Type":"ContainerStarted","Data":"b923278e5c578ad9a87d1dde50feeb49347d9f412601ea4226273b380ba794c5"} Jul 10 16:02:52.444211 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:52.444169 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-p7bpq" event={"ID":"91fca889-5351-4390-b556-d1e2ee0e9f10","Type":"ContainerStarted","Data":"10b8ea367aee7706cb6309091910d7ec1e009fa034582226401cf515c211c4e0"} Jul 10 16:02:52.444675 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:52.444182 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-p7bpq" podUID="91fca889-5351-4390-b556-d1e2ee0e9f10" containerName="registry-server" containerID="cri-o://10b8ea367aee7706cb6309091910d7ec1e009fa034582226401cf515c211c4e0" gracePeriod=2 Jul 10 16:02:52.445721 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:52.445686 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-w9vd9" event={"ID":"a004768d-ac8e-4ef3-9d38-a1547c29c362","Type":"ContainerStarted","Data":"d07e7c51597c7e3ddb2ebbf39b39ec80563685deff2057f3bb27cefee38c3aae"} Jul 10 16:02:52.481365 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:52.481320 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-p7bpq" podStartSLOduration=1.781139877 podStartE2EDuration="4.481301728s" totalImagesPullingTime="2.700161851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:02:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:02:52.461861724 +0000 UTC m=+461.433895099" watchObservedRunningTime="2026-07-10 16:02:52.481301728 +0000 UTC m=+461.453335102" Jul 10 16:02:52.481944 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:52.481917 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-w9vd9" podStartSLOduration=1.306469426 podStartE2EDuration="3.481905215s" totalImagesPullingTime="2.175435789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:02:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:02:52.480606621 +0000 UTC m=+461.452640021" watchObservedRunningTime="2026-07-10 16:02:52.481905215 +0000 UTC m=+461.453938589" Jul 10 16:02:52.682915 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:52.682893 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-p7bpq" Jul 10 16:02:52.794648 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:52.794609 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97t2q\" (UniqueName: \"kubernetes.io/projected/91fca889-5351-4390-b556-d1e2ee0e9f10-kube-api-access-97t2q\") pod \"91fca889-5351-4390-b556-d1e2ee0e9f10\" (UID: \"91fca889-5351-4390-b556-d1e2ee0e9f10\") " Jul 10 16:02:52.796698 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:52.796667 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91fca889-5351-4390-b556-d1e2ee0e9f10-kube-api-access-97t2q" (OuterVolumeSpecName: "kube-api-access-97t2q") pod "91fca889-5351-4390-b556-d1e2ee0e9f10" (UID: "91fca889-5351-4390-b556-d1e2ee0e9f10"). InnerVolumeSpecName "kube-api-access-97t2q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:02:52.895140 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:52.895110 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97t2q\" (UniqueName: \"kubernetes.io/projected/91fca889-5351-4390-b556-d1e2ee0e9f10-kube-api-access-97t2q\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:02:53.451373 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:53.451338 2560 generic.go:356] "Generic (PLEG): container finished" podID="91fca889-5351-4390-b556-d1e2ee0e9f10" containerID="10b8ea367aee7706cb6309091910d7ec1e009fa034582226401cf515c211c4e0" exitCode=0 Jul 10 16:02:53.451373 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:53.451378 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-p7bpq" Jul 10 16:02:53.451809 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:53.451422 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-p7bpq" event={"ID":"91fca889-5351-4390-b556-d1e2ee0e9f10","Type":"ContainerDied","Data":"10b8ea367aee7706cb6309091910d7ec1e009fa034582226401cf515c211c4e0"} Jul 10 16:02:53.451809 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:53.451473 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-p7bpq" event={"ID":"91fca889-5351-4390-b556-d1e2ee0e9f10","Type":"ContainerDied","Data":"c1ca74b9b06e5e9064d3d438a2d1125f9c3265ca3e602120dfd69222a89aa7b6"} Jul 10 16:02:53.451809 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:53.451500 2560 scope.go:117] "RemoveContainer" containerID="10b8ea367aee7706cb6309091910d7ec1e009fa034582226401cf515c211c4e0" Jul 10 16:02:53.460934 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:53.460915 2560 scope.go:117] "RemoveContainer" containerID="10b8ea367aee7706cb6309091910d7ec1e009fa034582226401cf515c211c4e0" Jul 10 16:02:53.461197 ip-10-0-129-101 kubenswrapper[2560]: E0710 16:02:53.461181 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10b8ea367aee7706cb6309091910d7ec1e009fa034582226401cf515c211c4e0\": container with ID starting with 10b8ea367aee7706cb6309091910d7ec1e009fa034582226401cf515c211c4e0 not found: ID does not exist" containerID="10b8ea367aee7706cb6309091910d7ec1e009fa034582226401cf515c211c4e0" Jul 10 16:02:53.461248 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:53.461204 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10b8ea367aee7706cb6309091910d7ec1e009fa034582226401cf515c211c4e0"} err="failed to get container status \"10b8ea367aee7706cb6309091910d7ec1e009fa034582226401cf515c211c4e0\": rpc error: code = NotFound desc = could not find container \"10b8ea367aee7706cb6309091910d7ec1e009fa034582226401cf515c211c4e0\": container with ID starting with 10b8ea367aee7706cb6309091910d7ec1e009fa034582226401cf515c211c4e0 not found: ID does not exist" Jul 10 16:02:53.472021 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:53.471995 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p7bpq"] Jul 10 16:02:53.476072 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:53.476052 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p7bpq"] Jul 10 16:02:53.551801 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:53.551762 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91fca889-5351-4390-b556-d1e2ee0e9f10" path="/var/lib/kubelet/pods/91fca889-5351-4390-b556-d1e2ee0e9f10/volumes" Jul 10 16:02:59.489796 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:59.489752 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-w9vd9" Jul 10 16:02:59.490350 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:59.489818 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-w9vd9" Jul 10 16:02:59.511799 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:02:59.511774 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-w9vd9" Jul 10 16:03:00.503913 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:00.503886 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-w9vd9" Jul 10 16:03:04.022663 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.022627 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh"] Jul 10 16:03:04.023094 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.022907 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91fca889-5351-4390-b556-d1e2ee0e9f10" containerName="registry-server" Jul 10 16:03:04.023094 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.022918 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="91fca889-5351-4390-b556-d1e2ee0e9f10" containerName="registry-server" Jul 10 16:03:04.023094 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.022994 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="91fca889-5351-4390-b556-d1e2ee0e9f10" containerName="registry-server" Jul 10 16:03:04.024711 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.024695 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh" Jul 10 16:03:04.027577 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.027557 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jbqvq\"" Jul 10 16:03:04.036180 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.036159 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh"] Jul 10 16:03:04.079902 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.079865 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd82ea12-c8d5-4256-bcdb-cd937fe07abd-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh\" (UID: \"dd82ea12-c8d5-4256-bcdb-cd937fe07abd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh" Jul 10 16:03:04.079902 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.079900 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd82ea12-c8d5-4256-bcdb-cd937fe07abd-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh\" (UID: \"dd82ea12-c8d5-4256-bcdb-cd937fe07abd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh" Jul 10 16:03:04.080118 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.079935 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzfr8\" (UniqueName: \"kubernetes.io/projected/dd82ea12-c8d5-4256-bcdb-cd937fe07abd-kube-api-access-mzfr8\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh\" (UID: \"dd82ea12-c8d5-4256-bcdb-cd937fe07abd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh" Jul 10 16:03:04.180755 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.180713 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd82ea12-c8d5-4256-bcdb-cd937fe07abd-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh\" (UID: \"dd82ea12-c8d5-4256-bcdb-cd937fe07abd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh" Jul 10 16:03:04.180755 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.180758 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd82ea12-c8d5-4256-bcdb-cd937fe07abd-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh\" (UID: \"dd82ea12-c8d5-4256-bcdb-cd937fe07abd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh" Jul 10 16:03:04.180949 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.180797 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzfr8\" (UniqueName: \"kubernetes.io/projected/dd82ea12-c8d5-4256-bcdb-cd937fe07abd-kube-api-access-mzfr8\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh\" (UID: \"dd82ea12-c8d5-4256-bcdb-cd937fe07abd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh" Jul 10 16:03:04.181144 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.181123 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd82ea12-c8d5-4256-bcdb-cd937fe07abd-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh\" (UID: \"dd82ea12-c8d5-4256-bcdb-cd937fe07abd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh" Jul 10 16:03:04.181144 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.181138 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd82ea12-c8d5-4256-bcdb-cd937fe07abd-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh\" (UID: \"dd82ea12-c8d5-4256-bcdb-cd937fe07abd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh" Jul 10 16:03:04.190522 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.190493 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzfr8\" (UniqueName: \"kubernetes.io/projected/dd82ea12-c8d5-4256-bcdb-cd937fe07abd-kube-api-access-mzfr8\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh\" (UID: \"dd82ea12-c8d5-4256-bcdb-cd937fe07abd\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh" Jul 10 16:03:04.333730 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.333642 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh" Jul 10 16:03:04.462629 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.462606 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh"] Jul 10 16:03:04.464902 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:03:04.464875 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd82ea12_c8d5_4256_bcdb_cd937fe07abd.slice/crio-0e0e61697dfb18b95e4ee9f160d1190b54cceaf077afd46e5301c411e758890b WatchSource:0}: Error finding container 0e0e61697dfb18b95e4ee9f160d1190b54cceaf077afd46e5301c411e758890b: Status 404 returned error can't find the container with id 0e0e61697dfb18b95e4ee9f160d1190b54cceaf077afd46e5301c411e758890b Jul 10 16:03:04.501081 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.501054 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh" event={"ID":"dd82ea12-c8d5-4256-bcdb-cd937fe07abd","Type":"ContainerStarted","Data":"0e0e61697dfb18b95e4ee9f160d1190b54cceaf077afd46e5301c411e758890b"} Jul 10 16:03:04.617994 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.617932 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n"] Jul 10 16:03:04.620182 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.620167 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n" Jul 10 16:03:04.630089 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.630062 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n"] Jul 10 16:03:04.685928 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.685906 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ghgr\" (UniqueName: \"kubernetes.io/projected/a7a1f505-d99d-4fc9-99b4-078f5842fb95-kube-api-access-7ghgr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n\" (UID: \"a7a1f505-d99d-4fc9-99b4-078f5842fb95\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n" Jul 10 16:03:04.686101 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.685948 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7a1f505-d99d-4fc9-99b4-078f5842fb95-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n\" (UID: \"a7a1f505-d99d-4fc9-99b4-078f5842fb95\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n" Jul 10 16:03:04.686101 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.686007 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7a1f505-d99d-4fc9-99b4-078f5842fb95-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n\" (UID: \"a7a1f505-d99d-4fc9-99b4-078f5842fb95\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n" Jul 10 16:03:04.786464 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.786424 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7a1f505-d99d-4fc9-99b4-078f5842fb95-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n\" (UID: \"a7a1f505-d99d-4fc9-99b4-078f5842fb95\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n" Jul 10 16:03:04.786643 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.786484 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ghgr\" (UniqueName: \"kubernetes.io/projected/a7a1f505-d99d-4fc9-99b4-078f5842fb95-kube-api-access-7ghgr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n\" (UID: \"a7a1f505-d99d-4fc9-99b4-078f5842fb95\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n" Jul 10 16:03:04.786643 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.786514 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7a1f505-d99d-4fc9-99b4-078f5842fb95-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n\" (UID: \"a7a1f505-d99d-4fc9-99b4-078f5842fb95\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n" Jul 10 16:03:04.786884 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.786867 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7a1f505-d99d-4fc9-99b4-078f5842fb95-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n\" (UID: \"a7a1f505-d99d-4fc9-99b4-078f5842fb95\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n" Jul 10 16:03:04.786918 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.786872 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7a1f505-d99d-4fc9-99b4-078f5842fb95-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n\" (UID: \"a7a1f505-d99d-4fc9-99b4-078f5842fb95\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n" Jul 10 16:03:04.795386 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.795360 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ghgr\" (UniqueName: \"kubernetes.io/projected/a7a1f505-d99d-4fc9-99b4-078f5842fb95-kube-api-access-7ghgr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n\" (UID: \"a7a1f505-d99d-4fc9-99b4-078f5842fb95\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n" Jul 10 16:03:04.929168 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:04.929101 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n" Jul 10 16:03:05.011146 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.011118 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5"] Jul 10 16:03:05.013979 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.013940 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5" Jul 10 16:03:05.022868 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.022721 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5"] Jul 10 16:03:05.055832 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.055805 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n"] Jul 10 16:03:05.057848 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:03:05.057824 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7a1f505_d99d_4fc9_99b4_078f5842fb95.slice/crio-cd408c8fe312aa9121a477b5b44a97946096a2d8d4e53c490deac92481812d4d WatchSource:0}: Error finding container cd408c8fe312aa9121a477b5b44a97946096a2d8d4e53c490deac92481812d4d: Status 404 returned error can't find the container with id cd408c8fe312aa9121a477b5b44a97946096a2d8d4e53c490deac92481812d4d Jul 10 16:03:05.089248 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.089221 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b4821b8-2b1a-4d1a-aa0c-d21554082e85-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5\" (UID: \"1b4821b8-2b1a-4d1a-aa0c-d21554082e85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5" Jul 10 16:03:05.089356 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.089262 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b4821b8-2b1a-4d1a-aa0c-d21554082e85-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5\" (UID: \"1b4821b8-2b1a-4d1a-aa0c-d21554082e85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5" Jul 10 16:03:05.089417 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.089349 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2875\" (UniqueName: \"kubernetes.io/projected/1b4821b8-2b1a-4d1a-aa0c-d21554082e85-kube-api-access-n2875\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5\" (UID: \"1b4821b8-2b1a-4d1a-aa0c-d21554082e85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5" Jul 10 16:03:05.190802 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.190744 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b4821b8-2b1a-4d1a-aa0c-d21554082e85-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5\" (UID: \"1b4821b8-2b1a-4d1a-aa0c-d21554082e85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5" Jul 10 16:03:05.190928 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.190805 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2875\" (UniqueName: \"kubernetes.io/projected/1b4821b8-2b1a-4d1a-aa0c-d21554082e85-kube-api-access-n2875\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5\" (UID: \"1b4821b8-2b1a-4d1a-aa0c-d21554082e85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5" Jul 10 16:03:05.190928 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.190832 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b4821b8-2b1a-4d1a-aa0c-d21554082e85-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5\" (UID: \"1b4821b8-2b1a-4d1a-aa0c-d21554082e85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5" Jul 10 16:03:05.191152 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.191137 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b4821b8-2b1a-4d1a-aa0c-d21554082e85-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5\" (UID: \"1b4821b8-2b1a-4d1a-aa0c-d21554082e85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5" Jul 10 16:03:05.191200 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.191139 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b4821b8-2b1a-4d1a-aa0c-d21554082e85-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5\" (UID: \"1b4821b8-2b1a-4d1a-aa0c-d21554082e85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5" Jul 10 16:03:05.208662 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.208636 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2875\" (UniqueName: \"kubernetes.io/projected/1b4821b8-2b1a-4d1a-aa0c-d21554082e85-kube-api-access-n2875\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5\" (UID: \"1b4821b8-2b1a-4d1a-aa0c-d21554082e85\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5" Jul 10 16:03:05.326299 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.326268 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5" Jul 10 16:03:05.421285 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.421257 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd"] Jul 10 16:03:05.424457 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.424432 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd" Jul 10 16:03:05.437672 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.437638 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd"] Jul 10 16:03:05.456538 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.456511 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5"] Jul 10 16:03:05.460815 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:03:05.460788 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b4821b8_2b1a_4d1a_aa0c_d21554082e85.slice/crio-345f45821df3e2a06a943d2cf65a59d0e2236bb620f5599790ec7900c9a09133 WatchSource:0}: Error finding container 345f45821df3e2a06a943d2cf65a59d0e2236bb620f5599790ec7900c9a09133: Status 404 returned error can't find the container with id 345f45821df3e2a06a943d2cf65a59d0e2236bb620f5599790ec7900c9a09133 Jul 10 16:03:05.493098 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.493072 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qtjk\" (UniqueName: \"kubernetes.io/projected/15ec01c2-78da-4669-a2ee-927426aea148-kube-api-access-9qtjk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd\" (UID: \"15ec01c2-78da-4669-a2ee-927426aea148\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd" Jul 10 16:03:05.493193 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.493112 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/15ec01c2-78da-4669-a2ee-927426aea148-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd\" (UID: \"15ec01c2-78da-4669-a2ee-927426aea148\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd" Jul 10 16:03:05.493193 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.493131 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/15ec01c2-78da-4669-a2ee-927426aea148-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd\" (UID: \"15ec01c2-78da-4669-a2ee-927426aea148\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd" Jul 10 16:03:05.507325 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.507297 2560 generic.go:356] "Generic (PLEG): container finished" podID="dd82ea12-c8d5-4256-bcdb-cd937fe07abd" containerID="d35a5c688679c958d0da566301d4e5f0fb694d179cd935f284c11bc8d20802d1" exitCode=0 Jul 10 16:03:05.507433 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.507379 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh" event={"ID":"dd82ea12-c8d5-4256-bcdb-cd937fe07abd","Type":"ContainerDied","Data":"d35a5c688679c958d0da566301d4e5f0fb694d179cd935f284c11bc8d20802d1"} Jul 10 16:03:05.509499 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.509476 2560 generic.go:356] "Generic (PLEG): container finished" podID="a7a1f505-d99d-4fc9-99b4-078f5842fb95" containerID="9761e842b58255b05e218aa209be940c3df0acccd40f9902a8d6bceea6abd69e" exitCode=0 Jul 10 16:03:05.509617 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.509558 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n" event={"ID":"a7a1f505-d99d-4fc9-99b4-078f5842fb95","Type":"ContainerDied","Data":"9761e842b58255b05e218aa209be940c3df0acccd40f9902a8d6bceea6abd69e"} Jul 10 16:03:05.509617 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.509609 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n" event={"ID":"a7a1f505-d99d-4fc9-99b4-078f5842fb95","Type":"ContainerStarted","Data":"cd408c8fe312aa9121a477b5b44a97946096a2d8d4e53c490deac92481812d4d"} Jul 10 16:03:05.510925 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.510904 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5" event={"ID":"1b4821b8-2b1a-4d1a-aa0c-d21554082e85","Type":"ContainerStarted","Data":"345f45821df3e2a06a943d2cf65a59d0e2236bb620f5599790ec7900c9a09133"} Jul 10 16:03:05.594507 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.594477 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qtjk\" (UniqueName: \"kubernetes.io/projected/15ec01c2-78da-4669-a2ee-927426aea148-kube-api-access-9qtjk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd\" (UID: \"15ec01c2-78da-4669-a2ee-927426aea148\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd" Jul 10 16:03:05.594618 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.594538 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/15ec01c2-78da-4669-a2ee-927426aea148-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd\" (UID: \"15ec01c2-78da-4669-a2ee-927426aea148\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd" Jul 10 16:03:05.594618 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.594565 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/15ec01c2-78da-4669-a2ee-927426aea148-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd\" (UID: \"15ec01c2-78da-4669-a2ee-927426aea148\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd" Jul 10 16:03:05.594849 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.594831 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/15ec01c2-78da-4669-a2ee-927426aea148-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd\" (UID: \"15ec01c2-78da-4669-a2ee-927426aea148\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd" Jul 10 16:03:05.594887 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.594874 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/15ec01c2-78da-4669-a2ee-927426aea148-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd\" (UID: \"15ec01c2-78da-4669-a2ee-927426aea148\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd" Jul 10 16:03:05.604298 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.604278 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qtjk\" (UniqueName: \"kubernetes.io/projected/15ec01c2-78da-4669-a2ee-927426aea148-kube-api-access-9qtjk\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd\" (UID: \"15ec01c2-78da-4669-a2ee-927426aea148\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd" Jul 10 16:03:05.736926 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.736851 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd" Jul 10 16:03:05.862651 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:05.862629 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd"] Jul 10 16:03:05.864530 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:03:05.864503 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15ec01c2_78da_4669_a2ee_927426aea148.slice/crio-c3535baee35ae2eedbdf81e22909ec1cd3dad3c2ec97b0b5fa9f864bd4a39e4f WatchSource:0}: Error finding container c3535baee35ae2eedbdf81e22909ec1cd3dad3c2ec97b0b5fa9f864bd4a39e4f: Status 404 returned error can't find the container with id c3535baee35ae2eedbdf81e22909ec1cd3dad3c2ec97b0b5fa9f864bd4a39e4f Jul 10 16:03:06.517131 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:06.517047 2560 generic.go:356] "Generic (PLEG): container finished" podID="dd82ea12-c8d5-4256-bcdb-cd937fe07abd" containerID="dfec321ebbb41491f77ea068458f3516763eb7d2c40eaa21de39f733f2fe850d" exitCode=0 Jul 10 16:03:06.517551 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:06.517132 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh" event={"ID":"dd82ea12-c8d5-4256-bcdb-cd937fe07abd","Type":"ContainerDied","Data":"dfec321ebbb41491f77ea068458f3516763eb7d2c40eaa21de39f733f2fe850d"} Jul 10 16:03:06.521776 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:06.521745 2560 generic.go:356] "Generic (PLEG): container finished" podID="15ec01c2-78da-4669-a2ee-927426aea148" containerID="b1f7e5ee5380ec956e89cd01da16d3835fc19b9ebb54cc47316cb43d5665aeb3" exitCode=0 Jul 10 16:03:06.521880 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:06.521779 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd" event={"ID":"15ec01c2-78da-4669-a2ee-927426aea148","Type":"ContainerDied","Data":"b1f7e5ee5380ec956e89cd01da16d3835fc19b9ebb54cc47316cb43d5665aeb3"} Jul 10 16:03:06.521880 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:06.521822 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd" event={"ID":"15ec01c2-78da-4669-a2ee-927426aea148","Type":"ContainerStarted","Data":"c3535baee35ae2eedbdf81e22909ec1cd3dad3c2ec97b0b5fa9f864bd4a39e4f"} Jul 10 16:03:06.523506 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:06.523474 2560 generic.go:356] "Generic (PLEG): container finished" podID="1b4821b8-2b1a-4d1a-aa0c-d21554082e85" containerID="7f6f7a7ebc74c4d0d040f3cf7409e559d04a504e3aa8e8745cb9d0822b5d8bc2" exitCode=0 Jul 10 16:03:06.523580 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:06.523533 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5" event={"ID":"1b4821b8-2b1a-4d1a-aa0c-d21554082e85","Type":"ContainerDied","Data":"7f6f7a7ebc74c4d0d040f3cf7409e559d04a504e3aa8e8745cb9d0822b5d8bc2"} Jul 10 16:03:07.533184 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:07.533154 2560 generic.go:356] "Generic (PLEG): container finished" podID="dd82ea12-c8d5-4256-bcdb-cd937fe07abd" containerID="f6dd9910f3b5b027e8a06044b7492ffe58232d363e04e66dc4f8dd56ac9067e4" exitCode=0 Jul 10 16:03:07.533569 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:07.533240 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh" event={"ID":"dd82ea12-c8d5-4256-bcdb-cd937fe07abd","Type":"ContainerDied","Data":"f6dd9910f3b5b027e8a06044b7492ffe58232d363e04e66dc4f8dd56ac9067e4"} Jul 10 16:03:07.534788 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:07.534769 2560 generic.go:356] "Generic (PLEG): container finished" podID="15ec01c2-78da-4669-a2ee-927426aea148" containerID="31358f23d9c8c3bc03af037bed42b0b9f6558b6a24bbe255657418663e0b9d89" exitCode=0 Jul 10 16:03:07.534870 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:07.534793 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd" event={"ID":"15ec01c2-78da-4669-a2ee-927426aea148","Type":"ContainerDied","Data":"31358f23d9c8c3bc03af037bed42b0b9f6558b6a24bbe255657418663e0b9d89"} Jul 10 16:03:07.536617 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:07.536592 2560 generic.go:356] "Generic (PLEG): container finished" podID="1b4821b8-2b1a-4d1a-aa0c-d21554082e85" containerID="aac9c88776a01f51d49f2a518965c4f7cf3250d3389dae23c745338472a467d1" exitCode=0 Jul 10 16:03:07.536688 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:07.536665 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5" event={"ID":"1b4821b8-2b1a-4d1a-aa0c-d21554082e85","Type":"ContainerDied","Data":"aac9c88776a01f51d49f2a518965c4f7cf3250d3389dae23c745338472a467d1"} Jul 10 16:03:08.543560 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:08.543522 2560 generic.go:356] "Generic (PLEG): container finished" podID="1b4821b8-2b1a-4d1a-aa0c-d21554082e85" containerID="f9451843421997092ccfb9f5c2ed0ff6a9ea5c9fef228702750676566645f151" exitCode=0 Jul 10 16:03:08.544014 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:08.543640 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5" event={"ID":"1b4821b8-2b1a-4d1a-aa0c-d21554082e85","Type":"ContainerDied","Data":"f9451843421997092ccfb9f5c2ed0ff6a9ea5c9fef228702750676566645f151"} Jul 10 16:03:08.545237 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:08.545215 2560 generic.go:356] "Generic (PLEG): container finished" podID="a7a1f505-d99d-4fc9-99b4-078f5842fb95" containerID="b2c956d1d2704a8c4726d27b62e5fa675c6e563791988de1366fb15c53c65123" exitCode=0 Jul 10 16:03:08.545324 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:08.545296 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n" event={"ID":"a7a1f505-d99d-4fc9-99b4-078f5842fb95","Type":"ContainerDied","Data":"b2c956d1d2704a8c4726d27b62e5fa675c6e563791988de1366fb15c53c65123"} Jul 10 16:03:08.547263 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:08.547246 2560 generic.go:356] "Generic (PLEG): container finished" podID="15ec01c2-78da-4669-a2ee-927426aea148" containerID="9096d66f99b0e09f30e3f7517732b4a51aef4ea1c358b4bb9c97d79a44d5573c" exitCode=0 Jul 10 16:03:08.547354 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:08.547306 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd" event={"ID":"15ec01c2-78da-4669-a2ee-927426aea148","Type":"ContainerDied","Data":"9096d66f99b0e09f30e3f7517732b4a51aef4ea1c358b4bb9c97d79a44d5573c"} Jul 10 16:03:08.731633 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:08.731605 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh" Jul 10 16:03:08.823677 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:08.823611 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd82ea12-c8d5-4256-bcdb-cd937fe07abd-util\") pod \"dd82ea12-c8d5-4256-bcdb-cd937fe07abd\" (UID: \"dd82ea12-c8d5-4256-bcdb-cd937fe07abd\") " Jul 10 16:03:08.823807 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:08.823688 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd82ea12-c8d5-4256-bcdb-cd937fe07abd-bundle\") pod \"dd82ea12-c8d5-4256-bcdb-cd937fe07abd\" (UID: \"dd82ea12-c8d5-4256-bcdb-cd937fe07abd\") " Jul 10 16:03:08.823807 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:08.823736 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzfr8\" (UniqueName: \"kubernetes.io/projected/dd82ea12-c8d5-4256-bcdb-cd937fe07abd-kube-api-access-mzfr8\") pod \"dd82ea12-c8d5-4256-bcdb-cd937fe07abd\" (UID: \"dd82ea12-c8d5-4256-bcdb-cd937fe07abd\") " Jul 10 16:03:08.824158 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:08.824139 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd82ea12-c8d5-4256-bcdb-cd937fe07abd-bundle" (OuterVolumeSpecName: "bundle") pod "dd82ea12-c8d5-4256-bcdb-cd937fe07abd" (UID: "dd82ea12-c8d5-4256-bcdb-cd937fe07abd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:03:08.825576 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:08.825554 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd82ea12-c8d5-4256-bcdb-cd937fe07abd-kube-api-access-mzfr8" (OuterVolumeSpecName: "kube-api-access-mzfr8") pod "dd82ea12-c8d5-4256-bcdb-cd937fe07abd" (UID: "dd82ea12-c8d5-4256-bcdb-cd937fe07abd"). InnerVolumeSpecName "kube-api-access-mzfr8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:03:08.829071 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:08.829049 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd82ea12-c8d5-4256-bcdb-cd937fe07abd-util" (OuterVolumeSpecName: "util") pod "dd82ea12-c8d5-4256-bcdb-cd937fe07abd" (UID: "dd82ea12-c8d5-4256-bcdb-cd937fe07abd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:03:08.924672 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:08.924640 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzfr8\" (UniqueName: \"kubernetes.io/projected/dd82ea12-c8d5-4256-bcdb-cd937fe07abd-kube-api-access-mzfr8\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:03:08.924672 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:08.924665 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dd82ea12-c8d5-4256-bcdb-cd937fe07abd-util\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:03:08.924672 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:08.924674 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dd82ea12-c8d5-4256-bcdb-cd937fe07abd-bundle\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:03:09.554270 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.554186 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh" event={"ID":"dd82ea12-c8d5-4256-bcdb-cd937fe07abd","Type":"ContainerDied","Data":"0e0e61697dfb18b95e4ee9f160d1190b54cceaf077afd46e5301c411e758890b"} Jul 10 16:03:09.554270 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.554223 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh" Jul 10 16:03:09.554270 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.554234 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e0e61697dfb18b95e4ee9f160d1190b54cceaf077afd46e5301c411e758890b" Jul 10 16:03:09.556252 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.556228 2560 generic.go:356] "Generic (PLEG): container finished" podID="a7a1f505-d99d-4fc9-99b4-078f5842fb95" containerID="c8b6f08f4ad0ec1cedd42f48234a94c37259b07b230996910eb2c2480fa4a19f" exitCode=0 Jul 10 16:03:09.556354 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.556308 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n" event={"ID":"a7a1f505-d99d-4fc9-99b4-078f5842fb95","Type":"ContainerDied","Data":"c8b6f08f4ad0ec1cedd42f48234a94c37259b07b230996910eb2c2480fa4a19f"} Jul 10 16:03:09.689387 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.689365 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5" Jul 10 16:03:09.714944 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.714922 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd" Jul 10 16:03:09.833094 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.833015 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/15ec01c2-78da-4669-a2ee-927426aea148-util\") pod \"15ec01c2-78da-4669-a2ee-927426aea148\" (UID: \"15ec01c2-78da-4669-a2ee-927426aea148\") " Jul 10 16:03:09.833094 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.833073 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b4821b8-2b1a-4d1a-aa0c-d21554082e85-util\") pod \"1b4821b8-2b1a-4d1a-aa0c-d21554082e85\" (UID: \"1b4821b8-2b1a-4d1a-aa0c-d21554082e85\") " Jul 10 16:03:09.833094 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.833093 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9qtjk\" (UniqueName: \"kubernetes.io/projected/15ec01c2-78da-4669-a2ee-927426aea148-kube-api-access-9qtjk\") pod \"15ec01c2-78da-4669-a2ee-927426aea148\" (UID: \"15ec01c2-78da-4669-a2ee-927426aea148\") " Jul 10 16:03:09.833359 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.833116 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/15ec01c2-78da-4669-a2ee-927426aea148-bundle\") pod \"15ec01c2-78da-4669-a2ee-927426aea148\" (UID: \"15ec01c2-78da-4669-a2ee-927426aea148\") " Jul 10 16:03:09.833359 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.833133 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b4821b8-2b1a-4d1a-aa0c-d21554082e85-bundle\") pod \"1b4821b8-2b1a-4d1a-aa0c-d21554082e85\" (UID: \"1b4821b8-2b1a-4d1a-aa0c-d21554082e85\") " Jul 10 16:03:09.833359 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.833242 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2875\" (UniqueName: \"kubernetes.io/projected/1b4821b8-2b1a-4d1a-aa0c-d21554082e85-kube-api-access-n2875\") pod \"1b4821b8-2b1a-4d1a-aa0c-d21554082e85\" (UID: \"1b4821b8-2b1a-4d1a-aa0c-d21554082e85\") " Jul 10 16:03:09.833703 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.833673 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b4821b8-2b1a-4d1a-aa0c-d21554082e85-bundle" (OuterVolumeSpecName: "bundle") pod "1b4821b8-2b1a-4d1a-aa0c-d21554082e85" (UID: "1b4821b8-2b1a-4d1a-aa0c-d21554082e85"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:03:09.833795 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.833772 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15ec01c2-78da-4669-a2ee-927426aea148-bundle" (OuterVolumeSpecName: "bundle") pod "15ec01c2-78da-4669-a2ee-927426aea148" (UID: "15ec01c2-78da-4669-a2ee-927426aea148"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:03:09.835366 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.835344 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b4821b8-2b1a-4d1a-aa0c-d21554082e85-kube-api-access-n2875" (OuterVolumeSpecName: "kube-api-access-n2875") pod "1b4821b8-2b1a-4d1a-aa0c-d21554082e85" (UID: "1b4821b8-2b1a-4d1a-aa0c-d21554082e85"). InnerVolumeSpecName "kube-api-access-n2875". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:03:09.835491 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.835472 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15ec01c2-78da-4669-a2ee-927426aea148-kube-api-access-9qtjk" (OuterVolumeSpecName: "kube-api-access-9qtjk") pod "15ec01c2-78da-4669-a2ee-927426aea148" (UID: "15ec01c2-78da-4669-a2ee-927426aea148"). InnerVolumeSpecName "kube-api-access-9qtjk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:03:09.841406 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.841370 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15ec01c2-78da-4669-a2ee-927426aea148-util" (OuterVolumeSpecName: "util") pod "15ec01c2-78da-4669-a2ee-927426aea148" (UID: "15ec01c2-78da-4669-a2ee-927426aea148"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:03:09.841890 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.841872 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b4821b8-2b1a-4d1a-aa0c-d21554082e85-util" (OuterVolumeSpecName: "util") pod "1b4821b8-2b1a-4d1a-aa0c-d21554082e85" (UID: "1b4821b8-2b1a-4d1a-aa0c-d21554082e85"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:03:09.934179 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.934147 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b4821b8-2b1a-4d1a-aa0c-d21554082e85-util\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:03:09.934179 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.934176 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9qtjk\" (UniqueName: \"kubernetes.io/projected/15ec01c2-78da-4669-a2ee-927426aea148-kube-api-access-9qtjk\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:03:09.934366 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.934189 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/15ec01c2-78da-4669-a2ee-927426aea148-bundle\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:03:09.934366 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.934201 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b4821b8-2b1a-4d1a-aa0c-d21554082e85-bundle\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:03:09.934366 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.934214 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2875\" (UniqueName: \"kubernetes.io/projected/1b4821b8-2b1a-4d1a-aa0c-d21554082e85-kube-api-access-n2875\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:03:09.934366 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:09.934224 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/15ec01c2-78da-4669-a2ee-927426aea148-util\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:03:10.562679 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:10.562597 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd" Jul 10 16:03:10.562679 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:10.562628 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd" event={"ID":"15ec01c2-78da-4669-a2ee-927426aea148","Type":"ContainerDied","Data":"c3535baee35ae2eedbdf81e22909ec1cd3dad3c2ec97b0b5fa9f864bd4a39e4f"} Jul 10 16:03:10.562679 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:10.562667 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3535baee35ae2eedbdf81e22909ec1cd3dad3c2ec97b0b5fa9f864bd4a39e4f" Jul 10 16:03:10.564608 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:10.564591 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5" Jul 10 16:03:10.564757 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:10.564590 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5" event={"ID":"1b4821b8-2b1a-4d1a-aa0c-d21554082e85","Type":"ContainerDied","Data":"345f45821df3e2a06a943d2cf65a59d0e2236bb620f5599790ec7900c9a09133"} Jul 10 16:03:10.564757 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:10.564705 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="345f45821df3e2a06a943d2cf65a59d0e2236bb620f5599790ec7900c9a09133" Jul 10 16:03:10.696419 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:10.696397 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n" Jul 10 16:03:10.843270 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:10.843195 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7a1f505-d99d-4fc9-99b4-078f5842fb95-util\") pod \"a7a1f505-d99d-4fc9-99b4-078f5842fb95\" (UID: \"a7a1f505-d99d-4fc9-99b4-078f5842fb95\") " Jul 10 16:03:10.843438 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:10.843273 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7a1f505-d99d-4fc9-99b4-078f5842fb95-bundle\") pod \"a7a1f505-d99d-4fc9-99b4-078f5842fb95\" (UID: \"a7a1f505-d99d-4fc9-99b4-078f5842fb95\") " Jul 10 16:03:10.843438 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:10.843310 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ghgr\" (UniqueName: \"kubernetes.io/projected/a7a1f505-d99d-4fc9-99b4-078f5842fb95-kube-api-access-7ghgr\") pod \"a7a1f505-d99d-4fc9-99b4-078f5842fb95\" (UID: \"a7a1f505-d99d-4fc9-99b4-078f5842fb95\") " Jul 10 16:03:10.843717 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:10.843690 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7a1f505-d99d-4fc9-99b4-078f5842fb95-bundle" (OuterVolumeSpecName: "bundle") pod "a7a1f505-d99d-4fc9-99b4-078f5842fb95" (UID: "a7a1f505-d99d-4fc9-99b4-078f5842fb95"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:03:10.845318 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:10.845296 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7a1f505-d99d-4fc9-99b4-078f5842fb95-kube-api-access-7ghgr" (OuterVolumeSpecName: "kube-api-access-7ghgr") pod "a7a1f505-d99d-4fc9-99b4-078f5842fb95" (UID: "a7a1f505-d99d-4fc9-99b4-078f5842fb95"). InnerVolumeSpecName "kube-api-access-7ghgr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:03:10.849724 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:10.849700 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7a1f505-d99d-4fc9-99b4-078f5842fb95-util" (OuterVolumeSpecName: "util") pod "a7a1f505-d99d-4fc9-99b4-078f5842fb95" (UID: "a7a1f505-d99d-4fc9-99b4-078f5842fb95"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:03:10.944469 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:10.944439 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ghgr\" (UniqueName: \"kubernetes.io/projected/a7a1f505-d99d-4fc9-99b4-078f5842fb95-kube-api-access-7ghgr\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:03:10.944469 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:10.944460 2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a7a1f505-d99d-4fc9-99b4-078f5842fb95-util\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:03:10.944622 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:10.944469 2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a7a1f505-d99d-4fc9-99b4-078f5842fb95-bundle\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:03:11.570013 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:11.569987 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n" event={"ID":"a7a1f505-d99d-4fc9-99b4-078f5842fb95","Type":"ContainerDied","Data":"cd408c8fe312aa9121a477b5b44a97946096a2d8d4e53c490deac92481812d4d"} Jul 10 16:03:11.570013 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:11.570002 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n" Jul 10 16:03:11.570355 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:11.570028 2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd408c8fe312aa9121a477b5b44a97946096a2d8d4e53c490deac92481812d4d" Jul 10 16:03:22.304737 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.304704 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk"] Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.304991 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15ec01c2-78da-4669-a2ee-927426aea148" containerName="extract" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305001 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="15ec01c2-78da-4669-a2ee-927426aea148" containerName="extract" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305012 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b4821b8-2b1a-4d1a-aa0c-d21554082e85" containerName="util" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305017 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b4821b8-2b1a-4d1a-aa0c-d21554082e85" containerName="util" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305032 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15ec01c2-78da-4669-a2ee-927426aea148" containerName="pull" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305037 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="15ec01c2-78da-4669-a2ee-927426aea148" containerName="pull" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305042 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7a1f505-d99d-4fc9-99b4-078f5842fb95" containerName="pull" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305048 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7a1f505-d99d-4fc9-99b4-078f5842fb95" containerName="pull" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305054 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7a1f505-d99d-4fc9-99b4-078f5842fb95" containerName="extract" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305062 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7a1f505-d99d-4fc9-99b4-078f5842fb95" containerName="extract" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305072 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b4821b8-2b1a-4d1a-aa0c-d21554082e85" containerName="pull" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305076 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b4821b8-2b1a-4d1a-aa0c-d21554082e85" containerName="pull" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305082 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd82ea12-c8d5-4256-bcdb-cd937fe07abd" containerName="extract" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305086 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd82ea12-c8d5-4256-bcdb-cd937fe07abd" containerName="extract" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305094 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7a1f505-d99d-4fc9-99b4-078f5842fb95" containerName="util" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305099 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7a1f505-d99d-4fc9-99b4-078f5842fb95" containerName="util" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305106 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b4821b8-2b1a-4d1a-aa0c-d21554082e85" containerName="extract" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305111 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b4821b8-2b1a-4d1a-aa0c-d21554082e85" containerName="extract" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305117 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd82ea12-c8d5-4256-bcdb-cd937fe07abd" containerName="util" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305122 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd82ea12-c8d5-4256-bcdb-cd937fe07abd" containerName="util" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305127 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15ec01c2-78da-4669-a2ee-927426aea148" containerName="util" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305132 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="15ec01c2-78da-4669-a2ee-927426aea148" containerName="util" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305140 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd82ea12-c8d5-4256-bcdb-cd937fe07abd" containerName="pull" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305145 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd82ea12-c8d5-4256-bcdb-cd937fe07abd" containerName="pull" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305188 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd82ea12-c8d5-4256-bcdb-cd937fe07abd" containerName="extract" Jul 10 16:03:22.305183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305196 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7a1f505-d99d-4fc9-99b4-078f5842fb95" containerName="extract" Jul 10 16:03:22.305942 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305201 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b4821b8-2b1a-4d1a-aa0c-d21554082e85" containerName="extract" Jul 10 16:03:22.305942 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.305208 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="15ec01c2-78da-4669-a2ee-927426aea148" containerName="extract" Jul 10 16:03:22.310499 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.310480 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk" Jul 10 16:03:22.313582 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.313565 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-5fbnt\"" Jul 10 16:03:22.336440 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.336415 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk"] Jul 10 16:03:22.435461 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.435434 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbmn2\" (UniqueName: \"kubernetes.io/projected/bb92dff5-13f5-461e-af01-38cbf44b62b4-kube-api-access-kbmn2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cr2zk\" (UID: \"bb92dff5-13f5-461e-af01-38cbf44b62b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk" Jul 10 16:03:22.435615 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.435499 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb92dff5-13f5-461e-af01-38cbf44b62b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cr2zk\" (UID: \"bb92dff5-13f5-461e-af01-38cbf44b62b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk" Jul 10 16:03:22.535886 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.535852 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb92dff5-13f5-461e-af01-38cbf44b62b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cr2zk\" (UID: \"bb92dff5-13f5-461e-af01-38cbf44b62b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk" Jul 10 16:03:22.536065 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.535899 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbmn2\" (UniqueName: \"kubernetes.io/projected/bb92dff5-13f5-461e-af01-38cbf44b62b4-kube-api-access-kbmn2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cr2zk\" (UID: \"bb92dff5-13f5-461e-af01-38cbf44b62b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk" Jul 10 16:03:22.536230 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.536213 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb92dff5-13f5-461e-af01-38cbf44b62b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cr2zk\" (UID: \"bb92dff5-13f5-461e-af01-38cbf44b62b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk" Jul 10 16:03:22.557949 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.557880 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbmn2\" (UniqueName: \"kubernetes.io/projected/bb92dff5-13f5-461e-af01-38cbf44b62b4-kube-api-access-kbmn2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cr2zk\" (UID: \"bb92dff5-13f5-461e-af01-38cbf44b62b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk" Jul 10 16:03:22.619479 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.619449 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk" Jul 10 16:03:22.756600 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:22.756577 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk"] Jul 10 16:03:22.758484 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:03:22.758455 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb92dff5_13f5_461e_af01_38cbf44b62b4.slice/crio-27ea169524b4617c337b4c96d0b4a80b22f918df606106e385b367d556bb3883 WatchSource:0}: Error finding container 27ea169524b4617c337b4c96d0b4a80b22f918df606106e385b367d556bb3883: Status 404 returned error can't find the container with id 27ea169524b4617c337b4c96d0b4a80b22f918df606106e385b367d556bb3883 Jul 10 16:03:23.623840 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:23.623795 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk" event={"ID":"bb92dff5-13f5-461e-af01-38cbf44b62b4","Type":"ContainerStarted","Data":"27ea169524b4617c337b4c96d0b4a80b22f918df606106e385b367d556bb3883"} Jul 10 16:03:25.011301 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:25.011269 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-j54kl"] Jul 10 16:03:25.014551 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:25.014528 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-j54kl" Jul 10 16:03:25.017933 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:25.017913 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-59k47\"" Jul 10 16:03:25.035788 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:25.035767 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-j54kl"] Jul 10 16:03:25.157838 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:25.157805 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbqk8\" (UniqueName: \"kubernetes.io/projected/d08ee4ef-a529-4b04-af3e-33995581b84b-kube-api-access-vbqk8\") pod \"authorino-operator-75cc8d5cb7-j54kl\" (UID: \"d08ee4ef-a529-4b04-af3e-33995581b84b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-j54kl" Jul 10 16:03:25.258948 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:25.258910 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbqk8\" (UniqueName: \"kubernetes.io/projected/d08ee4ef-a529-4b04-af3e-33995581b84b-kube-api-access-vbqk8\") pod \"authorino-operator-75cc8d5cb7-j54kl\" (UID: \"d08ee4ef-a529-4b04-af3e-33995581b84b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-j54kl" Jul 10 16:03:25.272902 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:25.272837 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbqk8\" (UniqueName: \"kubernetes.io/projected/d08ee4ef-a529-4b04-af3e-33995581b84b-kube-api-access-vbqk8\") pod \"authorino-operator-75cc8d5cb7-j54kl\" (UID: \"d08ee4ef-a529-4b04-af3e-33995581b84b\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-j54kl" Jul 10 16:03:25.324716 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:25.324688 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-j54kl" Jul 10 16:03:25.502610 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:25.502580 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-j54kl"] Jul 10 16:03:25.505579 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:03:25.505546 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd08ee4ef_a529_4b04_af3e_33995581b84b.slice/crio-56c8898075cefec614fabfcb7a7bfe185cdf095d13f64b3a08f654e3e8a2fbdb WatchSource:0}: Error finding container 56c8898075cefec614fabfcb7a7bfe185cdf095d13f64b3a08f654e3e8a2fbdb: Status 404 returned error can't find the container with id 56c8898075cefec614fabfcb7a7bfe185cdf095d13f64b3a08f654e3e8a2fbdb Jul 10 16:03:25.634908 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:25.634834 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-j54kl" event={"ID":"d08ee4ef-a529-4b04-af3e-33995581b84b","Type":"ContainerStarted","Data":"56c8898075cefec614fabfcb7a7bfe185cdf095d13f64b3a08f654e3e8a2fbdb"} Jul 10 16:03:30.661802 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:30.661766 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-j54kl" event={"ID":"d08ee4ef-a529-4b04-af3e-33995581b84b","Type":"ContainerStarted","Data":"8500a579bbc1abbbe378295b1aa9f47cfb43735b09984f77b1b806bd9915d25b"} Jul 10 16:03:30.662267 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:30.661859 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-j54kl" Jul 10 16:03:30.663410 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:30.663390 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk" event={"ID":"bb92dff5-13f5-461e-af01-38cbf44b62b4","Type":"ContainerStarted","Data":"11619b5c6be3b8319c797362031c437ccc8ea31712a8323f8d01c209ba6eb130"} Jul 10 16:03:30.663524 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:30.663508 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk" Jul 10 16:03:30.709373 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:30.709329 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-j54kl" podStartSLOduration=2.2656105220000002 podStartE2EDuration="6.709314277s" totalImagesPullingTime="4.443703755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:03:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:03:30.707691041 +0000 UTC m=+499.679724417" watchObservedRunningTime="2026-07-10 16:03:30.709314277 +0000 UTC m=+499.681347651" Jul 10 16:03:30.782167 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:30.782109 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk" podStartSLOduration=1.628900273 podStartE2EDuration="8.782094572s" totalImagesPullingTime="7.153194299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:03:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:03:30.778731537 +0000 UTC m=+499.750764912" watchObservedRunningTime="2026-07-10 16:03:30.782094572 +0000 UTC m=+499.754127947" Jul 10 16:03:41.670068 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:41.670037 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-j54kl" Jul 10 16:03:41.670547 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:41.670104 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk" Jul 10 16:03:43.723547 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:43.723513 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk"] Jul 10 16:03:43.723974 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:43.723694 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk" podUID="bb92dff5-13f5-461e-af01-38cbf44b62b4" containerName="manager" containerID="cri-o://11619b5c6be3b8319c797362031c437ccc8ea31712a8323f8d01c209ba6eb130" gracePeriod=2 Jul 10 16:03:43.743030 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:43.742972 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk"] Jul 10 16:03:43.753062 ip-10-0-129-101 kubenswrapper[2560]: E0710 16:03:43.753030 2560 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bb92dff5-13f5-461e-af01-38cbf44b62b4\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-cr2zk\": pods \"kuadrant-operator-controller-manager-8684c9ff49-cr2zk\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk" Jul 10 16:03:43.775717 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:43.775693 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k"] Jul 10 16:03:43.776015 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:43.776002 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb92dff5-13f5-461e-af01-38cbf44b62b4" containerName="manager" Jul 10 16:03:43.776075 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:43.776017 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb92dff5-13f5-461e-af01-38cbf44b62b4" containerName="manager" Jul 10 16:03:43.776075 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:43.776074 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb92dff5-13f5-461e-af01-38cbf44b62b4" containerName="manager" Jul 10 16:03:43.778777 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:43.778761 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k" Jul 10 16:03:43.783611 ip-10-0-129-101 kubenswrapper[2560]: E0710 16:03:43.783586 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cr2zk\" is forbidden: User \"system:node:ip-10-0-129-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-101.ec2.internal' and this object" podUID="bb92dff5-13f5-461e-af01-38cbf44b62b4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk" Jul 10 16:03:43.803608 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:43.803575 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k"] Jul 10 16:03:43.914668 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:43.914644 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-th79k\" (UID: \"dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k" Jul 10 16:03:43.914817 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:43.914678 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbsbw\" (UniqueName: \"kubernetes.io/projected/dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475-kube-api-access-sbsbw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-th79k\" (UID: \"dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k" Jul 10 16:03:43.965555 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:43.965536 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk" Jul 10 16:03:43.968073 ip-10-0-129-101 kubenswrapper[2560]: E0710 16:03:43.968050 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cr2zk\" is forbidden: User \"system:node:ip-10-0-129-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-101.ec2.internal' and this object" podUID="bb92dff5-13f5-461e-af01-38cbf44b62b4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk" Jul 10 16:03:44.015544 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.015475 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-th79k\" (UID: \"dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k" Jul 10 16:03:44.015544 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.015509 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbsbw\" (UniqueName: \"kubernetes.io/projected/dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475-kube-api-access-sbsbw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-th79k\" (UID: \"dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k" Jul 10 16:03:44.015852 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.015832 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-th79k\" (UID: \"dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k" Jul 10 16:03:44.034172 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.034145 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbsbw\" (UniqueName: \"kubernetes.io/projected/dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475-kube-api-access-sbsbw\") pod \"kuadrant-operator-controller-manager-8684c9ff49-th79k\" (UID: \"dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k" Jul 10 16:03:44.114916 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.114889 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k" Jul 10 16:03:44.116028 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.116011 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb92dff5-13f5-461e-af01-38cbf44b62b4-extensions-socket-volume\") pod \"bb92dff5-13f5-461e-af01-38cbf44b62b4\" (UID: \"bb92dff5-13f5-461e-af01-38cbf44b62b4\") " Jul 10 16:03:44.116116 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.116105 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbmn2\" (UniqueName: \"kubernetes.io/projected/bb92dff5-13f5-461e-af01-38cbf44b62b4-kube-api-access-kbmn2\") pod \"bb92dff5-13f5-461e-af01-38cbf44b62b4\" (UID: \"bb92dff5-13f5-461e-af01-38cbf44b62b4\") " Jul 10 16:03:44.116432 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.116410 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb92dff5-13f5-461e-af01-38cbf44b62b4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bb92dff5-13f5-461e-af01-38cbf44b62b4" (UID: "bb92dff5-13f5-461e-af01-38cbf44b62b4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:03:44.118120 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.118098 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb92dff5-13f5-461e-af01-38cbf44b62b4-kube-api-access-kbmn2" (OuterVolumeSpecName: "kube-api-access-kbmn2") pod "bb92dff5-13f5-461e-af01-38cbf44b62b4" (UID: "bb92dff5-13f5-461e-af01-38cbf44b62b4"). InnerVolumeSpecName "kube-api-access-kbmn2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:03:44.217363 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.217331 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbmn2\" (UniqueName: \"kubernetes.io/projected/bb92dff5-13f5-461e-af01-38cbf44b62b4-kube-api-access-kbmn2\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:03:44.217478 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.217367 2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb92dff5-13f5-461e-af01-38cbf44b62b4-extensions-socket-volume\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:03:44.250101 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.250075 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k"] Jul 10 16:03:44.253197 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:03:44.253171 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc1d9d2b_e7ae_4ed5_8d81_2ba2fa684475.slice/crio-c41472ba01f25d2e5dca8edf341c87a245fa3648cb79c61bfb9a765e2946c43d WatchSource:0}: Error finding container c41472ba01f25d2e5dca8edf341c87a245fa3648cb79c61bfb9a765e2946c43d: Status 404 returned error can't find the container with id c41472ba01f25d2e5dca8edf341c87a245fa3648cb79c61bfb9a765e2946c43d Jul 10 16:03:44.696094 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.696014 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw"] Jul 10 16:03:44.699368 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.699352 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw" Jul 10 16:03:44.702048 ip-10-0-129-101 kubenswrapper[2560]: E0710 16:03:44.702016 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cr2zk\" is forbidden: User \"system:node:ip-10-0-129-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-101.ec2.internal' and this object" podUID="bb92dff5-13f5-461e-af01-38cbf44b62b4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk" Jul 10 16:03:44.726264 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.726234 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw"] Jul 10 16:03:44.735584 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.735556 2560 generic.go:356] "Generic (PLEG): container finished" podID="bb92dff5-13f5-461e-af01-38cbf44b62b4" containerID="11619b5c6be3b8319c797362031c437ccc8ea31712a8323f8d01c209ba6eb130" exitCode=0 Jul 10 16:03:44.735728 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.735612 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk" Jul 10 16:03:44.735728 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.735654 2560 scope.go:117] "RemoveContainer" containerID="11619b5c6be3b8319c797362031c437ccc8ea31712a8323f8d01c209ba6eb130" Jul 10 16:03:44.738013 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.737669 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k" event={"ID":"dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475","Type":"ContainerStarted","Data":"6cd4f979e32e638002764ed1eabc6a0968b8d5e08ab9befdf8453d8479e40965"} Jul 10 16:03:44.738013 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.737723 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k" event={"ID":"dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475","Type":"ContainerStarted","Data":"c41472ba01f25d2e5dca8edf341c87a245fa3648cb79c61bfb9a765e2946c43d"} Jul 10 16:03:44.738013 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.737784 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k" Jul 10 16:03:44.752519 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.752487 2560 scope.go:117] "RemoveContainer" containerID="11619b5c6be3b8319c797362031c437ccc8ea31712a8323f8d01c209ba6eb130" Jul 10 16:03:44.753031 ip-10-0-129-101 kubenswrapper[2560]: E0710 16:03:44.753004 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11619b5c6be3b8319c797362031c437ccc8ea31712a8323f8d01c209ba6eb130\": container with ID starting with 11619b5c6be3b8319c797362031c437ccc8ea31712a8323f8d01c209ba6eb130 not found: ID does not exist" containerID="11619b5c6be3b8319c797362031c437ccc8ea31712a8323f8d01c209ba6eb130" Jul 10 16:03:44.753120 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.753041 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11619b5c6be3b8319c797362031c437ccc8ea31712a8323f8d01c209ba6eb130"} err="failed to get container status \"11619b5c6be3b8319c797362031c437ccc8ea31712a8323f8d01c209ba6eb130\": rpc error: code = NotFound desc = could not find container \"11619b5c6be3b8319c797362031c437ccc8ea31712a8323f8d01c209ba6eb130\": container with ID starting with 11619b5c6be3b8319c797362031c437ccc8ea31712a8323f8d01c209ba6eb130 not found: ID does not exist" Jul 10 16:03:44.786594 ip-10-0-129-101 kubenswrapper[2560]: E0710 16:03:44.786563 2560 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cr2zk\" is forbidden: User \"system:node:ip-10-0-129-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-101.ec2.internal' and this object" podUID="bb92dff5-13f5-461e-af01-38cbf44b62b4" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cr2zk" Jul 10 16:03:44.786685 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.786612 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k" podStartSLOduration=1.786597634 podStartE2EDuration="1.786597634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:03:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:03:44.784353936 +0000 UTC m=+513.756387311" watchObservedRunningTime="2026-07-10 16:03:44.786597634 +0000 UTC m=+513.758631009" Jul 10 16:03:44.823273 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.823237 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsp2p\" (UniqueName: \"kubernetes.io/projected/c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb-kube-api-access-lsp2p\") pod \"kuadrant-operator-controller-manager-86878c769f-nz9kw\" (UID: \"c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw" Jul 10 16:03:44.823395 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.823292 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nz9kw\" (UID: \"c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw" Jul 10 16:03:44.924422 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.924389 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsp2p\" (UniqueName: \"kubernetes.io/projected/c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb-kube-api-access-lsp2p\") pod \"kuadrant-operator-controller-manager-86878c769f-nz9kw\" (UID: \"c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw" Jul 10 16:03:44.924593 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.924439 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nz9kw\" (UID: \"c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw" Jul 10 16:03:44.924770 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.924752 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nz9kw\" (UID: \"c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw" Jul 10 16:03:44.935836 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:44.935810 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsp2p\" (UniqueName: \"kubernetes.io/projected/c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb-kube-api-access-lsp2p\") pod \"kuadrant-operator-controller-manager-86878c769f-nz9kw\" (UID: \"c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw" Jul 10 16:03:45.009071 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:45.009033 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw" Jul 10 16:03:45.147579 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:45.147560 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw"] Jul 10 16:03:45.149823 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:03:45.149801 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5bb0b79_3a11_4017_89a8_3a5e4ee70bdb.slice/crio-d22d6c4a986ee716032e4e9432e1cf3e2a155d898ece5b3374ed527ea003b795 WatchSource:0}: Error finding container d22d6c4a986ee716032e4e9432e1cf3e2a155d898ece5b3374ed527ea003b795: Status 404 returned error can't find the container with id d22d6c4a986ee716032e4e9432e1cf3e2a155d898ece5b3374ed527ea003b795 Jul 10 16:03:45.551636 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:45.551601 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb92dff5-13f5-461e-af01-38cbf44b62b4" path="/var/lib/kubelet/pods/bb92dff5-13f5-461e-af01-38cbf44b62b4/volumes" Jul 10 16:03:45.744723 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:45.744667 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw" event={"ID":"c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb","Type":"ContainerStarted","Data":"40d6fd5b78acfdb670b49a3be91e4d5e11615f2306e02aa6d83be807e5bca394"} Jul 10 16:03:45.745343 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:45.744841 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw" Jul 10 16:03:45.745343 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:45.744873 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw" event={"ID":"c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb","Type":"ContainerStarted","Data":"d22d6c4a986ee716032e4e9432e1cf3e2a155d898ece5b3374ed527ea003b795"} Jul 10 16:03:45.771969 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:45.771910 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw" podStartSLOduration=1.771897118 podStartE2EDuration="1.771897118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:03:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:03:45.770667787 +0000 UTC m=+514.742701182" watchObservedRunningTime="2026-07-10 16:03:45.771897118 +0000 UTC m=+514.743930493" Jul 10 16:03:55.748323 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:55.748291 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k" Jul 10 16:03:56.753919 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:56.753887 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw" Jul 10 16:03:56.838194 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:56.838162 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k"] Jul 10 16:03:56.838384 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:56.838344 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k" podUID="dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475" containerName="manager" containerID="cri-o://6cd4f979e32e638002764ed1eabc6a0968b8d5e08ab9befdf8453d8479e40965" gracePeriod=10 Jul 10 16:03:57.081123 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:57.081097 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k" Jul 10 16:03:57.121218 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:57.121192 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbsbw\" (UniqueName: \"kubernetes.io/projected/dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475-kube-api-access-sbsbw\") pod \"dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475\" (UID: \"dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475\") " Jul 10 16:03:57.121376 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:57.121277 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475-extensions-socket-volume\") pod \"dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475\" (UID: \"dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475\") " Jul 10 16:03:57.121651 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:57.121630 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475" (UID: "dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:03:57.123115 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:57.123092 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475-kube-api-access-sbsbw" (OuterVolumeSpecName: "kube-api-access-sbsbw") pod "dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475" (UID: "dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475"). InnerVolumeSpecName "kube-api-access-sbsbw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:03:57.222032 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:57.222005 2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475-extensions-socket-volume\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:03:57.222032 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:57.222027 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbsbw\" (UniqueName: \"kubernetes.io/projected/dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475-kube-api-access-sbsbw\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:03:57.806096 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:57.806063 2560 generic.go:356] "Generic (PLEG): container finished" podID="dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475" containerID="6cd4f979e32e638002764ed1eabc6a0968b8d5e08ab9befdf8453d8479e40965" exitCode=0 Jul 10 16:03:57.806485 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:57.806112 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k" Jul 10 16:03:57.806485 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:57.806151 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k" event={"ID":"dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475","Type":"ContainerDied","Data":"6cd4f979e32e638002764ed1eabc6a0968b8d5e08ab9befdf8453d8479e40965"} Jul 10 16:03:57.806485 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:57.806204 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k" event={"ID":"dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475","Type":"ContainerDied","Data":"c41472ba01f25d2e5dca8edf341c87a245fa3648cb79c61bfb9a765e2946c43d"} Jul 10 16:03:57.806485 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:57.806227 2560 scope.go:117] "RemoveContainer" containerID="6cd4f979e32e638002764ed1eabc6a0968b8d5e08ab9befdf8453d8479e40965" Jul 10 16:03:57.814923 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:57.814902 2560 scope.go:117] "RemoveContainer" containerID="6cd4f979e32e638002764ed1eabc6a0968b8d5e08ab9befdf8453d8479e40965" Jul 10 16:03:57.815184 ip-10-0-129-101 kubenswrapper[2560]: E0710 16:03:57.815166 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cd4f979e32e638002764ed1eabc6a0968b8d5e08ab9befdf8453d8479e40965\": container with ID starting with 6cd4f979e32e638002764ed1eabc6a0968b8d5e08ab9befdf8453d8479e40965 not found: ID does not exist" containerID="6cd4f979e32e638002764ed1eabc6a0968b8d5e08ab9befdf8453d8479e40965" Jul 10 16:03:57.815250 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:57.815193 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cd4f979e32e638002764ed1eabc6a0968b8d5e08ab9befdf8453d8479e40965"} err="failed to get container status \"6cd4f979e32e638002764ed1eabc6a0968b8d5e08ab9befdf8453d8479e40965\": rpc error: code = NotFound desc = could not find container \"6cd4f979e32e638002764ed1eabc6a0968b8d5e08ab9befdf8453d8479e40965\": container with ID starting with 6cd4f979e32e638002764ed1eabc6a0968b8d5e08ab9befdf8453d8479e40965 not found: ID does not exist" Jul 10 16:03:57.832931 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:57.832906 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k"] Jul 10 16:03:57.847101 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:57.847081 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-th79k"] Jul 10 16:03:59.551901 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:03:59.551864 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475" path="/var/lib/kubelet/pods/dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475/volumes" Jul 10 16:04:15.872592 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:04:15.872560 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:04:15.873030 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:04:15.872886 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475" containerName="manager" Jul 10 16:04:15.873030 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:04:15.872897 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475" containerName="manager" Jul 10 16:04:15.873030 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:04:15.872944 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc1d9d2b-e7ae-4ed5-8d81-2ba2fa684475" containerName="manager" Jul 10 16:04:15.875190 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:04:15.875174 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-clg9r" Jul 10 16:04:15.880077 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:04:15.880049 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 10 16:04:15.880208 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:04:15.880167 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jbqvq\"" Jul 10 16:04:15.928087 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:04:15.928055 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:04:15.958515 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:04:15.958487 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:04:15.967766 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:04:15.967735 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9b007e32-760f-4ad5-8ed7-9f9676cc8178-config-file\") pod \"limitador-limitador-54d6d988d5-clg9r\" (UID: \"9b007e32-760f-4ad5-8ed7-9f9676cc8178\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-clg9r" Jul 10 16:04:15.967888 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:04:15.967836 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzbbr\" (UniqueName: \"kubernetes.io/projected/9b007e32-760f-4ad5-8ed7-9f9676cc8178-kube-api-access-zzbbr\") pod \"limitador-limitador-54d6d988d5-clg9r\" (UID: \"9b007e32-760f-4ad5-8ed7-9f9676cc8178\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-clg9r" Jul 10 16:04:16.068849 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:04:16.068813 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzbbr\" (UniqueName: \"kubernetes.io/projected/9b007e32-760f-4ad5-8ed7-9f9676cc8178-kube-api-access-zzbbr\") pod \"limitador-limitador-54d6d988d5-clg9r\" (UID: \"9b007e32-760f-4ad5-8ed7-9f9676cc8178\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-clg9r" Jul 10 16:04:16.069076 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:04:16.068860 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9b007e32-760f-4ad5-8ed7-9f9676cc8178-config-file\") pod \"limitador-limitador-54d6d988d5-clg9r\" (UID: \"9b007e32-760f-4ad5-8ed7-9f9676cc8178\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-clg9r" Jul 10 16:04:16.069465 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:04:16.069447 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9b007e32-760f-4ad5-8ed7-9f9676cc8178-config-file\") pod \"limitador-limitador-54d6d988d5-clg9r\" (UID: \"9b007e32-760f-4ad5-8ed7-9f9676cc8178\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-clg9r" Jul 10 16:04:16.082526 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:04:16.082505 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzbbr\" (UniqueName: \"kubernetes.io/projected/9b007e32-760f-4ad5-8ed7-9f9676cc8178-kube-api-access-zzbbr\") pod \"limitador-limitador-54d6d988d5-clg9r\" (UID: \"9b007e32-760f-4ad5-8ed7-9f9676cc8178\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-clg9r" Jul 10 16:04:16.185593 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:04:16.185520 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-clg9r" Jul 10 16:04:16.342608 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:04:16.342582 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:04:16.344107 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:04:16.344078 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b007e32_760f_4ad5_8ed7_9f9676cc8178.slice/crio-74df00a89e34eaa5b8e5d3e8c5cf20182b395711148b1b6c09447f6120921749 WatchSource:0}: Error finding container 74df00a89e34eaa5b8e5d3e8c5cf20182b395711148b1b6c09447f6120921749: Status 404 returned error can't find the container with id 74df00a89e34eaa5b8e5d3e8c5cf20182b395711148b1b6c09447f6120921749 Jul 10 16:04:16.899645 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:04:16.899609 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-clg9r" event={"ID":"9b007e32-760f-4ad5-8ed7-9f9676cc8178","Type":"ContainerStarted","Data":"74df00a89e34eaa5b8e5d3e8c5cf20182b395711148b1b6c09447f6120921749"} Jul 10 16:04:19.917189 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:04:19.917153 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-clg9r" event={"ID":"9b007e32-760f-4ad5-8ed7-9f9676cc8178","Type":"ContainerStarted","Data":"bd7c09af1edc9d3915f23c6e1c4d21ae1a68bf7e9ddf3adfb460f06343ded83b"} Jul 10 16:04:19.917640 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:04:19.917248 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-clg9r" Jul 10 16:04:19.959948 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:04:19.959903 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-clg9r" podStartSLOduration=2.282779463 podStartE2EDuration="4.95988679s" totalImagesPullingTime="2.677107327s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:04:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:04:19.958181031 +0000 UTC m=+548.930214426" watchObservedRunningTime="2026-07-10 16:04:19.95988679 +0000 UTC m=+548.931920164" Jul 10 16:04:30.922050 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:04:30.922018 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-clg9r" Jul 10 16:05:11.439066 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:05:11.439039 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6s2sn_a7a2569d-7313-49a4-b283-aa416f5fdd9e/ovn-acl-logging/0.log" Jul 10 16:05:11.440185 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:05:11.440158 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6s2sn_a7a2569d-7313-49a4-b283-aa416f5fdd9e/ovn-acl-logging/0.log" Jul 10 16:05:31.674426 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:05:31.674393 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:06:00.183275 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:00.183239 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:06:15.277291 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:15.277255 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:06:23.061632 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:23.061597 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:06:41.759187 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:41.759155 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:06:52.390077 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.390021 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg"] Jul 10 16:06:52.393377 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.393359 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:06:52.396619 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.396600 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-z2mp7\"" Jul 10 16:06:52.397681 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.397661 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 10 16:06:52.397778 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.397666 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 10 16:06:52.397778 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.397693 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 10 16:06:52.404228 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.404208 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg"] Jul 10 16:06:52.534196 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.534160 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/356e5127-839c-42a9-ba7f-fed13feea3fa-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tmzwg\" (UID: \"356e5127-839c-42a9-ba7f-fed13feea3fa\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:06:52.534373 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.534207 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/356e5127-839c-42a9-ba7f-fed13feea3fa-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tmzwg\" (UID: \"356e5127-839c-42a9-ba7f-fed13feea3fa\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:06:52.534373 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.534227 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xzsp\" (UniqueName: \"kubernetes.io/projected/356e5127-839c-42a9-ba7f-fed13feea3fa-kube-api-access-7xzsp\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tmzwg\" (UID: \"356e5127-839c-42a9-ba7f-fed13feea3fa\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:06:52.534373 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.534328 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/356e5127-839c-42a9-ba7f-fed13feea3fa-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tmzwg\" (UID: \"356e5127-839c-42a9-ba7f-fed13feea3fa\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:06:52.534373 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.534365 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/356e5127-839c-42a9-ba7f-fed13feea3fa-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tmzwg\" (UID: \"356e5127-839c-42a9-ba7f-fed13feea3fa\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:06:52.534528 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.534420 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/356e5127-839c-42a9-ba7f-fed13feea3fa-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tmzwg\" (UID: \"356e5127-839c-42a9-ba7f-fed13feea3fa\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:06:52.635334 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.635291 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/356e5127-839c-42a9-ba7f-fed13feea3fa-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tmzwg\" (UID: \"356e5127-839c-42a9-ba7f-fed13feea3fa\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:06:52.635532 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.635338 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/356e5127-839c-42a9-ba7f-fed13feea3fa-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tmzwg\" (UID: \"356e5127-839c-42a9-ba7f-fed13feea3fa\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:06:52.635532 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.635363 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xzsp\" (UniqueName: \"kubernetes.io/projected/356e5127-839c-42a9-ba7f-fed13feea3fa-kube-api-access-7xzsp\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tmzwg\" (UID: \"356e5127-839c-42a9-ba7f-fed13feea3fa\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:06:52.635532 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.635401 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/356e5127-839c-42a9-ba7f-fed13feea3fa-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tmzwg\" (UID: \"356e5127-839c-42a9-ba7f-fed13feea3fa\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:06:52.635532 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.635432 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/356e5127-839c-42a9-ba7f-fed13feea3fa-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tmzwg\" (UID: \"356e5127-839c-42a9-ba7f-fed13feea3fa\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:06:52.635532 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.635466 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/356e5127-839c-42a9-ba7f-fed13feea3fa-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tmzwg\" (UID: \"356e5127-839c-42a9-ba7f-fed13feea3fa\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:06:52.635799 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.635777 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/356e5127-839c-42a9-ba7f-fed13feea3fa-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tmzwg\" (UID: \"356e5127-839c-42a9-ba7f-fed13feea3fa\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:06:52.635912 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.635886 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/356e5127-839c-42a9-ba7f-fed13feea3fa-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tmzwg\" (UID: \"356e5127-839c-42a9-ba7f-fed13feea3fa\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:06:52.636134 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.636111 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/356e5127-839c-42a9-ba7f-fed13feea3fa-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tmzwg\" (UID: \"356e5127-839c-42a9-ba7f-fed13feea3fa\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:06:52.638073 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.638042 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/356e5127-839c-42a9-ba7f-fed13feea3fa-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tmzwg\" (UID: \"356e5127-839c-42a9-ba7f-fed13feea3fa\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:06:52.638486 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.638459 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/356e5127-839c-42a9-ba7f-fed13feea3fa-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tmzwg\" (UID: \"356e5127-839c-42a9-ba7f-fed13feea3fa\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:06:52.644373 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.644313 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xzsp\" (UniqueName: \"kubernetes.io/projected/356e5127-839c-42a9-ba7f-fed13feea3fa-kube-api-access-7xzsp\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-tmzwg\" (UID: \"356e5127-839c-42a9-ba7f-fed13feea3fa\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:06:52.703397 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.703357 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:06:52.848545 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.848517 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg"] Jul 10 16:06:52.850269 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:06:52.850244 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod356e5127_839c_42a9_ba7f_fed13feea3fa.slice/crio-6236b2aa4a866d3c0eb86d40e79ef3853bec24cfdf5878753f9c042be3152ad4 WatchSource:0}: Error finding container 6236b2aa4a866d3c0eb86d40e79ef3853bec24cfdf5878753f9c042be3152ad4: Status 404 returned error can't find the container with id 6236b2aa4a866d3c0eb86d40e79ef3853bec24cfdf5878753f9c042be3152ad4 Jul 10 16:06:52.851684 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:52.851663 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 16:06:53.647677 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:53.647639 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" event={"ID":"356e5127-839c-42a9-ba7f-fed13feea3fa","Type":"ContainerStarted","Data":"6236b2aa4a866d3c0eb86d40e79ef3853bec24cfdf5878753f9c042be3152ad4"} Jul 10 16:06:54.566830 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:54.566799 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:06:54.656016 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:54.655986 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" event={"ID":"356e5127-839c-42a9-ba7f-fed13feea3fa","Type":"ContainerStarted","Data":"f19e869277403e6d72d8375dc380a384f10284c7498c4c4464cb3273d2dbcf18"} Jul 10 16:06:54.679266 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:06:54.679216 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" podStartSLOduration=1.584392883 podStartE2EDuration="2.67920062s" totalImagesPullingTime="1.094807737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:06:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:06:54.676645195 +0000 UTC m=+703.648678570" watchObservedRunningTime="2026-07-10 16:06:54.67920062 +0000 UTC m=+703.651233995" Jul 10 16:07:02.704437 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:07:02.704407 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:07:02.704809 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:07:02.704487 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:07:02.717023 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:07:02.716995 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:07:03.710630 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:07:03.710561 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-tmzwg" Jul 10 16:07:06.063232 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:07:06.063186 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:08:44.598339 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:08:44.598261 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:08:54.276726 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:08:54.276696 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:09:05.963745 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:09:05.963709 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:09:13.661532 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:09:13.661498 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:09:24.823814 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:09:24.823780 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:09:52.464804 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:09:52.464770 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:10:04.079622 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:10:04.079547 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:10:11.469515 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:10:11.469490 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6s2sn_a7a2569d-7313-49a4-b283-aa416f5fdd9e/ovn-acl-logging/0.log" Jul 10 16:10:11.469926 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:10:11.469730 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6s2sn_a7a2569d-7313-49a4-b283-aa416f5fdd9e/ovn-acl-logging/0.log" Jul 10 16:10:44.670565 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:10:44.670513 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:10:54.759070 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:10:54.759026 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:11:05.081217 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:11:05.081181 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:11:13.668730 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:11:13.668697 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:11:25.168415 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:11:25.168378 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:11:34.460070 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:11:34.459991 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:11:46.193149 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:11:46.193107 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:11:56.265907 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:11:56.265875 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:12:18.558497 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:12:18.558459 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:12:28.672168 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:12:28.672137 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:12:53.762909 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:12:53.762877 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:13:03.757895 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:13:03.757818 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:13:20.756048 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:13:20.756012 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:13:29.768330 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:13:29.768296 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:13:41.552423 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:13:41.552393 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:13:51.148578 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:13:51.148542 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:14:00.358005 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:14:00.357971 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:14:25.989227 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:14:25.989195 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw"] Jul 10 16:14:25.989881 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:14:25.989424 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw" podUID="c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb" containerName="manager" containerID="cri-o://40d6fd5b78acfdb670b49a3be91e4d5e11615f2306e02aa6d83be807e5bca394" gracePeriod=10 Jul 10 16:14:29.440794 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:14:29.440770 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw" Jul 10 16:14:29.560240 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:14:29.560209 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsp2p\" (UniqueName: \"kubernetes.io/projected/c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb-kube-api-access-lsp2p\") pod \"c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb\" (UID: \"c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb\") " Jul 10 16:14:29.560430 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:14:29.560314 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb-extensions-socket-volume\") pod \"c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb\" (UID: \"c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb\") " Jul 10 16:14:29.560672 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:14:29.560650 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb" (UID: "c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:14:29.562303 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:14:29.562281 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb-kube-api-access-lsp2p" (OuterVolumeSpecName: "kube-api-access-lsp2p") pod "c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb" (UID: "c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb"). InnerVolumeSpecName "kube-api-access-lsp2p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:14:29.661522 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:14:29.661479 2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb-extensions-socket-volume\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:14:29.661522 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:14:29.661512 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsp2p\" (UniqueName: \"kubernetes.io/projected/c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb-kube-api-access-lsp2p\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:14:29.847616 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:14:29.847529 2560 generic.go:356] "Generic (PLEG): container finished" podID="c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb" containerID="40d6fd5b78acfdb670b49a3be91e4d5e11615f2306e02aa6d83be807e5bca394" exitCode=0 Jul 10 16:14:29.847616 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:14:29.847573 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw" Jul 10 16:14:29.847847 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:14:29.847620 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw" event={"ID":"c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb","Type":"ContainerDied","Data":"40d6fd5b78acfdb670b49a3be91e4d5e11615f2306e02aa6d83be807e5bca394"} Jul 10 16:14:29.847847 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:14:29.847677 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw" event={"ID":"c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb","Type":"ContainerDied","Data":"d22d6c4a986ee716032e4e9432e1cf3e2a155d898ece5b3374ed527ea003b795"} Jul 10 16:14:29.847847 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:14:29.847709 2560 scope.go:117] "RemoveContainer" containerID="40d6fd5b78acfdb670b49a3be91e4d5e11615f2306e02aa6d83be807e5bca394" Jul 10 16:14:29.857220 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:14:29.857203 2560 scope.go:117] "RemoveContainer" containerID="40d6fd5b78acfdb670b49a3be91e4d5e11615f2306e02aa6d83be807e5bca394" Jul 10 16:14:29.857521 ip-10-0-129-101 kubenswrapper[2560]: E0710 16:14:29.857503 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40d6fd5b78acfdb670b49a3be91e4d5e11615f2306e02aa6d83be807e5bca394\": container with ID starting with 40d6fd5b78acfdb670b49a3be91e4d5e11615f2306e02aa6d83be807e5bca394 not found: ID does not exist" containerID="40d6fd5b78acfdb670b49a3be91e4d5e11615f2306e02aa6d83be807e5bca394" Jul 10 16:14:29.857579 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:14:29.857531 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40d6fd5b78acfdb670b49a3be91e4d5e11615f2306e02aa6d83be807e5bca394"} err="failed to get container status \"40d6fd5b78acfdb670b49a3be91e4d5e11615f2306e02aa6d83be807e5bca394\": rpc error: code = NotFound desc = could not find container \"40d6fd5b78acfdb670b49a3be91e4d5e11615f2306e02aa6d83be807e5bca394\": container with ID starting with 40d6fd5b78acfdb670b49a3be91e4d5e11615f2306e02aa6d83be807e5bca394 not found: ID does not exist" Jul 10 16:14:29.871791 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:14:29.871761 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw"] Jul 10 16:14:29.875358 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:14:29.875332 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nz9kw"] Jul 10 16:14:31.551519 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:14:31.551488 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb" path="/var/lib/kubelet/pods/c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb/volumes" Jul 10 16:15:00.150223 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:00.150189 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728335-xcnvk"] Jul 10 16:15:00.150702 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:00.150507 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb" containerName="manager" Jul 10 16:15:00.150702 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:00.150519 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb" containerName="manager" Jul 10 16:15:00.150702 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:00.150575 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5bb0b79-3a11-4017-89a8-3a5e4ee70bdb" containerName="manager" Jul 10 16:15:00.153433 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:00.153418 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728335-xcnvk" Jul 10 16:15:00.156509 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:00.156487 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 10 16:15:00.157581 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:00.157561 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nzzhf\"" Jul 10 16:15:00.157698 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:00.157566 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 10 16:15:00.162759 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:00.162733 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728335-xcnvk"] Jul 10 16:15:00.307521 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:00.307483 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnxk9\" (UniqueName: \"kubernetes.io/projected/0638d95b-e068-4726-a2ba-d2a2d3d7c4da-kube-api-access-lnxk9\") pod \"maas-api-key-cleanup-29728335-xcnvk\" (UID: \"0638d95b-e068-4726-a2ba-d2a2d3d7c4da\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728335-xcnvk" Jul 10 16:15:00.408626 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:00.408548 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnxk9\" (UniqueName: \"kubernetes.io/projected/0638d95b-e068-4726-a2ba-d2a2d3d7c4da-kube-api-access-lnxk9\") pod \"maas-api-key-cleanup-29728335-xcnvk\" (UID: \"0638d95b-e068-4726-a2ba-d2a2d3d7c4da\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728335-xcnvk" Jul 10 16:15:00.417037 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:00.416987 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnxk9\" (UniqueName: \"kubernetes.io/projected/0638d95b-e068-4726-a2ba-d2a2d3d7c4da-kube-api-access-lnxk9\") pod \"maas-api-key-cleanup-29728335-xcnvk\" (UID: \"0638d95b-e068-4726-a2ba-d2a2d3d7c4da\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728335-xcnvk" Jul 10 16:15:00.464801 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:00.464762 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728335-xcnvk" Jul 10 16:15:00.599983 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:00.599933 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728335-xcnvk"] Jul 10 16:15:00.603295 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:15:00.603262 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0638d95b_e068_4726_a2ba_d2a2d3d7c4da.slice/crio-e9b2c99aeb24953f2d315c38daa4096ae6a31ab8935c3f2199eb3f4d3d87592f WatchSource:0}: Error finding container e9b2c99aeb24953f2d315c38daa4096ae6a31ab8935c3f2199eb3f4d3d87592f: Status 404 returned error can't find the container with id e9b2c99aeb24953f2d315c38daa4096ae6a31ab8935c3f2199eb3f4d3d87592f Jul 10 16:15:00.605261 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:00.605242 2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 10 16:15:01.000128 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:01.000094 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728335-xcnvk" event={"ID":"0638d95b-e068-4726-a2ba-d2a2d3d7c4da","Type":"ContainerStarted","Data":"e9b2c99aeb24953f2d315c38daa4096ae6a31ab8935c3f2199eb3f4d3d87592f"} Jul 10 16:15:04.015749 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:04.015711 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728335-xcnvk" event={"ID":"0638d95b-e068-4726-a2ba-d2a2d3d7c4da","Type":"ContainerStarted","Data":"43d7a2db726f032b57e027ac1b0f7aa6f12d858be1df982ac307008f336247d5"} Jul 10 16:15:04.031253 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:04.031196 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728335-xcnvk" podStartSLOduration=1.874850001 podStartE2EDuration="4.031174272s" totalImagesPullingTime="2.156324271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:15:04.030706123 +0000 UTC m=+1193.002739498" watchObservedRunningTime="2026-07-10 16:15:04.031174272 +0000 UTC m=+1193.003207670" Jul 10 16:15:11.500545 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:11.500517 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6s2sn_a7a2569d-7313-49a4-b283-aa416f5fdd9e/ovn-acl-logging/0.log" Jul 10 16:15:11.502379 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:11.502358 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6s2sn_a7a2569d-7313-49a4-b283-aa416f5fdd9e/ovn-acl-logging/0.log" Jul 10 16:15:27.294408 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:27.294363 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8g7ms"] Jul 10 16:15:27.298751 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:27.298735 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8g7ms" Jul 10 16:15:27.303070 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:27.303045 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-5fbnt\"" Jul 10 16:15:27.309846 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:27.309824 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8283651d-1e04-494d-8f91-8bf92c2784e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8g7ms\" (UID: \"8283651d-1e04-494d-8f91-8bf92c2784e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8g7ms" Jul 10 16:15:27.309940 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:27.309853 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khqhb\" (UniqueName: \"kubernetes.io/projected/8283651d-1e04-494d-8f91-8bf92c2784e8-kube-api-access-khqhb\") pod \"kuadrant-operator-controller-manager-86878c769f-8g7ms\" (UID: \"8283651d-1e04-494d-8f91-8bf92c2784e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8g7ms" Jul 10 16:15:27.325125 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:27.325099 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8g7ms"] Jul 10 16:15:27.410405 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:27.410367 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8283651d-1e04-494d-8f91-8bf92c2784e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8g7ms\" (UID: \"8283651d-1e04-494d-8f91-8bf92c2784e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8g7ms" Jul 10 16:15:27.410538 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:27.410409 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khqhb\" (UniqueName: \"kubernetes.io/projected/8283651d-1e04-494d-8f91-8bf92c2784e8-kube-api-access-khqhb\") pod \"kuadrant-operator-controller-manager-86878c769f-8g7ms\" (UID: \"8283651d-1e04-494d-8f91-8bf92c2784e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8g7ms" Jul 10 16:15:27.410732 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:27.410713 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8283651d-1e04-494d-8f91-8bf92c2784e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-8g7ms\" (UID: \"8283651d-1e04-494d-8f91-8bf92c2784e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8g7ms" Jul 10 16:15:27.426040 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:27.426014 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khqhb\" (UniqueName: \"kubernetes.io/projected/8283651d-1e04-494d-8f91-8bf92c2784e8-kube-api-access-khqhb\") pod \"kuadrant-operator-controller-manager-86878c769f-8g7ms\" (UID: \"8283651d-1e04-494d-8f91-8bf92c2784e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8g7ms" Jul 10 16:15:27.607855 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:27.607784 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8g7ms" Jul 10 16:15:27.757308 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:27.757287 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8g7ms"] Jul 10 16:15:27.759246 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:15:27.759220 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8283651d_1e04_494d_8f91_8bf92c2784e8.slice/crio-d017537508fc370961913bc9146fa84929a0a4689b219a83185b9b1ad2d9d435 WatchSource:0}: Error finding container d017537508fc370961913bc9146fa84929a0a4689b219a83185b9b1ad2d9d435: Status 404 returned error can't find the container with id d017537508fc370961913bc9146fa84929a0a4689b219a83185b9b1ad2d9d435 Jul 10 16:15:28.126858 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:28.126821 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8g7ms" event={"ID":"8283651d-1e04-494d-8f91-8bf92c2784e8","Type":"ContainerStarted","Data":"7e759f57cd3727dced31132c948972156cf55ab348bec5329066ffa298874c2e"} Jul 10 16:15:28.127133 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:28.126871 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8g7ms" event={"ID":"8283651d-1e04-494d-8f91-8bf92c2784e8","Type":"ContainerStarted","Data":"d017537508fc370961913bc9146fa84929a0a4689b219a83185b9b1ad2d9d435"} Jul 10 16:15:28.127133 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:28.126901 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8g7ms" Jul 10 16:15:28.155017 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:28.154969 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8g7ms" podStartSLOduration=1.154941115 podStartE2EDuration="1.154941115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:15:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:15:28.147422686 +0000 UTC m=+1217.119456062" watchObservedRunningTime="2026-07-10 16:15:28.154941115 +0000 UTC m=+1217.126974487" Jul 10 16:15:39.134368 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:39.134335 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-8g7ms" Jul 10 16:15:48.557180 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:48.557153 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:15:57.356549 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:15:57.356517 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:16:00.012598 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:16:00.012560 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728335-xcnvk"] Jul 10 16:16:00.013010 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:16:00.012781 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728335-xcnvk" podUID="0638d95b-e068-4726-a2ba-d2a2d3d7c4da" containerName="cleanup" containerID="cri-o://43d7a2db726f032b57e027ac1b0f7aa6f12d858be1df982ac307008f336247d5" gracePeriod=30 Jul 10 16:16:09.862606 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:16:09.862530 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:16:18.754327 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:16:18.754287 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:16:28.156395 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:16:28.156362 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:16:30.180384 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:16:30.180357 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728335-xcnvk" Jul 10 16:16:30.223147 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:16:30.223115 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnxk9\" (UniqueName: \"kubernetes.io/projected/0638d95b-e068-4726-a2ba-d2a2d3d7c4da-kube-api-access-lnxk9\") pod \"0638d95b-e068-4726-a2ba-d2a2d3d7c4da\" (UID: \"0638d95b-e068-4726-a2ba-d2a2d3d7c4da\") " Jul 10 16:16:30.225178 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:16:30.225143 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0638d95b-e068-4726-a2ba-d2a2d3d7c4da-kube-api-access-lnxk9" (OuterVolumeSpecName: "kube-api-access-lnxk9") pod "0638d95b-e068-4726-a2ba-d2a2d3d7c4da" (UID: "0638d95b-e068-4726-a2ba-d2a2d3d7c4da"). InnerVolumeSpecName "kube-api-access-lnxk9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:16:30.323874 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:16:30.323792 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnxk9\" (UniqueName: \"kubernetes.io/projected/0638d95b-e068-4726-a2ba-d2a2d3d7c4da-kube-api-access-lnxk9\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:16:30.433735 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:16:30.433699 2560 generic.go:356] "Generic (PLEG): container finished" podID="0638d95b-e068-4726-a2ba-d2a2d3d7c4da" containerID="43d7a2db726f032b57e027ac1b0f7aa6f12d858be1df982ac307008f336247d5" exitCode=137 Jul 10 16:16:30.433896 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:16:30.433747 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728335-xcnvk" Jul 10 16:16:30.433896 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:16:30.433785 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728335-xcnvk" event={"ID":"0638d95b-e068-4726-a2ba-d2a2d3d7c4da","Type":"ContainerDied","Data":"43d7a2db726f032b57e027ac1b0f7aa6f12d858be1df982ac307008f336247d5"} Jul 10 16:16:30.433896 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:16:30.433841 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728335-xcnvk" event={"ID":"0638d95b-e068-4726-a2ba-d2a2d3d7c4da","Type":"ContainerDied","Data":"e9b2c99aeb24953f2d315c38daa4096ae6a31ab8935c3f2199eb3f4d3d87592f"} Jul 10 16:16:30.433896 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:16:30.433864 2560 scope.go:117] "RemoveContainer" containerID="43d7a2db726f032b57e027ac1b0f7aa6f12d858be1df982ac307008f336247d5" Jul 10 16:16:30.446461 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:16:30.446440 2560 scope.go:117] "RemoveContainer" containerID="43d7a2db726f032b57e027ac1b0f7aa6f12d858be1df982ac307008f336247d5" Jul 10 16:16:30.446711 ip-10-0-129-101 kubenswrapper[2560]: E0710 16:16:30.446691 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43d7a2db726f032b57e027ac1b0f7aa6f12d858be1df982ac307008f336247d5\": container with ID starting with 43d7a2db726f032b57e027ac1b0f7aa6f12d858be1df982ac307008f336247d5 not found: ID does not exist" containerID="43d7a2db726f032b57e027ac1b0f7aa6f12d858be1df982ac307008f336247d5" Jul 10 16:16:30.446778 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:16:30.446719 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43d7a2db726f032b57e027ac1b0f7aa6f12d858be1df982ac307008f336247d5"} err="failed to get container status \"43d7a2db726f032b57e027ac1b0f7aa6f12d858be1df982ac307008f336247d5\": rpc error: code = NotFound desc = could not find container \"43d7a2db726f032b57e027ac1b0f7aa6f12d858be1df982ac307008f336247d5\": container with ID starting with 43d7a2db726f032b57e027ac1b0f7aa6f12d858be1df982ac307008f336247d5 not found: ID does not exist" Jul 10 16:16:30.456851 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:16:30.456828 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728335-xcnvk"] Jul 10 16:16:30.460486 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:16:30.460467 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728335-xcnvk"] Jul 10 16:16:31.551354 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:16:31.551324 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0638d95b-e068-4726-a2ba-d2a2d3d7c4da" path="/var/lib/kubelet/pods/0638d95b-e068-4726-a2ba-d2a2d3d7c4da/volumes" Jul 10 16:16:37.403273 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:16:37.403239 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:16:48.554221 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:16:48.554188 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:16:59.162555 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:16:59.162520 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:17:07.761886 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:17:07.761854 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:17:18.354065 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:17:18.354028 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:17:30.658944 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:17:30.658906 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:17:38.144402 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:17:38.144318 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:17:56.542737 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:17:56.542701 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:18:05.460646 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:18:05.460614 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:18:15.858338 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:18:15.858303 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:18:28.047491 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:18:28.047453 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:18:43.551467 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:18:43.551439 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:18:54.741391 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:18:54.741351 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:19:03.348581 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:19:03.348496 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:19:12.749026 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:19:12.748991 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:19:23.151728 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:19:23.151692 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:19:32.148356 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:19:32.148321 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:19:45.653148 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:19:45.653115 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:19:54.853522 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:19:54.853485 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:20:05.855872 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:20:05.855838 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:20:11.532946 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:20:11.532916 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6s2sn_a7a2569d-7313-49a4-b283-aa416f5fdd9e/ovn-acl-logging/0.log" Jul 10 16:20:11.535170 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:20:11.535150 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6s2sn_a7a2569d-7313-49a4-b283-aa416f5fdd9e/ovn-acl-logging/0.log" Jul 10 16:20:16.146249 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:20:16.146214 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:20:27.745845 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:20:27.745808 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:20:38.054272 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:20:38.054199 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:20:48.148994 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:20:48.148945 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:20:59.450768 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:20:59.450738 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:21:09.845160 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:21:09.845125 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:21:20.744624 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:21:20.744589 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:21:29.839371 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:21:29.839334 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:21:38.848150 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:21:38.848115 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:25:11.562596 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:25:11.562561 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6s2sn_a7a2569d-7313-49a4-b283-aa416f5fdd9e/ovn-acl-logging/0.log" Jul 10 16:25:11.566491 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:25:11.566467 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6s2sn_a7a2569d-7313-49a4-b283-aa416f5fdd9e/ovn-acl-logging/0.log" Jul 10 16:27:49.068644 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.068615 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv"] Jul 10 16:27:49.069139 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.069120 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0638d95b-e068-4726-a2ba-d2a2d3d7c4da" containerName="cleanup" Jul 10 16:27:49.069187 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.069143 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0638d95b-e068-4726-a2ba-d2a2d3d7c4da" containerName="cleanup" Jul 10 16:27:49.069252 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.069240 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="0638d95b-e068-4726-a2ba-d2a2d3d7c4da" containerName="cleanup" Jul 10 16:27:49.072447 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.072425 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.075438 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.075420 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-68af1888-openshift-default-dockercfg-bj5lm\"" Jul 10 16:27:49.096361 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.096334 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv"] Jul 10 16:27:49.209103 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.209067 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-data\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.209274 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.209128 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psw7b\" (UniqueName: \"kubernetes.io/projected/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-kube-api-access-psw7b\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.209274 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.209195 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-envoy\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.209274 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.209268 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-workload-certs\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.209401 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.209292 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istiod-ca-cert\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.209401 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.209314 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-workload-socket\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.209401 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.209333 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-credential-socket\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.209401 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.209348 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-podinfo\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.209401 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.209369 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-token\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.309890 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.309853 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-workload-certs\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.309890 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.309890 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istiod-ca-cert\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.310169 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.309907 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-workload-socket\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.310169 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.309923 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-credential-socket\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.310169 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.309940 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-podinfo\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.310169 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.309981 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-token\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.310169 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.310008 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-data\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.310169 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.310075 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psw7b\" (UniqueName: \"kubernetes.io/projected/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-kube-api-access-psw7b\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.310169 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.310109 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-envoy\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.310540 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.310348 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-workload-socket\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.310540 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.310445 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-credential-socket\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.310540 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.310467 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-workload-certs\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.310680 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.310548 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-data\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.310819 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.310798 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istiod-ca-cert\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.312383 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.312361 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-envoy\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.312484 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.312416 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-podinfo\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.319246 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.319195 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psw7b\" (UniqueName: \"kubernetes.io/projected/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-kube-api-access-psw7b\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.319594 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.319569 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-token\") pod \"e2e-ait-68af1888-openshift-default-d685bd95-s5fpv\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.382759 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.382714 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:49.522431 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.522395 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv"] Jul 10 16:27:49.523454 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:27:49.523428 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbf74836_2bb1_442b_83aa_ee8a9d72c8d0.slice/crio-ffd77f691b92bc0bf27ffb909f6d9972bb70806e778f736b1bbe099ce25118d2 WatchSource:0}: Error finding container ffd77f691b92bc0bf27ffb909f6d9972bb70806e778f736b1bbe099ce25118d2: Status 404 returned error can't find the container with id ffd77f691b92bc0bf27ffb909f6d9972bb70806e778f736b1bbe099ce25118d2 Jul 10 16:27:49.525464 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.525431 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:27:49.525573 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.525494 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:27:49.525573 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.525521 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:27:49.729946 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.729914 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" event={"ID":"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0","Type":"ContainerStarted","Data":"321c5b226e8fc20c4f1c74608a66d2b31dbcc51d45dbad6f032ed1a27afb7622"} Jul 10 16:27:49.730145 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.729986 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" event={"ID":"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0","Type":"ContainerStarted","Data":"ffd77f691b92bc0bf27ffb909f6d9972bb70806e778f736b1bbe099ce25118d2"} Jul 10 16:27:49.751791 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:49.751741 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" podStartSLOduration=0.751725149 podStartE2EDuration="751.725149ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:27:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:27:49.749394661 +0000 UTC m=+1958.721428036" watchObservedRunningTime="2026-07-10 16:27:49.751725149 +0000 UTC m=+1958.723758524" Jul 10 16:27:50.383030 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:50.382996 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:50.388629 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:50.388603 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:50.736588 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:50.736548 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:50.737635 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:50.737603 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:27:55.593778 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:55.593737 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv"] Jul 10 16:27:55.594352 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:27:55.594010 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" podUID="dbf74836-2bb1-442b-83aa-ee8a9d72c8d0" containerName="istio-proxy" containerID="cri-o://321c5b226e8fc20c4f1c74608a66d2b31dbcc51d45dbad6f032ed1a27afb7622" gracePeriod=30 Jul 10 16:28:00.789466 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:00.788290 2560 generic.go:356] "Generic (PLEG): container finished" podID="dbf74836-2bb1-442b-83aa-ee8a9d72c8d0" containerID="321c5b226e8fc20c4f1c74608a66d2b31dbcc51d45dbad6f032ed1a27afb7622" exitCode=0 Jul 10 16:28:00.789466 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:00.788348 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" event={"ID":"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0","Type":"ContainerDied","Data":"321c5b226e8fc20c4f1c74608a66d2b31dbcc51d45dbad6f032ed1a27afb7622"} Jul 10 16:28:00.836836 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:00.836813 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:28:00.903926 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:00.903857 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-credential-socket\") pod \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " Jul 10 16:28:00.903926 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:00.903903 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-psw7b\" (UniqueName: \"kubernetes.io/projected/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-kube-api-access-psw7b\") pod \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " Jul 10 16:28:00.903926 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:00.903923 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-data\") pod \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " Jul 10 16:28:00.904223 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:00.903946 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-token\") pod \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " Jul 10 16:28:00.904223 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:00.904069 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-workload-certs\") pod \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " Jul 10 16:28:00.904223 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:00.904110 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-envoy\") pod \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " Jul 10 16:28:00.904223 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:00.904137 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-podinfo\") pod \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " Jul 10 16:28:00.904223 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:00.904157 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-workload-socket\") pod \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " Jul 10 16:28:00.904223 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:00.904177 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istiod-ca-cert\") pod \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\" (UID: \"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0\") " Jul 10 16:28:00.904223 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:00.904178 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dbf74836-2bb1-442b-83aa-ee8a9d72c8d0" (UID: "dbf74836-2bb1-442b-83aa-ee8a9d72c8d0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:28:00.904552 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:00.904321 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dbf74836-2bb1-442b-83aa-ee8a9d72c8d0" (UID: "dbf74836-2bb1-442b-83aa-ee8a9d72c8d0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:28:00.904552 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:00.904411 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-data" (OuterVolumeSpecName: "istio-data") pod "dbf74836-2bb1-442b-83aa-ee8a9d72c8d0" (UID: "dbf74836-2bb1-442b-83aa-ee8a9d72c8d0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:28:00.904552 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:00.904424 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-credential-socket\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:00.904552 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:00.904427 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dbf74836-2bb1-442b-83aa-ee8a9d72c8d0" (UID: "dbf74836-2bb1-442b-83aa-ee8a9d72c8d0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:28:00.904552 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:00.904446 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-workload-certs\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:00.904770 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:00.904607 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dbf74836-2bb1-442b-83aa-ee8a9d72c8d0" (UID: "dbf74836-2bb1-442b-83aa-ee8a9d72c8d0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 16:28:00.906159 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:00.906133 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dbf74836-2bb1-442b-83aa-ee8a9d72c8d0" (UID: "dbf74836-2bb1-442b-83aa-ee8a9d72c8d0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 16:28:00.906301 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:00.906283 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dbf74836-2bb1-442b-83aa-ee8a9d72c8d0" (UID: "dbf74836-2bb1-442b-83aa-ee8a9d72c8d0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:28:00.906462 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:00.906437 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-kube-api-access-psw7b" (OuterVolumeSpecName: "kube-api-access-psw7b") pod "dbf74836-2bb1-442b-83aa-ee8a9d72c8d0" (UID: "dbf74836-2bb1-442b-83aa-ee8a9d72c8d0"). InnerVolumeSpecName "kube-api-access-psw7b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:28:00.906530 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:00.906462 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-token" (OuterVolumeSpecName: "istio-token") pod "dbf74836-2bb1-442b-83aa-ee8a9d72c8d0" (UID: "dbf74836-2bb1-442b-83aa-ee8a9d72c8d0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:28:01.005316 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.005285 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-podinfo\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:01.005316 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.005311 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-workload-socket\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:01.005498 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.005324 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istiod-ca-cert\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:01.005498 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.005336 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psw7b\" (UniqueName: \"kubernetes.io/projected/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-kube-api-access-psw7b\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:01.005498 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.005349 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-data\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:01.005498 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.005359 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-token\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:01.005498 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.005369 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0-istio-envoy\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:01.501519 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.501483 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n"] Jul 10 16:28:01.502078 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.502053 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbf74836-2bb1-442b-83aa-ee8a9d72c8d0" containerName="istio-proxy" Jul 10 16:28:01.502210 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.502085 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbf74836-2bb1-442b-83aa-ee8a9d72c8d0" containerName="istio-proxy" Jul 10 16:28:01.502210 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.502191 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbf74836-2bb1-442b-83aa-ee8a9d72c8d0" containerName="istio-proxy" Jul 10 16:28:01.505829 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.505806 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.508787 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.508766 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-bcbb4aca-openshift-default-dockercfg-ghhcw\"" Jul 10 16:28:01.519303 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.519279 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n"] Jul 10 16:28:01.609242 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.609207 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-istio-envoy\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.609390 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.609260 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd45b\" (UniqueName: \"kubernetes.io/projected/45190c59-d158-4471-be24-73b1dc078bbc-kube-api-access-jd45b\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.609390 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.609372 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-istio-data\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.609472 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.609423 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45190c59-d158-4471-be24-73b1dc078bbc-istio-podinfo\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.609472 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.609449 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45190c59-d158-4471-be24-73b1dc078bbc-istiod-ca-cert\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.609556 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.609471 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-workload-socket\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.609556 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.609502 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45190c59-d158-4471-be24-73b1dc078bbc-istio-token\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.609556 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.609517 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-workload-certs\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.609556 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.609544 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-credential-socket\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.710152 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.710115 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-istio-data\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.710152 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.710156 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45190c59-d158-4471-be24-73b1dc078bbc-istio-podinfo\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.710394 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.710181 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45190c59-d158-4471-be24-73b1dc078bbc-istiod-ca-cert\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.710394 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.710203 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-workload-socket\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.710394 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.710242 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45190c59-d158-4471-be24-73b1dc078bbc-istio-token\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.710394 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.710265 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-workload-certs\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.710394 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.710339 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-credential-socket\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.710658 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.710401 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-istio-envoy\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.710658 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.710438 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd45b\" (UniqueName: \"kubernetes.io/projected/45190c59-d158-4471-be24-73b1dc078bbc-kube-api-access-jd45b\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.710658 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.710584 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-istio-data\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.710658 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.710600 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-workload-certs\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.710868 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.710671 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-workload-socket\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.710930 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.710879 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-credential-socket\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.711065 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.711046 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45190c59-d158-4471-be24-73b1dc078bbc-istiod-ca-cert\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.712630 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.712614 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-istio-envoy\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.712748 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.712732 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45190c59-d158-4471-be24-73b1dc078bbc-istio-podinfo\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.719649 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.719624 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd45b\" (UniqueName: \"kubernetes.io/projected/45190c59-d158-4471-be24-73b1dc078bbc-kube-api-access-jd45b\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.720112 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.720095 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45190c59-d158-4471-be24-73b1dc078bbc-istio-token\") pod \"e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.794192 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.794121 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" Jul 10 16:28:01.794576 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.794128 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv" event={"ID":"dbf74836-2bb1-442b-83aa-ee8a9d72c8d0","Type":"ContainerDied","Data":"ffd77f691b92bc0bf27ffb909f6d9972bb70806e778f736b1bbe099ce25118d2"} Jul 10 16:28:01.794576 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.794250 2560 scope.go:117] "RemoveContainer" containerID="321c5b226e8fc20c4f1c74608a66d2b31dbcc51d45dbad6f032ed1a27afb7622" Jul 10 16:28:01.815928 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.815904 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv"] Jul 10 16:28:01.819265 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.819245 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:01.822410 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.822389 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-68af1888-openshift-default-d685bd95-s5fpv"] Jul 10 16:28:01.952744 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.952719 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n"] Jul 10 16:28:01.953933 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:28:01.953893 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45190c59_d158_4471_be24_73b1dc078bbc.slice/crio-7a298c40c88b21691e809033967400fafc7ae720d6c14827dba8c24524b6ae53 WatchSource:0}: Error finding container 7a298c40c88b21691e809033967400fafc7ae720d6c14827dba8c24524b6ae53: Status 404 returned error can't find the container with id 7a298c40c88b21691e809033967400fafc7ae720d6c14827dba8c24524b6ae53 Jul 10 16:28:01.955932 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.955896 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:28:01.956047 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.955991 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:28:01.956047 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:01.956036 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:28:02.801144 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:02.801109 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" event={"ID":"45190c59-d158-4471-be24-73b1dc078bbc","Type":"ContainerStarted","Data":"5b92dae4b409831d6c24bac1c020c2d18029512c679f6b37a67b115c185423ac"} Jul 10 16:28:02.801570 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:02.801170 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" event={"ID":"45190c59-d158-4471-be24-73b1dc078bbc","Type":"ContainerStarted","Data":"7a298c40c88b21691e809033967400fafc7ae720d6c14827dba8c24524b6ae53"} Jul 10 16:28:02.820049 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:02.820024 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:02.828326 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:02.828283 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" podStartSLOduration=1.828269491 podStartE2EDuration="1.828269491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:28:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:28:02.826793884 +0000 UTC m=+1971.798827278" watchObservedRunningTime="2026-07-10 16:28:02.828269491 +0000 UTC m=+1971.800302934" Jul 10 16:28:03.552429 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:03.552395 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbf74836-2bb1-442b-83aa-ee8a9d72c8d0" path="/var/lib/kubelet/pods/dbf74836-2bb1-442b-83aa-ee8a9d72c8d0/volumes" Jul 10 16:28:03.824514 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:03.824447 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:04.811648 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:04.811613 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:04.812474 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:04.812458 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:05.250491 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:05.250448 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:28:08.902406 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:08.902364 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n"] Jul 10 16:28:08.902892 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:08.902608 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" podUID="45190c59-d158-4471-be24-73b1dc078bbc" containerName="istio-proxy" containerID="cri-o://5b92dae4b409831d6c24bac1c020c2d18029512c679f6b37a67b115c185423ac" gracePeriod=30 Jul 10 16:28:13.156035 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.155972 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:13.206340 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.206312 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-credential-socket\") pod \"45190c59-d158-4471-be24-73b1dc078bbc\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " Jul 10 16:28:13.206523 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.206359 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jd45b\" (UniqueName: \"kubernetes.io/projected/45190c59-d158-4471-be24-73b1dc078bbc-kube-api-access-jd45b\") pod \"45190c59-d158-4471-be24-73b1dc078bbc\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " Jul 10 16:28:13.206523 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.206391 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45190c59-d158-4471-be24-73b1dc078bbc-istiod-ca-cert\") pod \"45190c59-d158-4471-be24-73b1dc078bbc\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " Jul 10 16:28:13.206523 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.206408 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-istio-data\") pod \"45190c59-d158-4471-be24-73b1dc078bbc\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " Jul 10 16:28:13.206523 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.206456 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45190c59-d158-4471-be24-73b1dc078bbc-istio-token\") pod \"45190c59-d158-4471-be24-73b1dc078bbc\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " Jul 10 16:28:13.206523 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.206473 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-workload-socket\") pod \"45190c59-d158-4471-be24-73b1dc078bbc\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " Jul 10 16:28:13.206523 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.206513 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-istio-envoy\") pod \"45190c59-d158-4471-be24-73b1dc078bbc\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " Jul 10 16:28:13.206833 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.206546 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45190c59-d158-4471-be24-73b1dc078bbc-istio-podinfo\") pod \"45190c59-d158-4471-be24-73b1dc078bbc\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " Jul 10 16:28:13.206833 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.206573 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-workload-certs\") pod \"45190c59-d158-4471-be24-73b1dc078bbc\" (UID: \"45190c59-d158-4471-be24-73b1dc078bbc\") " Jul 10 16:28:13.206833 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.206648 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "45190c59-d158-4471-be24-73b1dc078bbc" (UID: "45190c59-d158-4471-be24-73b1dc078bbc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:28:13.206833 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.206784 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "45190c59-d158-4471-be24-73b1dc078bbc" (UID: "45190c59-d158-4471-be24-73b1dc078bbc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:28:13.207076 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.206902 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-credential-socket\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:13.207076 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.206902 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45190c59-d158-4471-be24-73b1dc078bbc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "45190c59-d158-4471-be24-73b1dc078bbc" (UID: "45190c59-d158-4471-be24-73b1dc078bbc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 16:28:13.207076 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.206919 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-workload-socket\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:13.207241 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.207147 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "45190c59-d158-4471-be24-73b1dc078bbc" (UID: "45190c59-d158-4471-be24-73b1dc078bbc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:28:13.207428 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.207404 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-istio-data" (OuterVolumeSpecName: "istio-data") pod "45190c59-d158-4471-be24-73b1dc078bbc" (UID: "45190c59-d158-4471-be24-73b1dc078bbc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:28:13.208786 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.208753 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45190c59-d158-4471-be24-73b1dc078bbc-istio-token" (OuterVolumeSpecName: "istio-token") pod "45190c59-d158-4471-be24-73b1dc078bbc" (UID: "45190c59-d158-4471-be24-73b1dc078bbc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:28:13.208877 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.208815 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "45190c59-d158-4471-be24-73b1dc078bbc" (UID: "45190c59-d158-4471-be24-73b1dc078bbc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:28:13.208877 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.208823 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45190c59-d158-4471-be24-73b1dc078bbc-kube-api-access-jd45b" (OuterVolumeSpecName: "kube-api-access-jd45b") pod "45190c59-d158-4471-be24-73b1dc078bbc" (UID: "45190c59-d158-4471-be24-73b1dc078bbc"). InnerVolumeSpecName "kube-api-access-jd45b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:28:13.208986 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.208935 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/45190c59-d158-4471-be24-73b1dc078bbc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "45190c59-d158-4471-be24-73b1dc078bbc" (UID: "45190c59-d158-4471-be24-73b1dc078bbc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 16:28:13.307868 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.307782 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jd45b\" (UniqueName: \"kubernetes.io/projected/45190c59-d158-4471-be24-73b1dc078bbc-kube-api-access-jd45b\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:13.307868 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.307821 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45190c59-d158-4471-be24-73b1dc078bbc-istiod-ca-cert\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:13.307868 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.307831 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-istio-data\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:13.307868 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.307839 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45190c59-d158-4471-be24-73b1dc078bbc-istio-token\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:13.307868 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.307847 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-istio-envoy\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:13.307868 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.307854 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45190c59-d158-4471-be24-73b1dc078bbc-istio-podinfo\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:13.307868 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.307861 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45190c59-d158-4471-be24-73b1dc078bbc-workload-certs\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:13.859241 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.859157 2560 generic.go:356] "Generic (PLEG): container finished" podID="45190c59-d158-4471-be24-73b1dc078bbc" containerID="5b92dae4b409831d6c24bac1c020c2d18029512c679f6b37a67b115c185423ac" exitCode=0 Jul 10 16:28:13.859241 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.859202 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" event={"ID":"45190c59-d158-4471-be24-73b1dc078bbc","Type":"ContainerDied","Data":"5b92dae4b409831d6c24bac1c020c2d18029512c679f6b37a67b115c185423ac"} Jul 10 16:28:13.859241 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.859208 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" Jul 10 16:28:13.859241 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.859242 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n" event={"ID":"45190c59-d158-4471-be24-73b1dc078bbc","Type":"ContainerDied","Data":"7a298c40c88b21691e809033967400fafc7ae720d6c14827dba8c24524b6ae53"} Jul 10 16:28:13.859527 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.859262 2560 scope.go:117] "RemoveContainer" containerID="5b92dae4b409831d6c24bac1c020c2d18029512c679f6b37a67b115c185423ac" Jul 10 16:28:13.867988 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.867975 2560 scope.go:117] "RemoveContainer" containerID="5b92dae4b409831d6c24bac1c020c2d18029512c679f6b37a67b115c185423ac" Jul 10 16:28:13.868220 ip-10-0-129-101 kubenswrapper[2560]: E0710 16:28:13.868203 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b92dae4b409831d6c24bac1c020c2d18029512c679f6b37a67b115c185423ac\": container with ID starting with 5b92dae4b409831d6c24bac1c020c2d18029512c679f6b37a67b115c185423ac not found: ID does not exist" containerID="5b92dae4b409831d6c24bac1c020c2d18029512c679f6b37a67b115c185423ac" Jul 10 16:28:13.868285 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.868234 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b92dae4b409831d6c24bac1c020c2d18029512c679f6b37a67b115c185423ac"} err="failed to get container status \"5b92dae4b409831d6c24bac1c020c2d18029512c679f6b37a67b115c185423ac\": rpc error: code = NotFound desc = could not find container \"5b92dae4b409831d6c24bac1c020c2d18029512c679f6b37a67b115c185423ac\": container with ID starting with 5b92dae4b409831d6c24bac1c020c2d18029512c679f6b37a67b115c185423ac not found: ID does not exist" Jul 10 16:28:13.879474 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.879448 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n"] Jul 10 16:28:13.883143 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:13.883123 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-bcbb4aca-openshift-default-c9ccd9cc9-rzt9n"] Jul 10 16:28:14.867784 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:14.867757 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5"] Jul 10 16:28:14.868190 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:14.868099 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45190c59-d158-4471-be24-73b1dc078bbc" containerName="istio-proxy" Jul 10 16:28:14.868190 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:14.868111 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="45190c59-d158-4471-be24-73b1dc078bbc" containerName="istio-proxy" Jul 10 16:28:14.868190 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:14.868169 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="45190c59-d158-4471-be24-73b1dc078bbc" containerName="istio-proxy" Jul 10 16:28:14.871283 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:14.871264 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:14.873996 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:14.873976 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-ffab410a-openshift-default-dockercfg-z446c\"" Jul 10 16:28:14.888453 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:14.888431 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5"] Jul 10 16:28:14.921068 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:14.921036 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb2033f3-705e-4008-a174-0c4cacf28449-istio-token\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:14.921635 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:14.921318 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-istio-envoy\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:14.921635 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:14.921353 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-workload-certs\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:14.921635 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:14.921374 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-istio-data\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:14.921635 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:14.921400 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-workload-socket\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:14.921635 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:14.921436 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-credential-socket\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:14.921635 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:14.921485 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz4fb\" (UniqueName: \"kubernetes.io/projected/cb2033f3-705e-4008-a174-0c4cacf28449-kube-api-access-tz4fb\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:14.921635 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:14.921510 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb2033f3-705e-4008-a174-0c4cacf28449-istio-podinfo\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:14.921635 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:14.921536 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb2033f3-705e-4008-a174-0c4cacf28449-istiod-ca-cert\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:15.022437 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.022395 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-istio-envoy\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:15.022437 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.022439 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-workload-certs\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:15.022699 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.022456 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-istio-data\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:15.022699 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.022474 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-workload-socket\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:15.022699 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.022506 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-credential-socket\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:15.022699 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.022536 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz4fb\" (UniqueName: \"kubernetes.io/projected/cb2033f3-705e-4008-a174-0c4cacf28449-kube-api-access-tz4fb\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:15.022699 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.022556 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb2033f3-705e-4008-a174-0c4cacf28449-istio-podinfo\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:15.022699 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.022573 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb2033f3-705e-4008-a174-0c4cacf28449-istiod-ca-cert\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:15.022699 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.022652 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb2033f3-705e-4008-a174-0c4cacf28449-istio-token\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:15.023048 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.022869 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-workload-certs\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:15.023114 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.023081 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-workload-socket\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:15.023176 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.023146 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-credential-socket\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:15.023176 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.023168 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-istio-data\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:15.023314 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.023297 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb2033f3-705e-4008-a174-0c4cacf28449-istiod-ca-cert\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:15.024818 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.024797 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-istio-envoy\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:15.025213 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.025191 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb2033f3-705e-4008-a174-0c4cacf28449-istio-podinfo\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:15.031775 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.031750 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz4fb\" (UniqueName: \"kubernetes.io/projected/cb2033f3-705e-4008-a174-0c4cacf28449-kube-api-access-tz4fb\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:15.031894 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.031875 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb2033f3-705e-4008-a174-0c4cacf28449-istio-token\") pod \"e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:15.182180 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.182096 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:15.322902 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.322876 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5"] Jul 10 16:28:15.324779 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:28:15.324755 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb2033f3_705e_4008_a174_0c4cacf28449.slice/crio-bf8fe5bba9314edf2036fb88837f682110908c0fe30e34e6619019194c66b034 WatchSource:0}: Error finding container bf8fe5bba9314edf2036fb88837f682110908c0fe30e34e6619019194c66b034: Status 404 returned error can't find the container with id bf8fe5bba9314edf2036fb88837f682110908c0fe30e34e6619019194c66b034 Jul 10 16:28:15.327222 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.327191 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:28:15.327301 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.327256 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:28:15.327301 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.327285 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:28:15.554452 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.554420 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45190c59-d158-4471-be24-73b1dc078bbc" path="/var/lib/kubelet/pods/45190c59-d158-4471-be24-73b1dc078bbc/volumes" Jul 10 16:28:15.873412 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.873329 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" event={"ID":"cb2033f3-705e-4008-a174-0c4cacf28449","Type":"ContainerStarted","Data":"d6e96065c4c2d734daef24908b5039e5884a7ebaedd8d72b2602ce139ad1b1a7"} Jul 10 16:28:15.873412 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.873395 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" event={"ID":"cb2033f3-705e-4008-a174-0c4cacf28449","Type":"ContainerStarted","Data":"bf8fe5bba9314edf2036fb88837f682110908c0fe30e34e6619019194c66b034"} Jul 10 16:28:15.901146 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:15.901096 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" podStartSLOduration=1.901081203 podStartE2EDuration="1.901081203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:28:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:28:15.900210035 +0000 UTC m=+1984.872243429" watchObservedRunningTime="2026-07-10 16:28:15.901081203 +0000 UTC m=+1984.873114579" Jul 10 16:28:16.182961 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:16.182861 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:16.187600 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:16.187580 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:16.754286 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:16.754257 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:28:16.878079 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:16.878051 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:16.879099 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:16.879078 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:20.736304 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:20.736271 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5"] Jul 10 16:28:20.736800 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:20.736440 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" podUID="cb2033f3-705e-4008-a174-0c4cacf28449" containerName="istio-proxy" containerID="cri-o://d6e96065c4c2d734daef24908b5039e5884a7ebaedd8d72b2602ce139ad1b1a7" gracePeriod=30 Jul 10 16:28:25.926118 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:25.926082 2560 generic.go:356] "Generic (PLEG): container finished" podID="cb2033f3-705e-4008-a174-0c4cacf28449" containerID="d6e96065c4c2d734daef24908b5039e5884a7ebaedd8d72b2602ce139ad1b1a7" exitCode=0 Jul 10 16:28:25.926434 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:25.926158 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" event={"ID":"cb2033f3-705e-4008-a174-0c4cacf28449","Type":"ContainerDied","Data":"d6e96065c4c2d734daef24908b5039e5884a7ebaedd8d72b2602ce139ad1b1a7"} Jul 10 16:28:26.005594 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.005538 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:26.119370 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.119343 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-workload-socket\") pod \"cb2033f3-705e-4008-a174-0c4cacf28449\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " Jul 10 16:28:26.119550 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.119377 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tz4fb\" (UniqueName: \"kubernetes.io/projected/cb2033f3-705e-4008-a174-0c4cacf28449-kube-api-access-tz4fb\") pod \"cb2033f3-705e-4008-a174-0c4cacf28449\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " Jul 10 16:28:26.119550 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.119396 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb2033f3-705e-4008-a174-0c4cacf28449-istio-podinfo\") pod \"cb2033f3-705e-4008-a174-0c4cacf28449\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " Jul 10 16:28:26.119550 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.119419 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-istio-data\") pod \"cb2033f3-705e-4008-a174-0c4cacf28449\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " Jul 10 16:28:26.119550 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.119446 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-workload-certs\") pod \"cb2033f3-705e-4008-a174-0c4cacf28449\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " Jul 10 16:28:26.119550 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.119510 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-istio-envoy\") pod \"cb2033f3-705e-4008-a174-0c4cacf28449\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " Jul 10 16:28:26.119825 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.119557 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb2033f3-705e-4008-a174-0c4cacf28449-istiod-ca-cert\") pod \"cb2033f3-705e-4008-a174-0c4cacf28449\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " Jul 10 16:28:26.119825 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.119616 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-credential-socket\") pod \"cb2033f3-705e-4008-a174-0c4cacf28449\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " Jul 10 16:28:26.119825 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.119628 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cb2033f3-705e-4008-a174-0c4cacf28449" (UID: "cb2033f3-705e-4008-a174-0c4cacf28449"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:28:26.119825 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.119642 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb2033f3-705e-4008-a174-0c4cacf28449-istio-token\") pod \"cb2033f3-705e-4008-a174-0c4cacf28449\" (UID: \"cb2033f3-705e-4008-a174-0c4cacf28449\") " Jul 10 16:28:26.119825 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.119758 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-istio-data" (OuterVolumeSpecName: "istio-data") pod "cb2033f3-705e-4008-a174-0c4cacf28449" (UID: "cb2033f3-705e-4008-a174-0c4cacf28449"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:28:26.119825 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.119798 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cb2033f3-705e-4008-a174-0c4cacf28449" (UID: "cb2033f3-705e-4008-a174-0c4cacf28449"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:28:26.120182 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.119894 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-istio-data\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:26.120182 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.119913 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-credential-socket\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:26.120182 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.119940 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb2033f3-705e-4008-a174-0c4cacf28449-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cb2033f3-705e-4008-a174-0c4cacf28449" (UID: "cb2033f3-705e-4008-a174-0c4cacf28449"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 16:28:26.120182 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.119926 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-workload-socket\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:26.120182 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.120051 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cb2033f3-705e-4008-a174-0c4cacf28449" (UID: "cb2033f3-705e-4008-a174-0c4cacf28449"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:28:26.121665 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.121635 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cb2033f3-705e-4008-a174-0c4cacf28449" (UID: "cb2033f3-705e-4008-a174-0c4cacf28449"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:28:26.121665 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.121661 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb2033f3-705e-4008-a174-0c4cacf28449-kube-api-access-tz4fb" (OuterVolumeSpecName: "kube-api-access-tz4fb") pod "cb2033f3-705e-4008-a174-0c4cacf28449" (UID: "cb2033f3-705e-4008-a174-0c4cacf28449"). InnerVolumeSpecName "kube-api-access-tz4fb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:28:26.121822 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.121661 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cb2033f3-705e-4008-a174-0c4cacf28449-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cb2033f3-705e-4008-a174-0c4cacf28449" (UID: "cb2033f3-705e-4008-a174-0c4cacf28449"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 16:28:26.121822 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.121794 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb2033f3-705e-4008-a174-0c4cacf28449-istio-token" (OuterVolumeSpecName: "istio-token") pod "cb2033f3-705e-4008-a174-0c4cacf28449" (UID: "cb2033f3-705e-4008-a174-0c4cacf28449"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:28:26.221350 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.221293 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-istio-envoy\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:26.221350 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.221331 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb2033f3-705e-4008-a174-0c4cacf28449-istiod-ca-cert\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:26.221350 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.221344 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb2033f3-705e-4008-a174-0c4cacf28449-istio-token\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:26.221350 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.221355 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tz4fb\" (UniqueName: \"kubernetes.io/projected/cb2033f3-705e-4008-a174-0c4cacf28449-kube-api-access-tz4fb\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:26.221350 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.221367 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb2033f3-705e-4008-a174-0c4cacf28449-istio-podinfo\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:26.221350 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.221379 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb2033f3-705e-4008-a174-0c4cacf28449-workload-certs\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:28:26.931718 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.931691 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" Jul 10 16:28:26.932262 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.931747 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5" event={"ID":"cb2033f3-705e-4008-a174-0c4cacf28449","Type":"ContainerDied","Data":"bf8fe5bba9314edf2036fb88837f682110908c0fe30e34e6619019194c66b034"} Jul 10 16:28:26.932262 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.931811 2560 scope.go:117] "RemoveContainer" containerID="d6e96065c4c2d734daef24908b5039e5884a7ebaedd8d72b2602ce139ad1b1a7" Jul 10 16:28:26.954239 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.954214 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5"] Jul 10 16:28:26.959386 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:26.959364 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-ffab410a-openshift-default-c964f97f-9g5f5"] Jul 10 16:28:27.552711 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:28:27.552680 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb2033f3-705e-4008-a174-0c4cacf28449" path="/var/lib/kubelet/pods/cb2033f3-705e-4008-a174-0c4cacf28449/volumes" Jul 10 16:29:55.003838 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:29:55.003809 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:30:00.138331 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:00.138297 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728350-k29f5"] Jul 10 16:30:00.138736 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:00.138689 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb2033f3-705e-4008-a174-0c4cacf28449" containerName="istio-proxy" Jul 10 16:30:00.138736 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:00.138705 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb2033f3-705e-4008-a174-0c4cacf28449" containerName="istio-proxy" Jul 10 16:30:00.138810 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:00.138781 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb2033f3-705e-4008-a174-0c4cacf28449" containerName="istio-proxy" Jul 10 16:30:00.141721 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:00.141705 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728350-k29f5" Jul 10 16:30:00.144290 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:00.144271 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 10 16:30:00.144506 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:00.144483 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-nzzhf\"" Jul 10 16:30:00.144586 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:00.144507 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 10 16:30:00.157344 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:00.157321 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728350-k29f5"] Jul 10 16:30:00.282822 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:00.282787 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pmnt\" (UniqueName: \"kubernetes.io/projected/4c5ec471-e07f-44c8-b1b2-2a47029524f8-kube-api-access-5pmnt\") pod \"maas-api-key-cleanup-29728350-k29f5\" (UID: \"4c5ec471-e07f-44c8-b1b2-2a47029524f8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728350-k29f5" Jul 10 16:30:00.383382 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:00.383352 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pmnt\" (UniqueName: \"kubernetes.io/projected/4c5ec471-e07f-44c8-b1b2-2a47029524f8-kube-api-access-5pmnt\") pod \"maas-api-key-cleanup-29728350-k29f5\" (UID: \"4c5ec471-e07f-44c8-b1b2-2a47029524f8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728350-k29f5" Jul 10 16:30:00.392520 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:00.392448 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pmnt\" (UniqueName: \"kubernetes.io/projected/4c5ec471-e07f-44c8-b1b2-2a47029524f8-kube-api-access-5pmnt\") pod \"maas-api-key-cleanup-29728350-k29f5\" (UID: \"4c5ec471-e07f-44c8-b1b2-2a47029524f8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728350-k29f5" Jul 10 16:30:00.452391 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:00.452360 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728350-k29f5" Jul 10 16:30:00.584176 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:00.584155 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728350-k29f5"] Jul 10 16:30:00.586444 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:30:00.586415 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c5ec471_e07f_44c8_b1b2_2a47029524f8.slice/crio-6f6def71332ef51522477eff8dadefca65d103d15b9f4cb4b9ad499a3f0af728 WatchSource:0}: Error finding container 6f6def71332ef51522477eff8dadefca65d103d15b9f4cb4b9ad499a3f0af728: Status 404 returned error can't find the container with id 6f6def71332ef51522477eff8dadefca65d103d15b9f4cb4b9ad499a3f0af728 Jul 10 16:30:01.406231 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:01.406193 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728350-k29f5" event={"ID":"4c5ec471-e07f-44c8-b1b2-2a47029524f8","Type":"ContainerStarted","Data":"3f821bce31d5d93b471fe4e54555cdb6443dea68d0e0e663905b19c97faa7713"} Jul 10 16:30:01.406624 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:01.406248 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728350-k29f5" event={"ID":"4c5ec471-e07f-44c8-b1b2-2a47029524f8","Type":"ContainerStarted","Data":"6f6def71332ef51522477eff8dadefca65d103d15b9f4cb4b9ad499a3f0af728"} Jul 10 16:30:01.422836 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:01.422788 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728350-k29f5" podStartSLOduration=1.422769806 podStartE2EDuration="1.422769806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:30:01.422019284 +0000 UTC m=+2090.394052660" watchObservedRunningTime="2026-07-10 16:30:01.422769806 +0000 UTC m=+2090.394803185" Jul 10 16:30:02.841278 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:02.841246 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:30:08.820417 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:08.820381 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx"] Jul 10 16:30:08.824168 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:08.824146 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:08.827578 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:08.827561 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ef5048a4-openshift-default-dockercfg-8jslf\"" Jul 10 16:30:08.845128 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:08.845104 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx"] Jul 10 16:30:08.956801 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:08.956764 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92s87\" (UniqueName: \"kubernetes.io/projected/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-kube-api-access-92s87\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:08.956978 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:08.956815 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-credential-socket\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:08.956978 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:08.956885 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istiod-ca-cert\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:08.956978 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:08.956944 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-workload-socket\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:08.957093 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:08.957009 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-envoy\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:08.957093 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:08.957032 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-token\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:08.957093 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:08.957063 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-podinfo\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:08.957188 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:08.957096 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-workload-certs\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:08.957188 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:08.957126 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-data\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:09.057658 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.057624 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istiod-ca-cert\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:09.057658 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.057662 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-workload-socket\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:09.057883 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.057708 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-envoy\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:09.057883 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.057738 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-token\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:09.057883 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.057777 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-podinfo\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:09.057883 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.057806 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-workload-certs\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:09.057883 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.057837 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-data\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:09.057883 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.057865 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92s87\" (UniqueName: \"kubernetes.io/projected/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-kube-api-access-92s87\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:09.058229 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.057915 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-credential-socket\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:09.058419 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.058391 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-workload-socket\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:09.058419 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.058407 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-data\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:09.058618 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.058462 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-credential-socket\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:09.058618 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.058474 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-workload-certs\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:09.058618 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.058478 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istiod-ca-cert\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:09.060143 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.060121 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-envoy\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:09.060343 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.060326 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-podinfo\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:09.066936 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.066906 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92s87\" (UniqueName: \"kubernetes.io/projected/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-kube-api-access-92s87\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:09.067400 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.067376 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-token\") pod \"e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:09.136989 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.136919 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:09.280989 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.280939 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx"] Jul 10 16:30:09.282245 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:30:09.282221 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f14fa60_0c47_43aa_88d2_7cf1caf878cd.slice/crio-dda471d98cba55b2bbbff5b6a8f37ad657fbcb832d69dad69bb534422e7637dc WatchSource:0}: Error finding container dda471d98cba55b2bbbff5b6a8f37ad657fbcb832d69dad69bb534422e7637dc: Status 404 returned error can't find the container with id dda471d98cba55b2bbbff5b6a8f37ad657fbcb832d69dad69bb534422e7637dc Jul 10 16:30:09.284526 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.284490 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:30:09.284602 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.284558 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:30:09.284602 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.284586 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:30:09.448513 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.448378 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" event={"ID":"9f14fa60-0c47-43aa-88d2-7cf1caf878cd","Type":"ContainerStarted","Data":"bb1e91fa3ffdd9ca516dcbc478793349f2f921eb919d26b520a3a7aec71440ca"} Jul 10 16:30:09.448513 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:09.448446 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" event={"ID":"9f14fa60-0c47-43aa-88d2-7cf1caf878cd","Type":"ContainerStarted","Data":"dda471d98cba55b2bbbff5b6a8f37ad657fbcb832d69dad69bb534422e7637dc"} Jul 10 16:30:10.137554 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:10.137519 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:10.142408 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:10.142385 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:10.186344 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:10.186293 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" podStartSLOduration=2.186276609 podStartE2EDuration="2.186276609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:30:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:30:09.483600028 +0000 UTC m=+2098.455633404" watchObservedRunningTime="2026-07-10 16:30:10.186276609 +0000 UTC m=+2099.158309983" Jul 10 16:30:10.454576 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:10.454479 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:10.455550 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:10.455532 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:30:11.597070 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:11.596884 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6s2sn_a7a2569d-7313-49a4-b283-aa416f5fdd9e/ovn-acl-logging/0.log" Jul 10 16:30:11.603248 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:11.603222 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6s2sn_a7a2569d-7313-49a4-b283-aa416f5fdd9e/ovn-acl-logging/0.log" Jul 10 16:30:11.698705 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:30:11.698669 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:31:00.012285 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:31:00.012244 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728350-k29f5"] Jul 10 16:31:00.012873 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:31:00.012520 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728350-k29f5" podUID="4c5ec471-e07f-44c8-b1b2-2a47029524f8" containerName="cleanup" containerID="cri-o://3f821bce31d5d93b471fe4e54555cdb6443dea68d0e0e663905b19c97faa7713" gracePeriod=30 Jul 10 16:31:30.198540 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:31:30.198515 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728350-k29f5" Jul 10 16:31:30.205979 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:31:30.205945 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pmnt\" (UniqueName: \"kubernetes.io/projected/4c5ec471-e07f-44c8-b1b2-2a47029524f8-kube-api-access-5pmnt\") pod \"4c5ec471-e07f-44c8-b1b2-2a47029524f8\" (UID: \"4c5ec471-e07f-44c8-b1b2-2a47029524f8\") " Jul 10 16:31:30.207842 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:31:30.207812 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c5ec471-e07f-44c8-b1b2-2a47029524f8-kube-api-access-5pmnt" (OuterVolumeSpecName: "kube-api-access-5pmnt") pod "4c5ec471-e07f-44c8-b1b2-2a47029524f8" (UID: "4c5ec471-e07f-44c8-b1b2-2a47029524f8"). InnerVolumeSpecName "kube-api-access-5pmnt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:31:30.306899 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:31:30.306808 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pmnt\" (UniqueName: \"kubernetes.io/projected/4c5ec471-e07f-44c8-b1b2-2a47029524f8-kube-api-access-5pmnt\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:31:30.836645 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:31:30.836609 2560 generic.go:356] "Generic (PLEG): container finished" podID="4c5ec471-e07f-44c8-b1b2-2a47029524f8" containerID="3f821bce31d5d93b471fe4e54555cdb6443dea68d0e0e663905b19c97faa7713" exitCode=137 Jul 10 16:31:30.836815 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:31:30.836658 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728350-k29f5" Jul 10 16:31:30.836815 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:31:30.836695 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728350-k29f5" event={"ID":"4c5ec471-e07f-44c8-b1b2-2a47029524f8","Type":"ContainerDied","Data":"3f821bce31d5d93b471fe4e54555cdb6443dea68d0e0e663905b19c97faa7713"} Jul 10 16:31:30.836815 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:31:30.836754 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29728350-k29f5" event={"ID":"4c5ec471-e07f-44c8-b1b2-2a47029524f8","Type":"ContainerDied","Data":"6f6def71332ef51522477eff8dadefca65d103d15b9f4cb4b9ad499a3f0af728"} Jul 10 16:31:30.836815 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:31:30.836776 2560 scope.go:117] "RemoveContainer" containerID="3f821bce31d5d93b471fe4e54555cdb6443dea68d0e0e663905b19c97faa7713" Jul 10 16:31:30.845389 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:31:30.845375 2560 scope.go:117] "RemoveContainer" containerID="3f821bce31d5d93b471fe4e54555cdb6443dea68d0e0e663905b19c97faa7713" Jul 10 16:31:30.845637 ip-10-0-129-101 kubenswrapper[2560]: E0710 16:31:30.845618 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f821bce31d5d93b471fe4e54555cdb6443dea68d0e0e663905b19c97faa7713\": container with ID starting with 3f821bce31d5d93b471fe4e54555cdb6443dea68d0e0e663905b19c97faa7713 not found: ID does not exist" containerID="3f821bce31d5d93b471fe4e54555cdb6443dea68d0e0e663905b19c97faa7713" Jul 10 16:31:30.845683 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:31:30.845649 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f821bce31d5d93b471fe4e54555cdb6443dea68d0e0e663905b19c97faa7713"} err="failed to get container status \"3f821bce31d5d93b471fe4e54555cdb6443dea68d0e0e663905b19c97faa7713\": rpc error: code = NotFound desc = could not find container \"3f821bce31d5d93b471fe4e54555cdb6443dea68d0e0e663905b19c97faa7713\": container with ID starting with 3f821bce31d5d93b471fe4e54555cdb6443dea68d0e0e663905b19c97faa7713 not found: ID does not exist" Jul 10 16:31:30.860863 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:31:30.860840 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728350-k29f5"] Jul 10 16:31:30.878372 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:31:30.878344 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29728350-k29f5"] Jul 10 16:31:31.552166 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:31:31.552132 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c5ec471-e07f-44c8-b1b2-2a47029524f8" path="/var/lib/kubelet/pods/4c5ec471-e07f-44c8-b1b2-2a47029524f8/volumes" Jul 10 16:33:28.406259 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:28.406225 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx"] Jul 10 16:33:28.408806 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:28.406474 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" podUID="9f14fa60-0c47-43aa-88d2-7cf1caf878cd" containerName="istio-proxy" containerID="cri-o://bb1e91fa3ffdd9ca516dcbc478793349f2f921eb919d26b520a3a7aec71440ca" gracePeriod=30 Jul 10 16:33:33.663988 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.663966 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:33:33.728615 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.728591 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-envoy\") pod \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " Jul 10 16:33:33.728788 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.728625 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istiod-ca-cert\") pod \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " Jul 10 16:33:33.728788 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.728647 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-token\") pod \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " Jul 10 16:33:33.728788 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.728685 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92s87\" (UniqueName: \"kubernetes.io/projected/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-kube-api-access-92s87\") pod \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " Jul 10 16:33:33.728788 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.728719 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-credential-socket\") pod \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " Jul 10 16:33:33.728788 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.728762 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-workload-socket\") pod \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " Jul 10 16:33:33.729091 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.728798 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-workload-certs\") pod \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " Jul 10 16:33:33.729091 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.728848 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-podinfo\") pod \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " Jul 10 16:33:33.729091 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.728878 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-data\") pod \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\" (UID: \"9f14fa60-0c47-43aa-88d2-7cf1caf878cd\") " Jul 10 16:33:33.729256 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.729103 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9f14fa60-0c47-43aa-88d2-7cf1caf878cd" (UID: "9f14fa60-0c47-43aa-88d2-7cf1caf878cd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:33:33.729256 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.729117 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9f14fa60-0c47-43aa-88d2-7cf1caf878cd" (UID: "9f14fa60-0c47-43aa-88d2-7cf1caf878cd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 16:33:33.729256 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.729118 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9f14fa60-0c47-43aa-88d2-7cf1caf878cd" (UID: "9f14fa60-0c47-43aa-88d2-7cf1caf878cd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:33:33.729406 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.729328 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-data" (OuterVolumeSpecName: "istio-data") pod "9f14fa60-0c47-43aa-88d2-7cf1caf878cd" (UID: "9f14fa60-0c47-43aa-88d2-7cf1caf878cd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:33:33.729406 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.729369 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9f14fa60-0c47-43aa-88d2-7cf1caf878cd" (UID: "9f14fa60-0c47-43aa-88d2-7cf1caf878cd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:33:33.730869 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.730838 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9f14fa60-0c47-43aa-88d2-7cf1caf878cd" (UID: "9f14fa60-0c47-43aa-88d2-7cf1caf878cd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:33:33.731003 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.730879 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-kube-api-access-92s87" (OuterVolumeSpecName: "kube-api-access-92s87") pod "9f14fa60-0c47-43aa-88d2-7cf1caf878cd" (UID: "9f14fa60-0c47-43aa-88d2-7cf1caf878cd"). InnerVolumeSpecName "kube-api-access-92s87". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:33:33.731176 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.731157 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9f14fa60-0c47-43aa-88d2-7cf1caf878cd" (UID: "9f14fa60-0c47-43aa-88d2-7cf1caf878cd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 16:33:33.731176 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.731161 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-token" (OuterVolumeSpecName: "istio-token") pod "9f14fa60-0c47-43aa-88d2-7cf1caf878cd" (UID: "9f14fa60-0c47-43aa-88d2-7cf1caf878cd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:33:33.829486 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.829461 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-workload-certs\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:33:33.829486 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.829479 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-podinfo\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:33:33.829486 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.829488 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-data\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:33:33.829641 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.829496 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-envoy\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:33:33.829641 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.829504 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istiod-ca-cert\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:33:33.829641 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.829512 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-istio-token\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:33:33.829641 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.829520 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92s87\" (UniqueName: \"kubernetes.io/projected/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-kube-api-access-92s87\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:33:33.829641 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.829529 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-credential-socket\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:33:33.829641 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:33.829537 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f14fa60-0c47-43aa-88d2-7cf1caf878cd-workload-socket\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:33:34.428478 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:34.428440 2560 generic.go:356] "Generic (PLEG): container finished" podID="9f14fa60-0c47-43aa-88d2-7cf1caf878cd" containerID="bb1e91fa3ffdd9ca516dcbc478793349f2f921eb919d26b520a3a7aec71440ca" exitCode=0 Jul 10 16:33:34.428689 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:34.428510 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" Jul 10 16:33:34.428689 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:34.428530 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" event={"ID":"9f14fa60-0c47-43aa-88d2-7cf1caf878cd","Type":"ContainerDied","Data":"bb1e91fa3ffdd9ca516dcbc478793349f2f921eb919d26b520a3a7aec71440ca"} Jul 10 16:33:34.428689 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:34.428624 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx" event={"ID":"9f14fa60-0c47-43aa-88d2-7cf1caf878cd","Type":"ContainerDied","Data":"dda471d98cba55b2bbbff5b6a8f37ad657fbcb832d69dad69bb534422e7637dc"} Jul 10 16:33:34.428689 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:34.428655 2560 scope.go:117] "RemoveContainer" containerID="bb1e91fa3ffdd9ca516dcbc478793349f2f921eb919d26b520a3a7aec71440ca" Jul 10 16:33:34.441108 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:34.441083 2560 scope.go:117] "RemoveContainer" containerID="bb1e91fa3ffdd9ca516dcbc478793349f2f921eb919d26b520a3a7aec71440ca" Jul 10 16:33:34.441400 ip-10-0-129-101 kubenswrapper[2560]: E0710 16:33:34.441381 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb1e91fa3ffdd9ca516dcbc478793349f2f921eb919d26b520a3a7aec71440ca\": container with ID starting with bb1e91fa3ffdd9ca516dcbc478793349f2f921eb919d26b520a3a7aec71440ca not found: ID does not exist" containerID="bb1e91fa3ffdd9ca516dcbc478793349f2f921eb919d26b520a3a7aec71440ca" Jul 10 16:33:34.441449 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:34.441409 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb1e91fa3ffdd9ca516dcbc478793349f2f921eb919d26b520a3a7aec71440ca"} err="failed to get container status \"bb1e91fa3ffdd9ca516dcbc478793349f2f921eb919d26b520a3a7aec71440ca\": rpc error: code = NotFound desc = could not find container \"bb1e91fa3ffdd9ca516dcbc478793349f2f921eb919d26b520a3a7aec71440ca\": container with ID starting with bb1e91fa3ffdd9ca516dcbc478793349f2f921eb919d26b520a3a7aec71440ca not found: ID does not exist" Jul 10 16:33:34.452974 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:34.452930 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx"] Jul 10 16:33:34.457695 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:34.457675 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ef5048a4-openshift-default-58457b8899-sgwbx"] Jul 10 16:33:35.551612 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:35.551579 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f14fa60-0c47-43aa-88d2-7cf1caf878cd" path="/var/lib/kubelet/pods/9f14fa60-0c47-43aa-88d2-7cf1caf878cd/volumes" Jul 10 16:33:48.434967 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.434929 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd"] Jul 10 16:33:48.435426 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.435317 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f14fa60-0c47-43aa-88d2-7cf1caf878cd" containerName="istio-proxy" Jul 10 16:33:48.435426 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.435330 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f14fa60-0c47-43aa-88d2-7cf1caf878cd" containerName="istio-proxy" Jul 10 16:33:48.435426 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.435344 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c5ec471-e07f-44c8-b1b2-2a47029524f8" containerName="cleanup" Jul 10 16:33:48.435426 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.435349 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c5ec471-e07f-44c8-b1b2-2a47029524f8" containerName="cleanup" Jul 10 16:33:48.435426 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.435410 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c5ec471-e07f-44c8-b1b2-2a47029524f8" containerName="cleanup" Jul 10 16:33:48.435426 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.435419 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f14fa60-0c47-43aa-88d2-7cf1caf878cd" containerName="istio-proxy" Jul 10 16:33:48.438232 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.438209 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.440608 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.440588 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-468a4585-openshift-default-dockercfg-qqjs4\"" Jul 10 16:33:48.454615 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.454592 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd"] Jul 10 16:33:48.528337 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.528269 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92mrh\" (UniqueName: \"kubernetes.io/projected/ec906093-3faa-41e1-bebe-358d72efcf42-kube-api-access-92mrh\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.528337 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.528308 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-workload-socket\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.528584 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.528409 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec906093-3faa-41e1-bebe-358d72efcf42-istiod-ca-cert\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.528584 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.528440 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-workload-certs\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.528584 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.528485 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-istio-data\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.528584 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.528507 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec906093-3faa-41e1-bebe-358d72efcf42-istio-token\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.528584 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.528582 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-credential-socket\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.528875 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.528694 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec906093-3faa-41e1-bebe-358d72efcf42-istio-podinfo\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.528875 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.528740 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-istio-envoy\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.629348 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.629313 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec906093-3faa-41e1-bebe-358d72efcf42-istiod-ca-cert\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.629348 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.629347 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-workload-certs\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.629603 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.629364 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-istio-data\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.629603 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.629380 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec906093-3faa-41e1-bebe-358d72efcf42-istio-token\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.629603 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.629412 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-credential-socket\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.629603 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.629462 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec906093-3faa-41e1-bebe-358d72efcf42-istio-podinfo\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.629603 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.629501 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-istio-envoy\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.629603 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.629531 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92mrh\" (UniqueName: \"kubernetes.io/projected/ec906093-3faa-41e1-bebe-358d72efcf42-kube-api-access-92mrh\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.629603 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.629552 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-workload-socket\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.630008 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.629869 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-workload-certs\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.630081 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.630056 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-credential-socket\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.630163 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.630141 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-istio-data\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.630213 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.630163 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-workload-socket\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.630365 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.630342 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec906093-3faa-41e1-bebe-358d72efcf42-istiod-ca-cert\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.632037 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.632015 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-istio-envoy\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.632241 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.632214 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec906093-3faa-41e1-bebe-358d72efcf42-istio-podinfo\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.640572 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.640551 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec906093-3faa-41e1-bebe-358d72efcf42-istio-token\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.640671 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.640557 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92mrh\" (UniqueName: \"kubernetes.io/projected/ec906093-3faa-41e1-bebe-358d72efcf42-kube-api-access-92mrh\") pod \"e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.748211 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.748182 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:48.878620 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.878595 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd"] Jul 10 16:33:48.880105 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:33:48.880078 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec906093_3faa_41e1_bebe_358d72efcf42.slice/crio-d797bfa939f8ce8c99b159dfa96ef290d189037194c8852acdc3feb0fab0dfef WatchSource:0}: Error finding container d797bfa939f8ce8c99b159dfa96ef290d189037194c8852acdc3feb0fab0dfef: Status 404 returned error can't find the container with id d797bfa939f8ce8c99b159dfa96ef290d189037194c8852acdc3feb0fab0dfef Jul 10 16:33:48.881923 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.881891 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:33:48.882027 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.881973 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:33:48.882027 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:48.882005 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:33:49.509183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:49.509146 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" event={"ID":"ec906093-3faa-41e1-bebe-358d72efcf42","Type":"ContainerStarted","Data":"82a7ff0089be2115b83cc40f02a09a33051103fffeaea7ac00dfea350fece729"} Jul 10 16:33:49.509645 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:49.509202 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" event={"ID":"ec906093-3faa-41e1-bebe-358d72efcf42","Type":"ContainerStarted","Data":"d797bfa939f8ce8c99b159dfa96ef290d189037194c8852acdc3feb0fab0dfef"} Jul 10 16:33:49.529973 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:49.529907 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" podStartSLOduration=1.529886235 podStartE2EDuration="1.529886235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:33:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:33:49.529194951 +0000 UTC m=+2318.501228326" watchObservedRunningTime="2026-07-10 16:33:49.529886235 +0000 UTC m=+2318.501919609" Jul 10 16:33:49.748673 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:49.748639 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:49.753688 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:49.753665 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:50.514677 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:50.514648 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:50.515657 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:50.515641 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:33:57.650101 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:33:57.650065 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:34:05.601053 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:05.600269 2560 patch_prober.go:28] interesting pod/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": read tcp 10.133.0.2:50556->10.133.0.47:15021: read: connection reset by peer" start-of-body= Jul 10 16:34:05.601053 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:05.600364 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" podUID="ec906093-3faa-41e1-bebe-358d72efcf42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": read tcp 10.133.0.2:50556->10.133.0.47:15021: read: connection reset by peer" Jul 10 16:34:06.595148 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:06.595111 2560 generic.go:356] "Generic (PLEG): container finished" podID="ec906093-3faa-41e1-bebe-358d72efcf42" containerID="82a7ff0089be2115b83cc40f02a09a33051103fffeaea7ac00dfea350fece729" exitCode=0 Jul 10 16:34:06.595348 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:06.595189 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" event={"ID":"ec906093-3faa-41e1-bebe-358d72efcf42","Type":"ContainerDied","Data":"82a7ff0089be2115b83cc40f02a09a33051103fffeaea7ac00dfea350fece729"} Jul 10 16:34:06.595649 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:06.595625 2560 scope.go:117] "RemoveContainer" containerID="82a7ff0089be2115b83cc40f02a09a33051103fffeaea7ac00dfea350fece729" Jul 10 16:34:06.596637 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:06.596610 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:34:06.596726 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:06.596691 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:34:06.596789 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:06.596737 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:34:07.604899 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:07.604856 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" event={"ID":"ec906093-3faa-41e1-bebe-358d72efcf42","Type":"ContainerStarted","Data":"7cde58043a083c594537c9e29a31aaddf2be80627d717179691dc2e7c8103478"} Jul 10 16:34:07.748411 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:07.748369 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:34:07.748653 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:07.748635 2560 patch_prober.go:28] interesting pod/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 10 16:34:07.748701 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:07.748679 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" podUID="ec906093-3faa-41e1-bebe-358d72efcf42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 10 16:34:08.748821 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:08.748779 2560 patch_prober.go:28] interesting pod/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 10 16:34:08.749263 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:08.748841 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" podUID="ec906093-3faa-41e1-bebe-358d72efcf42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 10 16:34:09.342344 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:09.342313 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:34:09.749132 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:09.749100 2560 patch_prober.go:28] interesting pod/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 10 16:34:09.749499 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:09.749156 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" podUID="ec906093-3faa-41e1-bebe-358d72efcf42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 10 16:34:10.749548 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:10.749508 2560 patch_prober.go:28] interesting pod/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Jul 10 16:34:10.749983 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:10.749573 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" podUID="ec906093-3faa-41e1-bebe-358d72efcf42" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Jul 10 16:34:11.752454 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:11.752424 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:34:11.752830 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:11.752705 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:34:11.753257 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:11.753240 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:34:20.970160 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:20.970126 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:34:35.451664 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:34:35.451626 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:35:09.343500 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:09.343459 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:35:11.635535 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:11.635505 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6s2sn_a7a2569d-7313-49a4-b283-aa416f5fdd9e/ovn-acl-logging/0.log" Jul 10 16:35:11.640442 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:11.640422 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6s2sn_a7a2569d-7313-49a4-b283-aa416f5fdd9e/ovn-acl-logging/0.log" Jul 10 16:35:12.330585 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:12.330550 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd"] Jul 10 16:35:12.330919 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:12.330891 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" podUID="ec906093-3faa-41e1-bebe-358d72efcf42" containerName="istio-proxy" containerID="cri-o://7cde58043a083c594537c9e29a31aaddf2be80627d717179691dc2e7c8103478" gracePeriod=30 Jul 10 16:35:16.256567 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:16.256531 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:35:17.582218 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.582186 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:35:17.693736 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.693644 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-credential-socket\") pod \"ec906093-3faa-41e1-bebe-358d72efcf42\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " Jul 10 16:35:17.693736 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.693708 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-istio-envoy\") pod \"ec906093-3faa-41e1-bebe-358d72efcf42\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " Jul 10 16:35:17.693736 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.693743 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec906093-3faa-41e1-bebe-358d72efcf42-istiod-ca-cert\") pod \"ec906093-3faa-41e1-bebe-358d72efcf42\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " Jul 10 16:35:17.694066 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.693778 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec906093-3faa-41e1-bebe-358d72efcf42-istio-token\") pod \"ec906093-3faa-41e1-bebe-358d72efcf42\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " Jul 10 16:35:17.694066 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.693808 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-istio-data\") pod \"ec906093-3faa-41e1-bebe-358d72efcf42\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " Jul 10 16:35:17.694066 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.693833 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92mrh\" (UniqueName: \"kubernetes.io/projected/ec906093-3faa-41e1-bebe-358d72efcf42-kube-api-access-92mrh\") pod \"ec906093-3faa-41e1-bebe-358d72efcf42\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " Jul 10 16:35:17.694066 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.693875 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-workload-socket\") pod \"ec906093-3faa-41e1-bebe-358d72efcf42\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " Jul 10 16:35:17.694066 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.693898 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-workload-certs\") pod \"ec906093-3faa-41e1-bebe-358d72efcf42\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " Jul 10 16:35:17.694066 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.693921 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec906093-3faa-41e1-bebe-358d72efcf42-istio-podinfo\") pod \"ec906093-3faa-41e1-bebe-358d72efcf42\" (UID: \"ec906093-3faa-41e1-bebe-358d72efcf42\") " Jul 10 16:35:17.694066 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.694006 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ec906093-3faa-41e1-bebe-358d72efcf42" (UID: "ec906093-3faa-41e1-bebe-358d72efcf42"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:35:17.694428 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.694240 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-credential-socket\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:35:17.694658 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.694619 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ec906093-3faa-41e1-bebe-358d72efcf42" (UID: "ec906093-3faa-41e1-bebe-358d72efcf42"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:35:17.694988 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.694862 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ec906093-3faa-41e1-bebe-358d72efcf42" (UID: "ec906093-3faa-41e1-bebe-358d72efcf42"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:35:17.694988 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.694975 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-istio-data" (OuterVolumeSpecName: "istio-data") pod "ec906093-3faa-41e1-bebe-358d72efcf42" (UID: "ec906093-3faa-41e1-bebe-358d72efcf42"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:35:17.694988 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.694981 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec906093-3faa-41e1-bebe-358d72efcf42-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ec906093-3faa-41e1-bebe-358d72efcf42" (UID: "ec906093-3faa-41e1-bebe-358d72efcf42"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 16:35:17.696557 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.696524 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ec906093-3faa-41e1-bebe-358d72efcf42-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ec906093-3faa-41e1-bebe-358d72efcf42" (UID: "ec906093-3faa-41e1-bebe-358d72efcf42"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 16:35:17.696633 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.696589 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ec906093-3faa-41e1-bebe-358d72efcf42" (UID: "ec906093-3faa-41e1-bebe-358d72efcf42"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:35:17.696633 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.696605 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec906093-3faa-41e1-bebe-358d72efcf42-istio-token" (OuterVolumeSpecName: "istio-token") pod "ec906093-3faa-41e1-bebe-358d72efcf42" (UID: "ec906093-3faa-41e1-bebe-358d72efcf42"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:35:17.696712 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.696629 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec906093-3faa-41e1-bebe-358d72efcf42-kube-api-access-92mrh" (OuterVolumeSpecName: "kube-api-access-92mrh") pod "ec906093-3faa-41e1-bebe-358d72efcf42" (UID: "ec906093-3faa-41e1-bebe-358d72efcf42"). InnerVolumeSpecName "kube-api-access-92mrh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:35:17.794757 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.794718 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec906093-3faa-41e1-bebe-358d72efcf42-istiod-ca-cert\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:35:17.794757 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.794750 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec906093-3faa-41e1-bebe-358d72efcf42-istio-token\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:35:17.794757 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.794762 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-istio-data\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:35:17.795017 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.794776 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92mrh\" (UniqueName: \"kubernetes.io/projected/ec906093-3faa-41e1-bebe-358d72efcf42-kube-api-access-92mrh\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:35:17.795017 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.794789 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-workload-socket\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:35:17.795017 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.794802 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-workload-certs\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:35:17.795017 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.794814 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec906093-3faa-41e1-bebe-358d72efcf42-istio-podinfo\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:35:17.795017 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.794824 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec906093-3faa-41e1-bebe-358d72efcf42-istio-envoy\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:35:17.954616 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.954528 2560 generic.go:356] "Generic (PLEG): container finished" podID="ec906093-3faa-41e1-bebe-358d72efcf42" containerID="7cde58043a083c594537c9e29a31aaddf2be80627d717179691dc2e7c8103478" exitCode=0 Jul 10 16:35:17.954616 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.954577 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" Jul 10 16:35:17.954790 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.954616 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" event={"ID":"ec906093-3faa-41e1-bebe-358d72efcf42","Type":"ContainerDied","Data":"7cde58043a083c594537c9e29a31aaddf2be80627d717179691dc2e7c8103478"} Jul 10 16:35:17.954790 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.954668 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd" event={"ID":"ec906093-3faa-41e1-bebe-358d72efcf42","Type":"ContainerDied","Data":"d797bfa939f8ce8c99b159dfa96ef290d189037194c8852acdc3feb0fab0dfef"} Jul 10 16:35:17.954790 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.954690 2560 scope.go:117] "RemoveContainer" containerID="7cde58043a083c594537c9e29a31aaddf2be80627d717179691dc2e7c8103478" Jul 10 16:35:17.970099 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.970074 2560 scope.go:117] "RemoveContainer" containerID="82a7ff0089be2115b83cc40f02a09a33051103fffeaea7ac00dfea350fece729" Jul 10 16:35:17.980400 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.980381 2560 scope.go:117] "RemoveContainer" containerID="7cde58043a083c594537c9e29a31aaddf2be80627d717179691dc2e7c8103478" Jul 10 16:35:17.980701 ip-10-0-129-101 kubenswrapper[2560]: E0710 16:35:17.980680 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cde58043a083c594537c9e29a31aaddf2be80627d717179691dc2e7c8103478\": container with ID starting with 7cde58043a083c594537c9e29a31aaddf2be80627d717179691dc2e7c8103478 not found: ID does not exist" containerID="7cde58043a083c594537c9e29a31aaddf2be80627d717179691dc2e7c8103478" Jul 10 16:35:17.980788 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.980716 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cde58043a083c594537c9e29a31aaddf2be80627d717179691dc2e7c8103478"} err="failed to get container status \"7cde58043a083c594537c9e29a31aaddf2be80627d717179691dc2e7c8103478\": rpc error: code = NotFound desc = could not find container \"7cde58043a083c594537c9e29a31aaddf2be80627d717179691dc2e7c8103478\": container with ID starting with 7cde58043a083c594537c9e29a31aaddf2be80627d717179691dc2e7c8103478 not found: ID does not exist" Jul 10 16:35:17.980788 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.980742 2560 scope.go:117] "RemoveContainer" containerID="82a7ff0089be2115b83cc40f02a09a33051103fffeaea7ac00dfea350fece729" Jul 10 16:35:17.981051 ip-10-0-129-101 kubenswrapper[2560]: E0710 16:35:17.981024 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82a7ff0089be2115b83cc40f02a09a33051103fffeaea7ac00dfea350fece729\": container with ID starting with 82a7ff0089be2115b83cc40f02a09a33051103fffeaea7ac00dfea350fece729 not found: ID does not exist" containerID="82a7ff0089be2115b83cc40f02a09a33051103fffeaea7ac00dfea350fece729" Jul 10 16:35:17.981188 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.981079 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82a7ff0089be2115b83cc40f02a09a33051103fffeaea7ac00dfea350fece729"} err="failed to get container status \"82a7ff0089be2115b83cc40f02a09a33051103fffeaea7ac00dfea350fece729\": rpc error: code = NotFound desc = could not find container \"82a7ff0089be2115b83cc40f02a09a33051103fffeaea7ac00dfea350fece729\": container with ID starting with 82a7ff0089be2115b83cc40f02a09a33051103fffeaea7ac00dfea350fece729 not found: ID does not exist" Jul 10 16:35:17.982460 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.982434 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd"] Jul 10 16:35:17.984352 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:17.984333 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-468a4585-openshift-default-5d58cd57dc-f58hd"] Jul 10 16:35:19.551967 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:19.551931 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec906093-3faa-41e1-bebe-358d72efcf42" path="/var/lib/kubelet/pods/ec906093-3faa-41e1-bebe-358d72efcf42/volumes" Jul 10 16:35:25.856208 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:35:25.856166 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:36:07.061056 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.061025 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf"] Jul 10 16:36:07.061657 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.061351 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec906093-3faa-41e1-bebe-358d72efcf42" containerName="istio-proxy" Jul 10 16:36:07.061657 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.061363 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec906093-3faa-41e1-bebe-358d72efcf42" containerName="istio-proxy" Jul 10 16:36:07.061657 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.061373 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec906093-3faa-41e1-bebe-358d72efcf42" containerName="istio-proxy" Jul 10 16:36:07.061657 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.061378 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec906093-3faa-41e1-bebe-358d72efcf42" containerName="istio-proxy" Jul 10 16:36:07.061657 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.061436 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec906093-3faa-41e1-bebe-358d72efcf42" containerName="istio-proxy" Jul 10 16:36:07.061657 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.061444 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec906093-3faa-41e1-bebe-358d72efcf42" containerName="istio-proxy" Jul 10 16:36:07.064389 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.064370 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.066966 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.066936 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-327c4a-openshift-default-dockercfg-vvmf9\"" Jul 10 16:36:07.078255 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.078231 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf"] Jul 10 16:36:07.232344 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.232310 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67901885-4a03-40f1-b6c5-006b81d1414d-istio-podinfo\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.232521 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.232355 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-workload-socket\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.232521 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.232394 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-credential-socket\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.232521 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.232433 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-workload-certs\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.232521 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.232461 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-istio-data\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.232521 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.232487 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-istio-envoy\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.232521 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.232506 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67901885-4a03-40f1-b6c5-006b81d1414d-istio-token\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.232785 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.232537 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67901885-4a03-40f1-b6c5-006b81d1414d-istiod-ca-cert\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.232785 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.232578 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vprg\" (UniqueName: \"kubernetes.io/projected/67901885-4a03-40f1-b6c5-006b81d1414d-kube-api-access-8vprg\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.333395 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.333308 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-istio-data\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.333395 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.333347 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-istio-envoy\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.333395 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.333372 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67901885-4a03-40f1-b6c5-006b81d1414d-istio-token\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.333681 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.333497 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67901885-4a03-40f1-b6c5-006b81d1414d-istiod-ca-cert\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.333681 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.333530 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vprg\" (UniqueName: \"kubernetes.io/projected/67901885-4a03-40f1-b6c5-006b81d1414d-kube-api-access-8vprg\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.333681 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.333583 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67901885-4a03-40f1-b6c5-006b81d1414d-istio-podinfo\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.333681 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.333639 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-workload-socket\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.333897 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.333692 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-credential-socket\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.333897 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.333731 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-workload-certs\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.333897 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.333818 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-istio-data\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.334092 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.334033 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-workload-socket\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.334312 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.334288 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67901885-4a03-40f1-b6c5-006b81d1414d-istiod-ca-cert\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.334441 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.334289 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-workload-certs\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.334441 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.334399 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-credential-socket\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.335766 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.335747 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-istio-envoy\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.336091 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.336062 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67901885-4a03-40f1-b6c5-006b81d1414d-istio-podinfo\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.342745 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.342718 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67901885-4a03-40f1-b6c5-006b81d1414d-istio-token\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.343551 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.343503 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vprg\" (UniqueName: \"kubernetes.io/projected/67901885-4a03-40f1-b6c5-006b81d1414d-kube-api-access-8vprg\") pod \"e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.376525 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.376488 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:07.527320 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.527293 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf"] Jul 10 16:36:07.529761 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:36:07.529732 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67901885_4a03_40f1_b6c5_006b81d1414d.slice/crio-8ec54b3f165b22a3570a85967e773f734cf489e83df17fcda587981d42094022 WatchSource:0}: Error finding container 8ec54b3f165b22a3570a85967e773f734cf489e83df17fcda587981d42094022: Status 404 returned error can't find the container with id 8ec54b3f165b22a3570a85967e773f734cf489e83df17fcda587981d42094022 Jul 10 16:36:07.531909 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.531871 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:36:07.531988 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.531973 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:36:07.532034 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:07.532013 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:36:08.195422 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:08.195387 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" event={"ID":"67901885-4a03-40f1-b6c5-006b81d1414d","Type":"ContainerStarted","Data":"05056ebf2e63dee8a77f4daeded600aa8cdef1271b42ab2ab05140df5c8d37b7"} Jul 10 16:36:08.195833 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:08.195443 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" event={"ID":"67901885-4a03-40f1-b6c5-006b81d1414d","Type":"ContainerStarted","Data":"8ec54b3f165b22a3570a85967e773f734cf489e83df17fcda587981d42094022"} Jul 10 16:36:08.222815 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:08.222764 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" podStartSLOduration=1.22274693 podStartE2EDuration="1.22274693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:36:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:36:08.221480372 +0000 UTC m=+2457.193513793" watchObservedRunningTime="2026-07-10 16:36:08.22274693 +0000 UTC m=+2457.194780304" Jul 10 16:36:08.377176 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:08.377131 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:08.382099 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:08.382070 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:09.201501 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:09.201468 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:09.202437 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:09.202416 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:15.426678 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.426642 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb"] Jul 10 16:36:15.431724 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.431702 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.434596 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.434571 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-eef8d7-openshift-default-dockercfg-9mlsm\"" Jul 10 16:36:15.445429 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.445405 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb"] Jul 10 16:36:15.609882 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.609849 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-credential-socket\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.610102 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.609895 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-istio-data\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.610102 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.609916 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-workload-certs\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.610102 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.609930 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-istio-envoy\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.610102 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.609999 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c13fc983-e85c-44b7-b863-d4a586fea409-istio-podinfo\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.610102 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.610086 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c13fc983-e85c-44b7-b863-d4a586fea409-istio-token\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.610274 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.610135 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9896h\" (UniqueName: \"kubernetes.io/projected/c13fc983-e85c-44b7-b863-d4a586fea409-kube-api-access-9896h\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.610274 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.610169 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c13fc983-e85c-44b7-b863-d4a586fea409-istiod-ca-cert\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.610274 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.610197 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-workload-socket\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.711120 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.711093 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-credential-socket\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.711298 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.711135 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-istio-data\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.711298 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.711155 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-workload-certs\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.711298 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.711248 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-istio-envoy\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.711456 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.711305 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c13fc983-e85c-44b7-b863-d4a586fea409-istio-podinfo\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.711456 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.711338 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c13fc983-e85c-44b7-b863-d4a586fea409-istio-token\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.711456 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.711374 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9896h\" (UniqueName: \"kubernetes.io/projected/c13fc983-e85c-44b7-b863-d4a586fea409-kube-api-access-9896h\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.711673 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.711531 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c13fc983-e85c-44b7-b863-d4a586fea409-istiod-ca-cert\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.711673 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.711577 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-workload-socket\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.711673 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.711592 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-workload-certs\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.711673 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.711612 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-credential-socket\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.711891 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.711765 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-istio-data\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.711949 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.711917 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-workload-socket\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.712126 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.712108 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c13fc983-e85c-44b7-b863-d4a586fea409-istiod-ca-cert\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.713911 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.713882 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-istio-envoy\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.714054 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.713930 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c13fc983-e85c-44b7-b863-d4a586fea409-istio-podinfo\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.719209 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.719184 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c13fc983-e85c-44b7-b863-d4a586fea409-istio-token\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.719209 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.719208 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9896h\" (UniqueName: \"kubernetes.io/projected/c13fc983-e85c-44b7-b863-d4a586fea409-kube-api-access-9896h\") pod \"e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.741749 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.741724 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:15.882228 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.882205 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb"] Jul 10 16:36:15.883985 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:36:15.883930 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc13fc983_e85c_44b7_b863_d4a586fea409.slice/crio-4cedb3baada011ce3db6c2d3f373280c5d6a5810cbb4b0ddf3a0e70a325e6f9b WatchSource:0}: Error finding container 4cedb3baada011ce3db6c2d3f373280c5d6a5810cbb4b0ddf3a0e70a325e6f9b: Status 404 returned error can't find the container with id 4cedb3baada011ce3db6c2d3f373280c5d6a5810cbb4b0ddf3a0e70a325e6f9b Jul 10 16:36:15.886276 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.886246 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:36:15.886372 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.886324 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:36:15.886435 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:15.886368 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:36:16.240235 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:16.240197 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" event={"ID":"c13fc983-e85c-44b7-b863-d4a586fea409","Type":"ContainerStarted","Data":"68d88d9115933c0f347fcdbd0a6785dcbab8672d0e5723481594316a0344287c"} Jul 10 16:36:16.240423 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:16.240262 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" event={"ID":"c13fc983-e85c-44b7-b863-d4a586fea409","Type":"ContainerStarted","Data":"4cedb3baada011ce3db6c2d3f373280c5d6a5810cbb4b0ddf3a0e70a325e6f9b"} Jul 10 16:36:16.265335 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:16.265276 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" podStartSLOduration=1.265258638 podStartE2EDuration="1.265258638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-10 16:36:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:36:16.262352332 +0000 UTC m=+2465.234385706" watchObservedRunningTime="2026-07-10 16:36:16.265258638 +0000 UTC m=+2465.237292012" Jul 10 16:36:16.742389 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:16.742349 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:16.747217 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:16.747187 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:17.245784 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:17.245745 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:17.246610 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:17.246587 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:36:17.843113 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:17.843074 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:36:25.243210 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:25.243163 2560 patch_prober.go:28] interesting pod/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 10 16:36:25.243597 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:25.243239 2560 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" podUID="67901885-4a03-40f1-b6c5-006b81d1414d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 10 16:36:26.295143 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:26.295106 2560 generic.go:356] "Generic (PLEG): container finished" podID="67901885-4a03-40f1-b6c5-006b81d1414d" containerID="05056ebf2e63dee8a77f4daeded600aa8cdef1271b42ab2ab05140df5c8d37b7" exitCode=0 Jul 10 16:36:26.295619 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:26.295187 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" event={"ID":"67901885-4a03-40f1-b6c5-006b81d1414d","Type":"ContainerDied","Data":"05056ebf2e63dee8a77f4daeded600aa8cdef1271b42ab2ab05140df5c8d37b7"} Jul 10 16:36:26.295701 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:26.295677 2560 scope.go:117] "RemoveContainer" containerID="05056ebf2e63dee8a77f4daeded600aa8cdef1271b42ab2ab05140df5c8d37b7" Jul 10 16:36:26.296766 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:26.296735 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:36:26.296840 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:26.296821 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:36:26.296881 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:26.296866 2560 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 10 16:36:27.308735 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:27.308255 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" event={"ID":"67901885-4a03-40f1-b6c5-006b81d1414d","Type":"ContainerStarted","Data":"47cae6d10c3a68149c028d098419779a5e00c10632fa270c47031e3432f064d1"} Jul 10 16:36:27.377336 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:27.377293 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:27.377639 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:27.377610 2560 patch_prober.go:28] interesting pod/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 10 16:36:27.377764 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:27.377677 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" podUID="67901885-4a03-40f1-b6c5-006b81d1414d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 10 16:36:28.377945 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:28.377903 2560 patch_prober.go:28] interesting pod/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 10 16:36:28.378370 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:28.377996 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" podUID="67901885-4a03-40f1-b6c5-006b81d1414d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 10 16:36:29.376881 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:29.376848 2560 patch_prober.go:28] interesting pod/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 10 16:36:29.377099 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:29.376918 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" podUID="67901885-4a03-40f1-b6c5-006b81d1414d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 10 16:36:30.377789 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:30.377744 2560 patch_prober.go:28] interesting pod/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 10 16:36:30.378240 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:30.377816 2560 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" podUID="67901885-4a03-40f1-b6c5-006b81d1414d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 10 16:36:31.381097 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:31.381068 2560 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:31.381490 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:31.381336 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:36:31.382004 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:36:31.381987 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:39:30.553162 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:30.553130 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-clg9r"] Jul 10 16:39:34.159226 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:34.159188 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf"] Jul 10 16:39:34.159696 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:34.159443 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" podUID="67901885-4a03-40f1-b6c5-006b81d1414d" containerName="istio-proxy" containerID="cri-o://47cae6d10c3a68149c028d098419779a5e00c10632fa270c47031e3432f064d1" gracePeriod=30 Jul 10 16:39:39.413644 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.413621 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:39:39.571429 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.571400 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-credential-socket\") pod \"67901885-4a03-40f1-b6c5-006b81d1414d\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " Jul 10 16:39:39.571429 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.571428 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-istio-data\") pod \"67901885-4a03-40f1-b6c5-006b81d1414d\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " Jul 10 16:39:39.571681 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.571464 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-istio-envoy\") pod \"67901885-4a03-40f1-b6c5-006b81d1414d\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " Jul 10 16:39:39.571681 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.571480 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67901885-4a03-40f1-b6c5-006b81d1414d-istio-token\") pod \"67901885-4a03-40f1-b6c5-006b81d1414d\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " Jul 10 16:39:39.571681 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.571514 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67901885-4a03-40f1-b6c5-006b81d1414d-istio-podinfo\") pod \"67901885-4a03-40f1-b6c5-006b81d1414d\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " Jul 10 16:39:39.571681 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.571540 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67901885-4a03-40f1-b6c5-006b81d1414d-istiod-ca-cert\") pod \"67901885-4a03-40f1-b6c5-006b81d1414d\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " Jul 10 16:39:39.571681 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.571592 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-workload-certs\") pod \"67901885-4a03-40f1-b6c5-006b81d1414d\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " Jul 10 16:39:39.571681 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.571616 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8vprg\" (UniqueName: \"kubernetes.io/projected/67901885-4a03-40f1-b6c5-006b81d1414d-kube-api-access-8vprg\") pod \"67901885-4a03-40f1-b6c5-006b81d1414d\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " Jul 10 16:39:39.571681 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.571660 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-workload-socket\") pod \"67901885-4a03-40f1-b6c5-006b81d1414d\" (UID: \"67901885-4a03-40f1-b6c5-006b81d1414d\") " Jul 10 16:39:39.572072 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.571714 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "67901885-4a03-40f1-b6c5-006b81d1414d" (UID: "67901885-4a03-40f1-b6c5-006b81d1414d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:39:39.572072 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.571918 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-credential-socket\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:39:39.572072 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.571941 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "67901885-4a03-40f1-b6c5-006b81d1414d" (UID: "67901885-4a03-40f1-b6c5-006b81d1414d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:39:39.572072 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.572013 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67901885-4a03-40f1-b6c5-006b81d1414d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "67901885-4a03-40f1-b6c5-006b81d1414d" (UID: "67901885-4a03-40f1-b6c5-006b81d1414d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 16:39:39.572282 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.572116 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "67901885-4a03-40f1-b6c5-006b81d1414d" (UID: "67901885-4a03-40f1-b6c5-006b81d1414d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:39:39.572709 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.572682 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-istio-data" (OuterVolumeSpecName: "istio-data") pod "67901885-4a03-40f1-b6c5-006b81d1414d" (UID: "67901885-4a03-40f1-b6c5-006b81d1414d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:39:39.573778 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.573755 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/67901885-4a03-40f1-b6c5-006b81d1414d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "67901885-4a03-40f1-b6c5-006b81d1414d" (UID: "67901885-4a03-40f1-b6c5-006b81d1414d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 16:39:39.573778 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.573763 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "67901885-4a03-40f1-b6c5-006b81d1414d" (UID: "67901885-4a03-40f1-b6c5-006b81d1414d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:39:39.573912 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.573794 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67901885-4a03-40f1-b6c5-006b81d1414d-istio-token" (OuterVolumeSpecName: "istio-token") pod "67901885-4a03-40f1-b6c5-006b81d1414d" (UID: "67901885-4a03-40f1-b6c5-006b81d1414d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:39:39.573912 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.573817 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67901885-4a03-40f1-b6c5-006b81d1414d-kube-api-access-8vprg" (OuterVolumeSpecName: "kube-api-access-8vprg") pod "67901885-4a03-40f1-b6c5-006b81d1414d" (UID: "67901885-4a03-40f1-b6c5-006b81d1414d"). InnerVolumeSpecName "kube-api-access-8vprg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:39:39.672877 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.672838 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/67901885-4a03-40f1-b6c5-006b81d1414d-istio-podinfo\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:39:39.672877 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.672872 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/67901885-4a03-40f1-b6c5-006b81d1414d-istiod-ca-cert\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:39:39.673144 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.672886 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-workload-certs\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:39:39.673144 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.672898 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vprg\" (UniqueName: \"kubernetes.io/projected/67901885-4a03-40f1-b6c5-006b81d1414d-kube-api-access-8vprg\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:39:39.673144 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.672912 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-workload-socket\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:39:39.673144 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.672925 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-istio-data\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:39:39.673144 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.672937 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/67901885-4a03-40f1-b6c5-006b81d1414d-istio-envoy\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:39:39.673144 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:39.672972 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/67901885-4a03-40f1-b6c5-006b81d1414d-istio-token\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:39:40.267013 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:40.266978 2560 generic.go:356] "Generic (PLEG): container finished" podID="67901885-4a03-40f1-b6c5-006b81d1414d" containerID="47cae6d10c3a68149c028d098419779a5e00c10632fa270c47031e3432f064d1" exitCode=0 Jul 10 16:39:40.267209 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:40.267036 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" Jul 10 16:39:40.267209 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:40.267040 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" event={"ID":"67901885-4a03-40f1-b6c5-006b81d1414d","Type":"ContainerDied","Data":"47cae6d10c3a68149c028d098419779a5e00c10632fa270c47031e3432f064d1"} Jul 10 16:39:40.267209 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:40.267151 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf" event={"ID":"67901885-4a03-40f1-b6c5-006b81d1414d","Type":"ContainerDied","Data":"8ec54b3f165b22a3570a85967e773f734cf489e83df17fcda587981d42094022"} Jul 10 16:39:40.267209 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:40.267172 2560 scope.go:117] "RemoveContainer" containerID="47cae6d10c3a68149c028d098419779a5e00c10632fa270c47031e3432f064d1" Jul 10 16:39:40.276490 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:40.276472 2560 scope.go:117] "RemoveContainer" containerID="05056ebf2e63dee8a77f4daeded600aa8cdef1271b42ab2ab05140df5c8d37b7" Jul 10 16:39:40.284235 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:40.284218 2560 scope.go:117] "RemoveContainer" containerID="47cae6d10c3a68149c028d098419779a5e00c10632fa270c47031e3432f064d1" Jul 10 16:39:40.284475 ip-10-0-129-101 kubenswrapper[2560]: E0710 16:39:40.284453 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47cae6d10c3a68149c028d098419779a5e00c10632fa270c47031e3432f064d1\": container with ID starting with 47cae6d10c3a68149c028d098419779a5e00c10632fa270c47031e3432f064d1 not found: ID does not exist" containerID="47cae6d10c3a68149c028d098419779a5e00c10632fa270c47031e3432f064d1" Jul 10 16:39:40.284550 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:40.284482 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47cae6d10c3a68149c028d098419779a5e00c10632fa270c47031e3432f064d1"} err="failed to get container status \"47cae6d10c3a68149c028d098419779a5e00c10632fa270c47031e3432f064d1\": rpc error: code = NotFound desc = could not find container \"47cae6d10c3a68149c028d098419779a5e00c10632fa270c47031e3432f064d1\": container with ID starting with 47cae6d10c3a68149c028d098419779a5e00c10632fa270c47031e3432f064d1 not found: ID does not exist" Jul 10 16:39:40.284550 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:40.284499 2560 scope.go:117] "RemoveContainer" containerID="05056ebf2e63dee8a77f4daeded600aa8cdef1271b42ab2ab05140df5c8d37b7" Jul 10 16:39:40.284709 ip-10-0-129-101 kubenswrapper[2560]: E0710 16:39:40.284689 2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05056ebf2e63dee8a77f4daeded600aa8cdef1271b42ab2ab05140df5c8d37b7\": container with ID starting with 05056ebf2e63dee8a77f4daeded600aa8cdef1271b42ab2ab05140df5c8d37b7 not found: ID does not exist" containerID="05056ebf2e63dee8a77f4daeded600aa8cdef1271b42ab2ab05140df5c8d37b7" Jul 10 16:39:40.284747 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:40.284713 2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05056ebf2e63dee8a77f4daeded600aa8cdef1271b42ab2ab05140df5c8d37b7"} err="failed to get container status \"05056ebf2e63dee8a77f4daeded600aa8cdef1271b42ab2ab05140df5c8d37b7\": rpc error: code = NotFound desc = could not find container \"05056ebf2e63dee8a77f4daeded600aa8cdef1271b42ab2ab05140df5c8d37b7\": container with ID starting with 05056ebf2e63dee8a77f4daeded600aa8cdef1271b42ab2ab05140df5c8d37b7 not found: ID does not exist" Jul 10 16:39:40.289926 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:40.289905 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf"] Jul 10 16:39:40.294188 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:40.294169 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-327c4a-openshift-default-65d565b7d5-ch4hf"] Jul 10 16:39:41.064039 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:41.064008 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb"] Jul 10 16:39:41.064495 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:41.064290 2560 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" podUID="c13fc983-e85c-44b7-b863-d4a586fea409" containerName="istio-proxy" containerID="cri-o://68d88d9115933c0f347fcdbd0a6785dcbab8672d0e5723481594316a0344287c" gracePeriod=30 Jul 10 16:39:41.552084 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:41.552051 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67901885-4a03-40f1-b6c5-006b81d1414d" path="/var/lib/kubelet/pods/67901885-4a03-40f1-b6c5-006b81d1414d/volumes" Jul 10 16:39:46.299716 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.299690 2560 generic.go:356] "Generic (PLEG): container finished" podID="c13fc983-e85c-44b7-b863-d4a586fea409" containerID="68d88d9115933c0f347fcdbd0a6785dcbab8672d0e5723481594316a0344287c" exitCode=0 Jul 10 16:39:46.300043 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.299771 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" event={"ID":"c13fc983-e85c-44b7-b863-d4a586fea409","Type":"ContainerDied","Data":"68d88d9115933c0f347fcdbd0a6785dcbab8672d0e5723481594316a0344287c"} Jul 10 16:39:46.313410 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.313390 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:39:46.425806 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.425723 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c13fc983-e85c-44b7-b863-d4a586fea409-istio-token\") pod \"c13fc983-e85c-44b7-b863-d4a586fea409\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " Jul 10 16:39:46.425806 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.425764 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c13fc983-e85c-44b7-b863-d4a586fea409-istio-podinfo\") pod \"c13fc983-e85c-44b7-b863-d4a586fea409\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " Jul 10 16:39:46.425806 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.425780 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c13fc983-e85c-44b7-b863-d4a586fea409-istiod-ca-cert\") pod \"c13fc983-e85c-44b7-b863-d4a586fea409\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " Jul 10 16:39:46.425806 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.425804 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-credential-socket\") pod \"c13fc983-e85c-44b7-b863-d4a586fea409\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " Jul 10 16:39:46.426169 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.425825 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-istio-envoy\") pod \"c13fc983-e85c-44b7-b863-d4a586fea409\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " Jul 10 16:39:46.426169 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.426010 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-workload-socket\") pod \"c13fc983-e85c-44b7-b863-d4a586fea409\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " Jul 10 16:39:46.426169 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.426051 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-workload-certs\") pod \"c13fc983-e85c-44b7-b863-d4a586fea409\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " Jul 10 16:39:46.426169 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.426139 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-istio-data\") pod \"c13fc983-e85c-44b7-b863-d4a586fea409\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " Jul 10 16:39:46.426169 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.426140 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c13fc983-e85c-44b7-b863-d4a586fea409" (UID: "c13fc983-e85c-44b7-b863-d4a586fea409"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:39:46.426437 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.426182 2560 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9896h\" (UniqueName: \"kubernetes.io/projected/c13fc983-e85c-44b7-b863-d4a586fea409-kube-api-access-9896h\") pod \"c13fc983-e85c-44b7-b863-d4a586fea409\" (UID: \"c13fc983-e85c-44b7-b863-d4a586fea409\") " Jul 10 16:39:46.426437 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.426348 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c13fc983-e85c-44b7-b863-d4a586fea409" (UID: "c13fc983-e85c-44b7-b863-d4a586fea409"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:39:46.426437 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.426376 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c13fc983-e85c-44b7-b863-d4a586fea409-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c13fc983-e85c-44b7-b863-d4a586fea409" (UID: "c13fc983-e85c-44b7-b863-d4a586fea409"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 10 16:39:46.426601 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.426521 2560 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c13fc983-e85c-44b7-b863-d4a586fea409-istiod-ca-cert\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:39:46.426601 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.426545 2560 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-credential-socket\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:39:46.426601 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.426561 2560 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-workload-socket\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:39:46.426765 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.426742 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c13fc983-e85c-44b7-b863-d4a586fea409" (UID: "c13fc983-e85c-44b7-b863-d4a586fea409"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:39:46.426864 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.426842 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-istio-data" (OuterVolumeSpecName: "istio-data") pod "c13fc983-e85c-44b7-b863-d4a586fea409" (UID: "c13fc983-e85c-44b7-b863-d4a586fea409"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:39:46.428196 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.428170 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c13fc983-e85c-44b7-b863-d4a586fea409" (UID: "c13fc983-e85c-44b7-b863-d4a586fea409"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 10 16:39:46.428196 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.428185 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c13fc983-e85c-44b7-b863-d4a586fea409-istio-token" (OuterVolumeSpecName: "istio-token") pod "c13fc983-e85c-44b7-b863-d4a586fea409" (UID: "c13fc983-e85c-44b7-b863-d4a586fea409"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:39:46.428368 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.428182 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c13fc983-e85c-44b7-b863-d4a586fea409-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c13fc983-e85c-44b7-b863-d4a586fea409" (UID: "c13fc983-e85c-44b7-b863-d4a586fea409"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 10 16:39:46.428519 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.428502 2560 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c13fc983-e85c-44b7-b863-d4a586fea409-kube-api-access-9896h" (OuterVolumeSpecName: "kube-api-access-9896h") pod "c13fc983-e85c-44b7-b863-d4a586fea409" (UID: "c13fc983-e85c-44b7-b863-d4a586fea409"). InnerVolumeSpecName "kube-api-access-9896h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 10 16:39:46.527887 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.527857 2560 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-istio-data\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:39:46.527887 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.527884 2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9896h\" (UniqueName: \"kubernetes.io/projected/c13fc983-e85c-44b7-b863-d4a586fea409-kube-api-access-9896h\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:39:46.527887 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.527893 2560 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c13fc983-e85c-44b7-b863-d4a586fea409-istio-token\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:39:46.528140 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.527904 2560 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c13fc983-e85c-44b7-b863-d4a586fea409-istio-podinfo\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:39:46.528140 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.527912 2560 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-istio-envoy\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:39:46.528140 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.527920 2560 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c13fc983-e85c-44b7-b863-d4a586fea409-workload-certs\") on node \"ip-10-0-129-101.ec2.internal\" DevicePath \"\"" Jul 10 16:39:46.789207 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:46.789181 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-65stt_447098e2-a5fe-4c36-833b-59631e845e53/manager/0.log" Jul 10 16:39:47.305254 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:47.305232 2560 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" Jul 10 16:39:47.305635 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:47.305231 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb" event={"ID":"c13fc983-e85c-44b7-b863-d4a586fea409","Type":"ContainerDied","Data":"4cedb3baada011ce3db6c2d3f373280c5d6a5810cbb4b0ddf3a0e70a325e6f9b"} Jul 10 16:39:47.305635 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:47.305368 2560 scope.go:117] "RemoveContainer" containerID="68d88d9115933c0f347fcdbd0a6785dcbab8672d0e5723481594316a0344287c" Jul 10 16:39:47.320068 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:47.320049 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-f9w9q_86b97add-9066-4d71-bdac-4f9f923d55f5/manager/0.log" Jul 10 16:39:47.328801 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:47.328780 2560 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb"] Jul 10 16:39:47.331941 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:47.331921 2560 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-eef8d7-openshift-default-7f86d8957c-45gbb"] Jul 10 16:39:47.445972 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:47.445927 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-765dfd8876-858s5_02f79ace-8bc2-4499-9551-5218dad2e06a/manager/0.log" Jul 10 16:39:47.453370 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:47.453350 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-765dfd8876-858s5_02f79ace-8bc2-4499-9551-5218dad2e06a/copy-manifests/0.log" Jul 10 16:39:47.551878 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:47.551846 2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c13fc983-e85c-44b7-b863-d4a586fea409" path="/var/lib/kubelet/pods/c13fc983-e85c-44b7-b863-d4a586fea409/volumes" Jul 10 16:39:48.582935 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:48.582908 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n_a7a1f505-d99d-4fc9-99b4-078f5842fb95/util/0.log" Jul 10 16:39:48.590009 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:48.589991 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n_a7a1f505-d99d-4fc9-99b4-078f5842fb95/pull/0.log" Jul 10 16:39:48.596542 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:48.596527 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n_a7a1f505-d99d-4fc9-99b4-078f5842fb95/extract/0.log" Jul 10 16:39:48.715603 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:48.715573 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5_1b4821b8-2b1a-4d1a-aa0c-d21554082e85/pull/0.log" Jul 10 16:39:48.722173 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:48.722154 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5_1b4821b8-2b1a-4d1a-aa0c-d21554082e85/extract/0.log" Jul 10 16:39:48.728501 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:48.728481 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5_1b4821b8-2b1a-4d1a-aa0c-d21554082e85/util/0.log" Jul 10 16:39:48.851778 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:48.851709 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd_15ec01c2-78da-4669-a2ee-927426aea148/util/0.log" Jul 10 16:39:48.858707 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:48.858685 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd_15ec01c2-78da-4669-a2ee-927426aea148/pull/0.log" Jul 10 16:39:48.865684 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:48.865665 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd_15ec01c2-78da-4669-a2ee-927426aea148/extract/0.log" Jul 10 16:39:48.989929 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:48.989888 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh_dd82ea12-c8d5-4256-bcdb-cd937fe07abd/util/0.log" Jul 10 16:39:48.997678 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:48.997649 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh_dd82ea12-c8d5-4256-bcdb-cd937fe07abd/pull/0.log" Jul 10 16:39:49.004739 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:49.004716 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh_dd82ea12-c8d5-4256-bcdb-cd937fe07abd/extract/0.log" Jul 10 16:39:49.259860 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:49.259829 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-j54kl_d08ee4ef-a529-4b04-af3e-33995581b84b/manager/0.log" Jul 10 16:39:49.634210 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:49.634131 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-w9vd9_a004768d-ac8e-4ef3-9d38-a1547c29c362/registry-server/0.log" Jul 10 16:39:49.769490 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:49.769457 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-8g7ms_8283651d-1e04-494d-8f91-8bf92c2784e8/manager/0.log" Jul 10 16:39:49.896769 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:49.896692 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-clg9r_9b007e32-760f-4ad5-8ed7-9f9676cc8178/limitador/0.log" Jul 10 16:39:50.544315 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:50.544287 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5f67m_6205ec9a-afd1-4e7d-97b6-56b60c00847a/discovery/0.log" Jul 10 16:39:50.664603 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:50.664558 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b74489749-cvbvn_4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8/kube-auth-proxy/0.log" Jul 10 16:39:52.147158 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:39:52.147128 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-tmzwg_356e5127-839c-42a9-ba7f-fed13feea3fa/main/0.log" Jul 10 16:40:00.002857 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:00.002809 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-h9z25_7511b558-f953-4c6b-a90b-c2843281f2cc/global-pull-secret-syncer/0.log" Jul 10 16:40:00.179131 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:00.179097 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8bbgg_d5c8652b-3f54-4b1d-8bbb-597347aba689/konnectivity-agent/0.log" Jul 10 16:40:00.233444 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:00.233408 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-101.ec2.internal_c72755a90b814da38c5f2e7b423a2466/haproxy/0.log" Jul 10 16:40:04.797601 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:04.797513 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n_a7a1f505-d99d-4fc9-99b4-078f5842fb95/extract/0.log" Jul 10 16:40:04.832840 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:04.832812 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n_a7a1f505-d99d-4fc9-99b4-078f5842fb95/util/0.log" Jul 10 16:40:04.857695 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:04.857670 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7598cp2n_a7a1f505-d99d-4fc9-99b4-078f5842fb95/pull/0.log" Jul 10 16:40:04.904669 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:04.904649 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5_1b4821b8-2b1a-4d1a-aa0c-d21554082e85/extract/0.log" Jul 10 16:40:04.955517 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:04.955491 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5_1b4821b8-2b1a-4d1a-aa0c-d21554082e85/util/0.log" Jul 10 16:40:04.986657 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:04.986631 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e027nh5_1b4821b8-2b1a-4d1a-aa0c-d21554082e85/pull/0.log" Jul 10 16:40:05.027845 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:05.027821 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd_15ec01c2-78da-4669-a2ee-927426aea148/extract/0.log" Jul 10 16:40:05.051006 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:05.050926 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd_15ec01c2-78da-4669-a2ee-927426aea148/util/0.log" Jul 10 16:40:05.074189 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:05.074169 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jjwnd_15ec01c2-78da-4669-a2ee-927426aea148/pull/0.log" Jul 10 16:40:05.101812 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:05.101788 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh_dd82ea12-c8d5-4256-bcdb-cd937fe07abd/extract/0.log" Jul 10 16:40:05.124725 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:05.124703 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh_dd82ea12-c8d5-4256-bcdb-cd937fe07abd/util/0.log" Jul 10 16:40:05.150336 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:05.150307 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fwsbh_dd82ea12-c8d5-4256-bcdb-cd937fe07abd/pull/0.log" Jul 10 16:40:05.435909 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:05.435818 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-j54kl_d08ee4ef-a529-4b04-af3e-33995581b84b/manager/0.log" Jul 10 16:40:05.536997 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:05.536969 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-w9vd9_a004768d-ac8e-4ef3-9d38-a1547c29c362/registry-server/0.log" Jul 10 16:40:05.624031 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:05.623994 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-8g7ms_8283651d-1e04-494d-8f91-8bf92c2784e8/manager/0.log" Jul 10 16:40:05.669290 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:05.669256 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-clg9r_9b007e32-760f-4ad5-8ed7-9f9676cc8178/limitador/0.log" Jul 10 16:40:07.835944 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:07.835916 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hdxsb_1347d2e6-ddae-479d-92d5-603b85a892c3/node-exporter/0.log" Jul 10 16:40:07.858630 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:07.858603 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hdxsb_1347d2e6-ddae-479d-92d5-603b85a892c3/kube-rbac-proxy/0.log" Jul 10 16:40:07.881749 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:07.881720 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hdxsb_1347d2e6-ddae-479d-92d5-603b85a892c3/init-textfile/0.log" Jul 10 16:40:09.986982 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:09.986937 2560 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8"] Jul 10 16:40:09.987352 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:09.987276 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67901885-4a03-40f1-b6c5-006b81d1414d" containerName="istio-proxy" Jul 10 16:40:09.987352 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:09.987287 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="67901885-4a03-40f1-b6c5-006b81d1414d" containerName="istio-proxy" Jul 10 16:40:09.987352 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:09.987298 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c13fc983-e85c-44b7-b863-d4a586fea409" containerName="istio-proxy" Jul 10 16:40:09.987352 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:09.987303 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="c13fc983-e85c-44b7-b863-d4a586fea409" containerName="istio-proxy" Jul 10 16:40:09.987352 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:09.987318 2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67901885-4a03-40f1-b6c5-006b81d1414d" containerName="istio-proxy" Jul 10 16:40:09.987352 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:09.987326 2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="67901885-4a03-40f1-b6c5-006b81d1414d" containerName="istio-proxy" Jul 10 16:40:09.987538 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:09.987376 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="c13fc983-e85c-44b7-b863-d4a586fea409" containerName="istio-proxy" Jul 10 16:40:09.987538 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:09.987388 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="67901885-4a03-40f1-b6c5-006b81d1414d" containerName="istio-proxy" Jul 10 16:40:09.987538 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:09.987398 2560 memory_manager.go:368] "RemoveStaleState removing state" podUID="67901885-4a03-40f1-b6c5-006b81d1414d" containerName="istio-proxy" Jul 10 16:40:09.990525 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:09.990508 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8" Jul 10 16:40:09.992819 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:09.992801 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lf2rd\"/\"kube-root-ca.crt\"" Jul 10 16:40:09.992927 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:09.992808 2560 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lf2rd\"/\"openshift-service-ca.crt\"" Jul 10 16:40:09.993677 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:09.993663 2560 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lf2rd\"/\"default-dockercfg-tt42s\"" Jul 10 16:40:10.000819 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:10.000796 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8"] Jul 10 16:40:10.028907 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:10.028875 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdmxh\" (UniqueName: \"kubernetes.io/projected/52d1ab13-506d-4b17-a95d-2177134bedbe-kube-api-access-sdmxh\") pod \"perf-node-gather-daemonset-66xz8\" (UID: \"52d1ab13-506d-4b17-a95d-2177134bedbe\") " pod="openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8" Jul 10 16:40:10.029091 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:10.028916 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52d1ab13-506d-4b17-a95d-2177134bedbe-sys\") pod \"perf-node-gather-daemonset-66xz8\" (UID: \"52d1ab13-506d-4b17-a95d-2177134bedbe\") " pod="openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8" Jul 10 16:40:10.029091 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:10.028933 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/52d1ab13-506d-4b17-a95d-2177134bedbe-lib-modules\") pod \"perf-node-gather-daemonset-66xz8\" (UID: \"52d1ab13-506d-4b17-a95d-2177134bedbe\") " pod="openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8" Jul 10 16:40:10.029091 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:10.029051 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/52d1ab13-506d-4b17-a95d-2177134bedbe-proc\") pod \"perf-node-gather-daemonset-66xz8\" (UID: \"52d1ab13-506d-4b17-a95d-2177134bedbe\") " pod="openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8" Jul 10 16:40:10.029223 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:10.029109 2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/52d1ab13-506d-4b17-a95d-2177134bedbe-podres\") pod \"perf-node-gather-daemonset-66xz8\" (UID: \"52d1ab13-506d-4b17-a95d-2177134bedbe\") " pod="openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8" Jul 10 16:40:10.130307 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:10.130274 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/52d1ab13-506d-4b17-a95d-2177134bedbe-proc\") pod \"perf-node-gather-daemonset-66xz8\" (UID: \"52d1ab13-506d-4b17-a95d-2177134bedbe\") " pod="openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8" Jul 10 16:40:10.130485 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:10.130332 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/52d1ab13-506d-4b17-a95d-2177134bedbe-podres\") pod \"perf-node-gather-daemonset-66xz8\" (UID: \"52d1ab13-506d-4b17-a95d-2177134bedbe\") " pod="openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8" Jul 10 16:40:10.130485 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:10.130382 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdmxh\" (UniqueName: \"kubernetes.io/projected/52d1ab13-506d-4b17-a95d-2177134bedbe-kube-api-access-sdmxh\") pod \"perf-node-gather-daemonset-66xz8\" (UID: \"52d1ab13-506d-4b17-a95d-2177134bedbe\") " pod="openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8" Jul 10 16:40:10.130485 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:10.130409 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/52d1ab13-506d-4b17-a95d-2177134bedbe-proc\") pod \"perf-node-gather-daemonset-66xz8\" (UID: \"52d1ab13-506d-4b17-a95d-2177134bedbe\") " pod="openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8" Jul 10 16:40:10.130485 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:10.130422 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52d1ab13-506d-4b17-a95d-2177134bedbe-sys\") pod \"perf-node-gather-daemonset-66xz8\" (UID: \"52d1ab13-506d-4b17-a95d-2177134bedbe\") " pod="openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8" Jul 10 16:40:10.130485 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:10.130469 2560 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/52d1ab13-506d-4b17-a95d-2177134bedbe-lib-modules\") pod \"perf-node-gather-daemonset-66xz8\" (UID: \"52d1ab13-506d-4b17-a95d-2177134bedbe\") " pod="openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8" Jul 10 16:40:10.130666 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:10.130475 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/52d1ab13-506d-4b17-a95d-2177134bedbe-sys\") pod \"perf-node-gather-daemonset-66xz8\" (UID: \"52d1ab13-506d-4b17-a95d-2177134bedbe\") " pod="openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8" Jul 10 16:40:10.130666 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:10.130519 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/52d1ab13-506d-4b17-a95d-2177134bedbe-podres\") pod \"perf-node-gather-daemonset-66xz8\" (UID: \"52d1ab13-506d-4b17-a95d-2177134bedbe\") " pod="openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8" Jul 10 16:40:10.130666 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:10.130559 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/52d1ab13-506d-4b17-a95d-2177134bedbe-lib-modules\") pod \"perf-node-gather-daemonset-66xz8\" (UID: \"52d1ab13-506d-4b17-a95d-2177134bedbe\") " pod="openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8" Jul 10 16:40:10.142944 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:10.142913 2560 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdmxh\" (UniqueName: \"kubernetes.io/projected/52d1ab13-506d-4b17-a95d-2177134bedbe-kube-api-access-sdmxh\") pod \"perf-node-gather-daemonset-66xz8\" (UID: \"52d1ab13-506d-4b17-a95d-2177134bedbe\") " pod="openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8" Jul 10 16:40:10.300931 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:10.300840 2560 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8" Jul 10 16:40:10.431738 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:10.428930 2560 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8"] Jul 10 16:40:10.433057 ip-10-0-129-101 kubenswrapper[2560]: W0710 16:40:10.433021 2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod52d1ab13_506d_4b17_a95d_2177134bedbe.slice/crio-22ec42f6994129e3995512ec69665e173ad6889764f944ddd4e57700eb64fbf9 WatchSource:0}: Error finding container 22ec42f6994129e3995512ec69665e173ad6889764f944ddd4e57700eb64fbf9: Status 404 returned error can't find the container with id 22ec42f6994129e3995512ec69665e173ad6889764f944ddd4e57700eb64fbf9 Jul 10 16:40:11.430798 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:11.430768 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8" event={"ID":"52d1ab13-506d-4b17-a95d-2177134bedbe","Type":"ContainerStarted","Data":"788fa46489f1961df47803f0c5d5bbc8c3d6ed18c256efe2a7e812e6f6e41113"} Jul 10 16:40:11.431183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:11.430815 2560 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8" event={"ID":"52d1ab13-506d-4b17-a95d-2177134bedbe","Type":"ContainerStarted","Data":"22ec42f6994129e3995512ec69665e173ad6889764f944ddd4e57700eb64fbf9"} Jul 10 16:40:11.431183 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:11.430858 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8" Jul 10 16:40:11.446969 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:11.446918 2560 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8" podStartSLOduration=2.446904364 podStartE2EDuration="2.446904364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-10 16:40:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-10 16:40:11.445406281 +0000 UTC m=+2700.417439657" watchObservedRunningTime="2026-07-10 16:40:11.446904364 +0000 UTC m=+2700.418937742" Jul 10 16:40:11.664989 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:11.664963 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6s2sn_a7a2569d-7313-49a4-b283-aa416f5fdd9e/ovn-acl-logging/0.log" Jul 10 16:40:11.676285 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:11.676263 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6s2sn_a7a2569d-7313-49a4-b283-aa416f5fdd9e/ovn-acl-logging/0.log" Jul 10 16:40:12.457911 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:12.457884 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4g9ns_0159b035-3359-4e09-bb64-9237da0c327c/dns/0.log" Jul 10 16:40:12.480005 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:12.479975 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4g9ns_0159b035-3359-4e09-bb64-9237da0c327c/kube-rbac-proxy/0.log" Jul 10 16:40:12.658691 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:12.658663 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ngcrw_81e1bb75-b227-4f4a-81d4-6843cebccc61/dns-node-resolver/0.log" Jul 10 16:40:13.171968 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:13.171933 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6b954bbc8f-7tjtk_60bc3007-ed90-4dff-8a98-4dfd79ab3965/registry/0.log" Jul 10 16:40:13.218226 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:13.218199 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-8vfqs_18946076-1464-4aac-bbe1-3c4d2b4b6ae9/node-ca/0.log" Jul 10 16:40:14.222281 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:14.222253 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5f67m_6205ec9a-afd1-4e7d-97b6-56b60c00847a/discovery/0.log" Jul 10 16:40:14.243805 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:14.243779 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b74489749-cvbvn_4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8/kube-auth-proxy/0.log" Jul 10 16:40:15.124115 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:15.124086 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-22xxq_47620ce7-6020-406a-9b19-92c89d6fbcc2/serve-healthcheck-canary/0.log" Jul 10 16:40:15.669058 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:15.669030 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fl48s_aaffad33-edc2-43c8-b65d-424739fd004e/kube-rbac-proxy/0.log" Jul 10 16:40:15.693141 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:15.693119 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fl48s_aaffad33-edc2-43c8-b65d-424739fd004e/exporter/0.log" Jul 10 16:40:15.718131 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:15.718107 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fl48s_aaffad33-edc2-43c8-b65d-424739fd004e/extractor/0.log" Jul 10 16:40:17.445390 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:17.445363 2560 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-lf2rd/perf-node-gather-daemonset-66xz8" Jul 10 16:40:17.962213 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:17.962188 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-65stt_447098e2-a5fe-4c36-833b-59631e845e53/manager/0.log" Jul 10 16:40:18.290825 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:18.290744 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-f9w9q_86b97add-9066-4d71-bdac-4f9f923d55f5/manager/0.log" Jul 10 16:40:18.315914 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:18.315885 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-765dfd8876-858s5_02f79ace-8bc2-4499-9551-5218dad2e06a/manager/0.log" Jul 10 16:40:18.339363 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:18.339341 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-765dfd8876-858s5_02f79ace-8bc2-4499-9551-5218dad2e06a/copy-manifests/0.log" Jul 10 16:40:20.621873 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:20.621843 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5f67m_6205ec9a-afd1-4e7d-97b6-56b60c00847a/discovery/0.log" Jul 10 16:40:20.632234 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:20.632212 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b74489749-cvbvn_4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8/kube-auth-proxy/0.log" Jul 10 16:40:22.041213 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:22.041173 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-5f67m_6205ec9a-afd1-4e7d-97b6-56b60c00847a/discovery/0.log" Jul 10 16:40:22.054571 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:22.054544 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b74489749-cvbvn_4381cadb-1ee8-4ef5-8e83-4b7a5bebb1f8/kube-auth-proxy/0.log" Jul 10 16:40:25.132197 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:25.132161 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-n5ppz_436e585c-0f4c-49a0-a6b0-d9a7eed47fab/migrator/0.log" Jul 10 16:40:25.154256 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:25.154221 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-n5ppz_436e585c-0f4c-49a0-a6b0-d9a7eed47fab/graceful-termination/0.log" Jul 10 16:40:26.674875 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:26.674847 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cfcr7_92d76a94-65a8-457e-8a57-bc3729beeb80/kube-multus-additional-cni-plugins/0.log" Jul 10 16:40:26.719946 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:26.719915 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cfcr7_92d76a94-65a8-457e-8a57-bc3729beeb80/egress-router-binary-copy/0.log" Jul 10 16:40:26.760719 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:26.760691 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cfcr7_92d76a94-65a8-457e-8a57-bc3729beeb80/cni-plugins/0.log" Jul 10 16:40:26.794706 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:26.794674 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cfcr7_92d76a94-65a8-457e-8a57-bc3729beeb80/bond-cni-plugin/0.log" Jul 10 16:40:26.833792 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:26.833768 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cfcr7_92d76a94-65a8-457e-8a57-bc3729beeb80/routeoverride-cni/0.log" Jul 10 16:40:26.876761 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:26.876730 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cfcr7_92d76a94-65a8-457e-8a57-bc3729beeb80/whereabouts-cni-bincopy/0.log" Jul 10 16:40:26.904689 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:26.904666 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cfcr7_92d76a94-65a8-457e-8a57-bc3729beeb80/whereabouts-cni/0.log" Jul 10 16:40:27.288181 ip-10-0-129-101 kubenswrapper[2560]: I0710 16:40:27.288150 2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-z6k5g_017c05d8-db4b-427e-b6de-19e1d1264b97/kube-multus/0.log"