Jul 01 22:19:21.357783 ip-10-0-129-238 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 01 22:19:21.357798 ip-10-0-129-238 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 01 22:19:21.357809 ip-10-0-129-238 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 01 22:19:21.358160 ip-10-0-129-238 systemd[1]: Failed to start Kubernetes Kubelet. Jul 01 22:19:30.879249 ip-10-0-129-238 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Jul 01 22:19:30.879265 ip-10-0-129-238 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 4d418641569d4e4b8860f7ae36e80066 -- Jul 01 22:20:59.134888 ip-10-0-129-238 systemd[1]: Starting Kubernetes Kubelet... Jul 01 22:20:59.599878 ip-10-0-129-238 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 22:20:59.599878 ip-10-0-129-238 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 01 22:20:59.599878 ip-10-0-129-238 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 22:20:59.599878 ip-10-0-129-238 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 01 22:20:59.599878 ip-10-0-129-238 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 22:20:59.599878 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.594035 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 01 22:20:59.599878 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599165 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 22:20:59.599878 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599176 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 22:20:59.599878 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599180 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 22:20:59.599878 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599183 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 22:20:59.599878 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599188 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 22:20:59.599878 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599193 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 22:20:59.599878 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599196 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 22:20:59.599878 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599199 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 22:20:59.614063 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599202 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 01 22:20:59.614063 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599204 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 22:20:59.614063 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599208 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 22:20:59.614063 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599210 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 22:20:59.614063 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599213 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 22:20:59.614063 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599216 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 22:20:59.614063 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599219 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 22:20:59.614063 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599222 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 22:20:59.614063 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599225 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 22:20:59.614063 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599227 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 22:20:59.614063 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599230 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 22:20:59.614063 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599234 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 22:20:59.614063 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599238 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 22:20:59.614063 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599241 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 22:20:59.614063 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599244 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 22:20:59.614063 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599247 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 22:20:59.614063 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599250 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 22:20:59.614063 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599252 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 22:20:59.614063 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599256 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 22:20:59.614063 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599258 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 22:20:59.616156 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599261 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 22:20:59.616156 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599264 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 22:20:59.616156 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599267 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 22:20:59.616156 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599269 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 22:20:59.616156 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599272 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 22:20:59.616156 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599275 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 22:20:59.616156 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599277 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 22:20:59.616156 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599280 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 22:20:59.616156 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599282 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 22:20:59.616156 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599285 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 22:20:59.616156 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599287 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 22:20:59.616156 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599290 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 22:20:59.616156 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599292 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 22:20:59.616156 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599295 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 22:20:59.616156 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599299 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 22:20:59.616156 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599302 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 22:20:59.616156 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599305 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 22:20:59.616156 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599308 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 22:20:59.616156 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599311 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 22:20:59.616156 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599314 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 22:20:59.617904 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599317 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 22:20:59.617904 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599320 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 22:20:59.617904 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599324 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 22:20:59.617904 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599326 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 22:20:59.617904 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599329 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 22:20:59.617904 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599332 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 22:20:59.617904 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599335 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 22:20:59.617904 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599337 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 22:20:59.617904 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599340 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 22:20:59.617904 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599342 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 22:20:59.617904 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599345 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 22:20:59.617904 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599348 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 22:20:59.617904 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599350 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 22:20:59.617904 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599353 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 22:20:59.617904 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599356 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 22:20:59.617904 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599359 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 22:20:59.617904 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599362 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 22:20:59.617904 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599364 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 22:20:59.617904 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599367 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 22:20:59.620228 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599369 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 22:20:59.620228 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599372 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 22:20:59.620228 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599374 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 22:20:59.620228 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599376 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 22:20:59.620228 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599379 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 22:20:59.620228 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599381 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 22:20:59.620228 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599384 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 22:20:59.620228 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599386 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 22:20:59.620228 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599389 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 22:20:59.620228 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599392 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 22:20:59.620228 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599394 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 22:20:59.620228 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599397 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 22:20:59.620228 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599406 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 22:20:59.620228 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599409 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 22:20:59.620228 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599411 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 22:20:59.620228 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599414 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 22:20:59.620228 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599416 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 22:20:59.620228 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599419 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 22:20:59.620228 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599421 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 22:20:59.620228 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599424 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 22:20:59.634157 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599426 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 22:20:59.634157 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599429 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 22:20:59.634157 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599431 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 22:20:59.634157 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599434 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 22:20:59.634157 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599437 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 22:20:59.634157 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599440 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 22:20:59.634157 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599442 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 22:20:59.634157 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599445 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 22:20:59.634157 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599447 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 22:20:59.634157 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599450 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 22:20:59.634157 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599452 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 22:20:59.634157 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.599455 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 22:20:59.634157 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600158 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 22:20:59.634157 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600165 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 22:20:59.634157 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600168 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 22:20:59.634157 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600170 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 22:20:59.634157 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600173 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 22:20:59.634157 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600175 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 22:20:59.634157 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600179 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 22:20:59.636312 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600182 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 22:20:59.636312 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600184 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 22:20:59.636312 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600187 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 22:20:59.636312 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600189 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 22:20:59.636312 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600192 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 01 22:20:59.636312 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600194 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 22:20:59.636312 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600198 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 22:20:59.636312 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600202 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 22:20:59.636312 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600206 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 22:20:59.636312 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600209 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 22:20:59.636312 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600212 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 22:20:59.636312 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600214 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 22:20:59.636312 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600217 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 22:20:59.636312 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600220 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 22:20:59.636312 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600222 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 22:20:59.636312 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600225 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 22:20:59.636312 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600228 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 22:20:59.636312 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600230 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 22:20:59.636312 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600233 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 22:20:59.636312 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600236 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 22:20:59.638187 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600238 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 22:20:59.638187 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600241 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 22:20:59.638187 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600244 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 22:20:59.638187 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600247 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 22:20:59.638187 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600250 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 22:20:59.638187 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600252 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 22:20:59.638187 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600256 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 22:20:59.638187 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600260 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 22:20:59.638187 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600262 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 22:20:59.638187 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600265 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 22:20:59.638187 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600267 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 22:20:59.638187 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600270 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 22:20:59.638187 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600272 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 22:20:59.638187 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600275 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 22:20:59.638187 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600277 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 22:20:59.638187 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600280 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 22:20:59.638187 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600282 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 22:20:59.638187 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600284 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 22:20:59.638187 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600287 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 22:20:59.640541 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600291 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 22:20:59.640541 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600293 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 22:20:59.640541 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600296 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 22:20:59.640541 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600298 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 22:20:59.640541 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600301 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 22:20:59.640541 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600303 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 22:20:59.640541 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600306 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 22:20:59.640541 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600308 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 22:20:59.640541 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600311 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 22:20:59.640541 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600313 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 22:20:59.640541 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600316 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 22:20:59.640541 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600319 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 22:20:59.640541 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600321 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 22:20:59.640541 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600323 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 22:20:59.640541 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600326 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 22:20:59.640541 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600329 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 22:20:59.640541 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600332 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 22:20:59.640541 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600334 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 22:20:59.640541 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600338 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 22:20:59.640541 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600341 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 22:20:59.706795 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600344 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 22:20:59.706795 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600346 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 22:20:59.706795 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600349 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 22:20:59.706795 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600352 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 22:20:59.706795 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600355 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 22:20:59.706795 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600358 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 22:20:59.706795 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600361 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 22:20:59.706795 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600364 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 22:20:59.706795 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600366 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 22:20:59.706795 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600369 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 22:20:59.706795 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600372 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 22:20:59.706795 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600374 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 22:20:59.706795 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600377 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 22:20:59.706795 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600380 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 22:20:59.706795 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600383 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 22:20:59.706795 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600386 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 22:20:59.706795 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600390 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 22:20:59.706795 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600393 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 22:20:59.706795 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600395 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 22:20:59.661894 ip-10-0-129-238 systemd[1]: Started Kubernetes Kubelet. Jul 01 22:20:59.886774 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600398 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 22:20:59.886774 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600401 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 22:20:59.886774 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600403 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 22:20:59.886774 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600406 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 22:20:59.886774 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600408 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 22:20:59.886774 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600411 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 22:20:59.886774 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600413 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 22:20:59.886774 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600416 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 22:20:59.886774 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600419 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 22:20:59.886774 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600421 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 22:20:59.886774 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600424 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 22:20:59.886774 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600427 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 22:20:59.886774 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600429 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 22:20:59.886774 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600432 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 22:20:59.886774 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600793 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 22:20:59.886774 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600798 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 22:20:59.886774 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600801 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 22:20:59.886774 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600804 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 22:20:59.886774 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600807 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 22:20:59.886774 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600810 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 22:20:59.900428 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600812 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 22:20:59.900428 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600815 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 22:20:59.900428 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600817 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 22:20:59.900428 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600820 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 22:20:59.900428 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600823 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 22:20:59.900428 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600826 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 22:20:59.900428 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600829 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 22:20:59.900428 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600831 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 22:20:59.900428 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600834 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 22:20:59.900428 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600837 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 22:20:59.900428 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600839 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 22:20:59.900428 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600842 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 22:20:59.900428 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600844 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 22:20:59.900428 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600847 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 22:20:59.900428 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600850 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 22:20:59.900428 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600852 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 22:20:59.900428 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600854 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 22:20:59.900428 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600857 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 22:20:59.900428 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600861 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 22:20:59.900428 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600863 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 22:20:59.901242 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600866 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 22:20:59.901242 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600869 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 22:20:59.901242 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600872 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 22:20:59.901242 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600874 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 22:20:59.901242 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600877 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 22:20:59.901242 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600880 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 22:20:59.901242 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600882 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 22:20:59.901242 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600885 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 22:20:59.901242 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600887 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 22:20:59.901242 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600890 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 22:20:59.901242 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600892 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 22:20:59.901242 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600895 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 22:20:59.901242 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600898 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 22:20:59.901242 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600901 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 22:20:59.901242 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600904 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 22:20:59.901242 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600906 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 22:20:59.901242 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600908 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 22:20:59.901242 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600911 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 22:20:59.901242 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600914 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 22:20:59.901242 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600917 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 22:20:59.901885 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600919 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 22:20:59.901885 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600922 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 22:20:59.901885 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600924 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 22:20:59.901885 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600927 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 22:20:59.901885 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600929 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 22:20:59.901885 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600932 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 22:20:59.901885 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600934 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 22:20:59.901885 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600937 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 22:20:59.901885 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600939 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 22:20:59.901885 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600941 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 22:20:59.901885 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600945 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 22:20:59.901885 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600948 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 22:20:59.901885 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600952 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 22:20:59.901885 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600955 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 22:20:59.901885 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600959 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 22:20:59.901885 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600961 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 22:20:59.901885 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600964 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 22:20:59.901885 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600967 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 22:20:59.901885 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600969 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 22:20:59.901885 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600972 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 22:20:59.902512 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600974 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 22:20:59.902512 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600977 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 22:20:59.902512 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600979 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 22:20:59.902512 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600983 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 22:20:59.902512 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600987 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 22:20:59.902512 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600990 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 22:20:59.902512 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600993 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 22:20:59.902512 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600996 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 22:20:59.902512 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.600998 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 22:20:59.902512 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601001 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 22:20:59.902512 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601004 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 22:20:59.902512 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601007 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 22:20:59.902512 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601023 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 22:20:59.902512 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601027 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 22:20:59.902512 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601029 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 22:20:59.902512 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601032 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 22:20:59.902512 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601034 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 22:20:59.902512 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601037 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 22:20:59.902512 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601040 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 22:20:59.903101 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601042 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 22:20:59.903101 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601045 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 22:20:59.903101 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601048 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 22:20:59.903101 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601051 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 22:20:59.903101 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601054 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 22:20:59.903101 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601058 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 22:20:59.903101 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601060 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 22:20:59.903101 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601063 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 22:20:59.903101 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601065 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 22:20:59.903101 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601069 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 22:20:59.903101 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601072 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 22:20:59.903101 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601075 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 01 22:20:59.903101 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601078 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 22:20:59.903101 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601081 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 22:20:59.903101 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601157 2580 flags.go:64] FLAG: --address="0.0.0.0" Jul 01 22:20:59.903101 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601172 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 01 22:20:59.903101 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601182 2580 flags.go:64] FLAG: --anonymous-auth="true" Jul 01 22:20:59.903101 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601188 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 01 22:20:59.903101 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601193 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 01 22:20:59.903101 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601197 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 01 22:20:59.903101 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601204 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601209 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601212 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601215 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601219 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601222 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601226 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601229 2580 flags.go:64] FLAG: --cgroup-root="" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601232 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601235 2580 flags.go:64] FLAG: --client-ca-file="" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601238 2580 flags.go:64] FLAG: --cloud-provider="external" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601241 2580 flags.go:64] FLAG: --cluster-dns="[]" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601245 2580 flags.go:64] FLAG: --cluster-domain="" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601248 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601251 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601256 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601259 2580 flags.go:64] FLAG: --container-log-max-files="5" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601264 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601267 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601270 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601274 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601277 2580 flags.go:64] FLAG: --contention-profiling="false" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601280 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601283 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 01 22:20:59.903801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601286 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601289 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601293 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601296 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601299 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601302 2580 flags.go:64] FLAG: --enable-load-reader="false" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601305 2580 flags.go:64] FLAG: --enable-server="true" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601307 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601312 2580 flags.go:64] FLAG: --event-burst="100" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601315 2580 flags.go:64] FLAG: --event-qps="50" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601318 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601321 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601323 2580 flags.go:64] FLAG: --eviction-hard="" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601327 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601330 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601333 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601336 2580 flags.go:64] FLAG: --eviction-soft="" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601339 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601342 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601345 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601348 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601351 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601354 2580 flags.go:64] FLAG: --fail-swap-on="true" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601358 2580 flags.go:64] FLAG: --feature-gates="" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601362 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jul 01 22:20:59.904462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601366 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601369 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601372 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601375 2580 flags.go:64] FLAG: --healthz-port="10248" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601378 2580 flags.go:64] FLAG: --help="false" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601381 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-129-238.ec2.internal" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601384 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601387 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601390 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601393 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601396 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601399 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601402 2580 flags.go:64] FLAG: --image-service-endpoint="" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601405 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601408 2580 flags.go:64] FLAG: --kube-api-burst="100" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601411 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601414 2580 flags.go:64] FLAG: --kube-api-qps="50" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601416 2580 flags.go:64] FLAG: --kube-reserved="" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601419 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601422 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601425 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601428 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601431 2580 flags.go:64] FLAG: --lock-file="" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601434 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 01 22:20:59.905276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601437 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 01 22:20:59.905985 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601439 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 01 22:20:59.905985 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601444 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jul 01 22:20:59.905985 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601447 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 01 22:20:59.905985 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601450 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jul 01 22:20:59.905985 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601453 2580 flags.go:64] FLAG: --logging-format="text" Jul 01 22:20:59.905985 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601457 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 01 22:20:59.905985 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601460 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 01 22:20:59.905985 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601464 2580 flags.go:64] FLAG: --manifest-url="" Jul 01 22:20:59.905985 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601467 2580 flags.go:64] FLAG: --manifest-url-header="" Jul 01 22:20:59.905985 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601471 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 01 22:20:59.905985 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601474 2580 flags.go:64] FLAG: --max-open-files="1000000" Jul 01 22:20:59.905985 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601478 2580 flags.go:64] FLAG: --max-pods="110" Jul 01 22:20:59.905985 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601481 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 01 22:20:59.905985 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601484 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 01 22:20:59.905985 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601487 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jul 01 22:20:59.905985 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601490 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 01 22:20:59.905985 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601493 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 01 22:20:59.905985 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601496 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 01 22:20:59.905985 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601499 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 01 22:20:59.905985 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601505 2580 flags.go:64] FLAG: --node-status-max-images="50" Jul 01 22:20:59.905985 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601508 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 01 22:20:59.905985 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601512 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jul 01 22:20:59.905985 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601515 2580 flags.go:64] FLAG: --pod-cidr="" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601518 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601523 2580 flags.go:64] FLAG: --pod-manifest-path="" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601525 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601528 2580 flags.go:64] FLAG: --pods-per-core="0" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601531 2580 flags.go:64] FLAG: --port="10250" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601534 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601537 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d14f6287dbb091e8" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601540 2580 flags.go:64] FLAG: --qos-reserved="" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601544 2580 flags.go:64] FLAG: --read-only-port="10255" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601547 2580 flags.go:64] FLAG: --register-node="true" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601563 2580 flags.go:64] FLAG: --register-with-taints="" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601566 2580 flags.go:64] FLAG: --registry-burst="10" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601569 2580 flags.go:64] FLAG: --registry-qps="5" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601572 2580 flags.go:64] FLAG: --reserved-cpus="" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601575 2580 flags.go:64] FLAG: --reserved-memory="" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601580 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601584 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601587 2580 flags.go:64] FLAG: --rotate-certificates="false" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601590 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601593 2580 flags.go:64] FLAG: --runonce="false" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601596 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601599 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601602 2580 flags.go:64] FLAG: --seccomp-default="false" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601605 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 01 22:20:59.906739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601608 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601611 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601614 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601617 2580 flags.go:64] FLAG: --storage-driver-password="root" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601620 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601623 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601626 2580 flags.go:64] FLAG: --storage-driver-user="root" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601628 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601631 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601634 2580 flags.go:64] FLAG: --system-cgroups="" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601637 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601643 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601646 2580 flags.go:64] FLAG: --tls-cert-file="" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601649 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601652 2580 flags.go:64] FLAG: --tls-min-version="" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601655 2580 flags.go:64] FLAG: --tls-private-key-file="" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601658 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601661 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601664 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601667 2580 flags.go:64] FLAG: --v="2" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601671 2580 flags.go:64] FLAG: --version="false" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601674 2580 flags.go:64] FLAG: --vmodule="" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601678 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.601682 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601809 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 22:20:59.907435 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601813 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 22:20:59.908213 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601817 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 22:20:59.908213 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601820 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 22:20:59.908213 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601823 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 22:20:59.908213 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601826 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 22:20:59.908213 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601828 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 22:20:59.908213 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601831 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 22:20:59.908213 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601834 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 22:20:59.908213 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601836 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 22:20:59.908213 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601839 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 22:20:59.908213 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601842 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 22:20:59.908213 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601844 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 22:20:59.908213 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601847 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 22:20:59.908213 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601850 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 22:20:59.908213 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601853 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 22:20:59.908213 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601855 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 22:20:59.908213 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601857 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 22:20:59.908213 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601860 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 22:20:59.908213 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601862 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 22:20:59.908213 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601865 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 22:20:59.908213 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601867 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 22:20:59.908828 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601870 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 01 22:20:59.908828 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601872 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 22:20:59.908828 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601875 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 22:20:59.908828 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601877 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 22:20:59.908828 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601880 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 22:20:59.908828 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601883 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 22:20:59.908828 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601885 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 22:20:59.908828 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601888 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 22:20:59.908828 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601892 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 22:20:59.908828 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601897 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 22:20:59.908828 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601901 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 22:20:59.908828 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601904 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 22:20:59.908828 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601907 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 22:20:59.908828 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601910 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 22:20:59.908828 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601915 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 22:20:59.908828 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601917 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 22:20:59.908828 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601921 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 22:20:59.908828 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601924 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 22:20:59.908828 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601927 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 22:20:59.909535 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601930 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 22:20:59.909535 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601934 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 22:20:59.909535 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601937 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 22:20:59.909535 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601939 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 22:20:59.909535 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601942 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 22:20:59.909535 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601944 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 22:20:59.909535 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601947 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 22:20:59.909535 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601950 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 22:20:59.909535 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601952 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 22:20:59.909535 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601955 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 22:20:59.909535 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601957 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 22:20:59.909535 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601960 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 22:20:59.909535 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601962 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 22:20:59.909535 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601965 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 22:20:59.909535 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601967 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 22:20:59.909535 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601970 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 22:20:59.909535 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601972 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 22:20:59.909535 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601975 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 22:20:59.909535 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601978 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 22:20:59.909535 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601981 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 22:20:59.910110 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601984 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 22:20:59.910110 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601986 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 22:20:59.910110 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601991 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 22:20:59.910110 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601995 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 22:20:59.910110 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.601998 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 22:20:59.910110 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602001 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 22:20:59.910110 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602003 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 22:20:59.910110 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602006 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 22:20:59.910110 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602009 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 22:20:59.910110 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602011 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 22:20:59.910110 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602014 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 22:20:59.910110 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602016 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 22:20:59.910110 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602018 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 22:20:59.910110 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602022 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 22:20:59.910110 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602026 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 22:20:59.910110 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602029 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 22:20:59.910110 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602032 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 22:20:59.910110 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602035 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 22:20:59.910110 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602037 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 22:20:59.910110 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602040 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 22:20:59.910732 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602042 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 22:20:59.910732 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602045 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 22:20:59.910732 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602048 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 22:20:59.910732 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602050 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 22:20:59.910732 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602053 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 22:20:59.910732 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602055 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 22:20:59.910732 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602058 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 22:20:59.910732 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602060 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 22:20:59.910732 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602063 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 22:20:59.910732 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602065 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 22:20:59.910732 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602068 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 22:20:59.910732 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602071 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 22:20:59.910732 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602073 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 22:20:59.910732 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602076 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 22:20:59.910732 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602080 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 22:20:59.910732 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602084 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 22:20:59.910732 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602087 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 22:20:59.910732 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.602090 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 22:20:59.911345 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.602978 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 22:20:59.911345 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.610347 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 01 22:20:59.911345 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.610361 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 01 22:20:59.911345 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.610380 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 01 22:20:59.911345 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.610389 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 01 22:20:59.911345 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610442 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 22:20:59.911345 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610447 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 22:20:59.911345 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610451 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 22:20:59.911345 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610454 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 22:20:59.911345 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610457 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 22:20:59.911345 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610460 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 22:20:59.911345 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610463 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 22:20:59.911345 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610466 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 22:20:59.911345 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610470 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 22:20:59.911345 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610472 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 22:20:59.911948 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610475 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 22:20:59.911948 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610478 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 22:20:59.911948 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610481 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 22:20:59.911948 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610484 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 22:20:59.911948 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610487 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 22:20:59.911948 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610490 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 22:20:59.911948 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610493 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 22:20:59.911948 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610495 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 22:20:59.911948 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610498 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 22:20:59.911948 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610500 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 22:20:59.911948 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610503 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 22:20:59.911948 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610506 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 22:20:59.911948 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610509 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 22:20:59.911948 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610512 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 22:20:59.911948 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610514 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 22:20:59.911948 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610519 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 22:20:59.911948 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610522 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 22:20:59.911948 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610525 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 22:20:59.911948 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610527 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 22:20:59.911948 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610530 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 22:20:59.912560 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610532 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 22:20:59.912560 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610535 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 22:20:59.912560 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610537 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 22:20:59.912560 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610540 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 22:20:59.912560 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610542 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 22:20:59.912560 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610545 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 22:20:59.912560 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610548 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 22:20:59.912560 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610565 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 22:20:59.912560 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610568 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 22:20:59.912560 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610570 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 22:20:59.912560 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610573 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 22:20:59.912560 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610577 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 22:20:59.912560 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610580 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 22:20:59.912560 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610583 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 22:20:59.912560 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610586 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 22:20:59.912560 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610589 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 22:20:59.912560 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610592 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 22:20:59.912560 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610595 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 22:20:59.912560 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610597 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 22:20:59.913160 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610601 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 22:20:59.913160 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610605 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 22:20:59.913160 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610608 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 22:20:59.913160 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610611 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 22:20:59.913160 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610615 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 22:20:59.913160 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610619 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 22:20:59.913160 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610622 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 22:20:59.913160 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610624 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 22:20:59.913160 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610628 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 22:20:59.913160 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610631 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 22:20:59.913160 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610634 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 22:20:59.913160 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610637 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 22:20:59.913160 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610641 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 22:20:59.913160 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610644 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 22:20:59.913160 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610646 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 22:20:59.913160 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610649 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 22:20:59.913160 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610651 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 22:20:59.913160 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610654 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 22:20:59.913160 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610656 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 22:21:00.090937 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610659 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 22:21:00.090937 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610662 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 22:21:00.090937 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610664 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 22:21:00.090937 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610667 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 22:21:00.090937 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610669 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 22:21:00.090937 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610672 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 22:21:00.090937 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610674 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 22:21:00.090937 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610677 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 22:21:00.090937 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610680 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 22:21:00.090937 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610682 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 22:21:00.090937 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610685 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 22:21:00.090937 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610688 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 22:21:00.090937 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610691 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 01 22:21:00.090937 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610693 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 22:21:00.090937 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610696 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 22:21:00.090937 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610699 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 22:21:00.090937 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610701 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 22:21:00.090937 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610703 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 22:21:00.090937 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610706 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 22:21:00.090937 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610709 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 22:21:00.091943 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610711 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 22:21:00.091943 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610715 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 22:21:00.091943 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610718 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 22:21:00.091943 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610721 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 22:21:00.091943 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610723 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 22:21:00.091943 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610727 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 22:21:00.091943 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610730 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 22:21:00.091943 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610732 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 22:21:00.091943 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610735 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 22:21:00.091943 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610737 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 22:21:00.091943 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610739 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 22:21:00.091943 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.610744 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 22:21:00.091943 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610851 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 22:21:00.091943 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610856 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 22:21:00.091943 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610859 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 22:21:00.093634 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610862 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 22:21:00.093634 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610864 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 22:21:00.093634 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610867 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 22:21:00.093634 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610870 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 22:21:00.093634 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610873 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 22:21:00.093634 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610876 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 01 22:21:00.093634 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610879 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 22:21:00.093634 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610881 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 22:21:00.093634 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610884 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 22:21:00.093634 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610887 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 22:21:00.093634 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610890 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 22:21:00.093634 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610893 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 22:21:00.093634 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610895 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 22:21:00.093634 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610897 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 22:21:00.093634 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610900 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 22:21:00.093634 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610902 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 22:21:00.093634 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610905 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 22:21:00.093634 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610908 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 22:21:00.093634 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610910 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 22:21:00.093634 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610913 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 22:21:00.093634 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610916 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 22:21:00.094220 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610919 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 22:21:00.094220 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610922 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 22:21:00.094220 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610925 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 22:21:00.094220 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610928 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 22:21:00.094220 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610930 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 22:21:00.094220 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610933 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 22:21:00.094220 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610936 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 22:21:00.094220 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610938 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 22:21:00.094220 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610941 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 22:21:00.094220 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610945 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 22:21:00.094220 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610949 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 22:21:00.094220 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610952 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 22:21:00.094220 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610954 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 22:21:00.094220 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610958 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 22:21:00.094220 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610961 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 22:21:00.094220 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610964 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 22:21:00.094220 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610967 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 22:21:00.094220 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610969 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 22:21:00.094220 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610972 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 22:21:00.094734 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610974 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 22:21:00.094734 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610977 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 22:21:00.094734 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610980 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 22:21:00.094734 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610982 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 22:21:00.094734 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610985 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 22:21:00.094734 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610987 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 22:21:00.094734 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610990 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 22:21:00.094734 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610992 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 22:21:00.094734 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610995 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 22:21:00.094734 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.610997 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 22:21:00.094734 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611000 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 22:21:00.094734 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611004 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 22:21:00.094734 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611006 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 22:21:00.094734 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611009 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 22:21:00.094734 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611013 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 22:21:00.094734 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611016 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 22:21:00.094734 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611019 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 22:21:00.094734 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611022 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 22:21:00.094734 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611025 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 22:21:00.095271 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611027 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 22:21:00.095271 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611030 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 22:21:00.095271 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611033 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 22:21:00.095271 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611035 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 22:21:00.095271 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611038 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 22:21:00.095271 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611040 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 22:21:00.095271 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611043 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 22:21:00.095271 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611046 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 22:21:00.095271 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611048 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 22:21:00.095271 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611051 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 22:21:00.095271 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611053 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 22:21:00.095271 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611056 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 22:21:00.095271 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611060 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 22:21:00.095271 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611063 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 22:21:00.095271 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611066 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 22:21:00.095271 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611069 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 22:21:00.095271 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611073 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 22:21:00.095271 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611076 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 22:21:00.095271 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611079 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 22:21:00.095823 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611081 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 22:21:00.095823 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611084 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 22:21:00.095823 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611086 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 22:21:00.095823 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611089 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 22:21:00.095823 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611091 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 22:21:00.095823 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611096 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 22:21:00.095823 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611098 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 22:21:00.095823 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611101 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 22:21:00.095823 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611105 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 22:21:00.095823 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611107 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 22:21:00.095823 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611110 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 22:21:00.095823 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611112 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 22:21:00.095823 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611115 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 22:21:00.095823 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611117 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 22:21:00.095823 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611120 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 22:21:00.095823 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611123 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 22:21:00.095823 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611125 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 22:21:00.095823 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:20:59.611128 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 22:21:00.096328 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.611132 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 22:21:00.096328 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.611221 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jul 01 22:21:00.096328 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.615760 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 01 22:21:00.096328 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.617179 2580 server.go:1019] "Starting client certificate rotation" Jul 01 22:21:00.096328 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.617277 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 22:21:00.096328 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.617311 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 22:21:00.096328 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.619832 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 22:21:00.096328 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.620912 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 22:21:00.096328 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.628297 2580 log.go:25] "Validated CRI v1 runtime API" Jul 01 22:21:00.096328 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.630333 2580 log.go:25] "Validated CRI v1 image API" Jul 01 22:21:00.096328 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.631137 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 01 22:21:00.096328 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.633812 2580 fs.go:135] Filesystem UUIDs: map[108fc52f-4bc0-4ca4-bd10-4537f701de36:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 96b4c9c0-371c-4d96-93da-dc74e94cad96:/dev/nvme0n1p4] Jul 01 22:21:00.096702 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.633835 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 01 22:21:00.096802 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.641107 2580 manager.go:216] Machine: {Timestamp:2026-07-01 22:20:59.640652771 +0000 UTC m=+0.388093352 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100456 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d2c3677220a0e225b294f25705541 SystemUUID:ec2d2c36-7722-0a0e-225b-294f25705541 BootID:4d418641-569d-4e4b-8860-f7ae36e80066 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0a:cc:22:23:61 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0a:cc:22:23:61 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:2c:1f:1d:ec:38 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 01 22:21:00.096802 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.641197 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 01 22:21:00.096802 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.641262 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 01 22:21:00.096802 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.643310 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 01 22:21:00.096802 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.643336 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-238.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 01 22:21:00.096802 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.644933 2580 topology_manager.go:138] "Creating topology manager with none policy" Jul 01 22:21:00.096802 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.644944 2580 container_manager_linux.go:309] "Creating device plugin manager" Jul 01 22:21:00.096802 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.644952 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.645849 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.646613 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.649101 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.649221 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.649250 2580 kubelet.go:486] "Attempting to sync node with API server" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.649258 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.649271 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.649280 2580 kubelet.go:398] "Adding apiserver pod source" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.649312 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.650731 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.650955 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.650993 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.654118 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.656004 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.656040 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.656671 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.656683 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.656689 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.656694 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.656703 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.656708 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.656968 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.656974 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.656981 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.656987 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.656995 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.657002 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.658078 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.658085 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.661158 2580 server.go:1268] "Started kubelet" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.661252 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.661306 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.662111 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.662279 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.664538 2580 server.go:310] "Adding debug handlers to kubelet server" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:20:59.667599 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:20:59.667731 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-238.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:20:59.667749 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 01 22:21:00.097083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.667814 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-238.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.668006 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.668019 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:20:59.668120 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-238.ec2.internal\" not found" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.668702 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.668903 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.668916 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.669031 2580 reconstruct.go:97] "Volume reconstruction finished" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.669037 2580 reconciler.go:29] "Reconciler: start to sync state" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:20:59.667769 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-238.ec2.internal.18be4a8a245815ec default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-238.ec2.internal,UID:ip-10-0-129-238.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-238.ec2.internal,},FirstTimestamp:2026-07-01 22:20:59.660989932 +0000 UTC m=+0.408430513,LastTimestamp:2026-07-01 22:20:59.660989932 +0000 UTC m=+0.408430513,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-238.ec2.internal,}" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.669460 2580 factory.go:55] Registering systemd factory Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.669475 2580 factory.go:223] Registration of the systemd container factory successfully Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.669732 2580 factory.go:153] Registering CRI-O factory Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.669745 2580 factory.go:223] Registration of the crio container factory successfully Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.669793 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.669816 2580 factory.go:103] Registering Raw factory Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.669831 2580 manager.go:1196] Started watching for new ooms in manager Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.670504 2580 manager.go:349] Starting recovery of all containers Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:20:59.674996 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-238.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:20:59.675169 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.682928 2580 manager.go:354] Recovery completed Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.689275 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.691379 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-238.ec2.internal" event="NodeHasSufficientMemory" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.691400 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-238.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.691409 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-238.ec2.internal" event="NodeHasSufficientPID" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.691874 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.691882 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.691884 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.691895 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.692571 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-238.ec2.internal" event="NodeHasSufficientMemory" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.692594 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-238.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.692604 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-238.ec2.internal" event="NodeHasSufficientPID" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.692844 2580 policy_none.go:49] "None policy: Start" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.692868 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 01 22:21:00.239419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.692877 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 01 22:21:00.240940 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:20:59.693362 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-238.ec2.internal.18be4a8a2627fa0c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-238.ec2.internal,UID:ip-10-0-129-238.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-238.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-238.ec2.internal,},FirstTimestamp:2026-07-01 22:20:59.6913915 +0000 UTC m=+0.438832081,LastTimestamp:2026-07-01 22:20:59.6913915 +0000 UTC m=+0.438832081,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-238.ec2.internal,}" Jul 01 22:21:00.240940 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.693751 2580 policy_none.go:47] "Start" Jul 01 22:21:00.240940 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.700270 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fspm7" Jul 01 22:21:00.240940 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:20:59.706951 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-238.ec2.internal.18be4a8a26282b67 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-238.ec2.internal,UID:ip-10-0-129-238.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-129-238.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-129-238.ec2.internal,},FirstTimestamp:2026-07-01 22:20:59.691404135 +0000 UTC m=+0.438844716,LastTimestamp:2026-07-01 22:20:59.691404135 +0000 UTC m=+0.438844716,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-238.ec2.internal,}" Jul 01 22:21:00.240940 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.707755 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fspm7" Jul 01 22:21:00.240940 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:20:59.708920 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-238.ec2.internal.18be4a8a26284d9c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-238.ec2.internal,UID:ip-10-0-129-238.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-129-238.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-129-238.ec2.internal,},FirstTimestamp:2026-07-01 22:20:59.691412892 +0000 UTC m=+0.438853472,LastTimestamp:2026-07-01 22:20:59.691412892 +0000 UTC m=+0.438853472,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-238.ec2.internal,}" Jul 01 22:21:00.240940 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.737102 2580 manager.go:337] "Starting Device Plugin manager" Jul 01 22:21:00.240940 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:20:59.737132 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 01 22:21:00.240940 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.737141 2580 server.go:87] "Starting device plugin registration server" Jul 01 22:21:00.240940 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.737329 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 01 22:21:00.240940 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.737794 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 01 22:21:00.240940 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.737891 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 01 22:21:00.240940 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.737961 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 01 22:21:00.240940 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.737969 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 01 22:21:00.240940 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:20:59.738468 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 01 22:21:00.240940 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:20:59.738518 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-238.ec2.internal\" not found" Jul 01 22:21:00.240940 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.739318 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 01 22:21:00.240940 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.740792 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 01 22:21:00.240940 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.740829 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 01 22:21:00.240940 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.740857 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:20:59.740891 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.742996 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.838741 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.839513 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-238.ec2.internal" event="NodeHasSufficientMemory" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.839537 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-238.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.839547 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-238.ec2.internal" event="NodeHasSufficientPID" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.839583 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-238.ec2.internal" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.841643 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-238.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-238.ec2.internal"] Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.841693 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.842416 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-238.ec2.internal" event="NodeHasSufficientMemory" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.842438 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-238.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.842452 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-238.ec2.internal" event="NodeHasSufficientPID" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.843641 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.843761 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-238.ec2.internal" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.843791 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.844284 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-238.ec2.internal" event="NodeHasSufficientMemory" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.844311 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-238.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.844284 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-238.ec2.internal" event="NodeHasSufficientMemory" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.844322 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-238.ec2.internal" event="NodeHasSufficientPID" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.844337 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-238.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.844349 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-238.ec2.internal" event="NodeHasSufficientPID" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.845389 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-238.ec2.internal" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.845413 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.846142 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-238.ec2.internal" event="NodeHasSufficientMemory" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.846164 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-238.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.846179 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-238.ec2.internal" event="NodeHasSufficientPID" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.846786 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-238.ec2.internal" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:20:59.846806 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-238.ec2.internal\": node \"ip-10-0-129-238.ec2.internal\" not found" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.870224 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d73a9c7b8912df8d8409030b111886c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-238.ec2.internal\" (UID: \"9d73a9c7b8912df8d8409030b111886c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-238.ec2.internal" Jul 01 22:21:00.241546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.870243 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7d254b613571f06d3af95550e0820f84-config\") pod \"kube-apiserver-proxy-ip-10-0-129-238.ec2.internal\" (UID: \"7d254b613571f06d3af95550e0820f84\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-238.ec2.internal" Jul 01 22:21:00.242377 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.870259 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9d73a9c7b8912df8d8409030b111886c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-238.ec2.internal\" (UID: \"9d73a9c7b8912df8d8409030b111886c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-238.ec2.internal" Jul 01 22:21:00.242377 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:20:59.877062 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-238.ec2.internal\" not found" node="ip-10-0-129-238.ec2.internal" Jul 01 22:21:00.242377 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:20:59.877214 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-238.ec2.internal\" not found" Jul 01 22:21:00.242377 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:20:59.881874 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-238.ec2.internal\" not found" node="ip-10-0-129-238.ec2.internal" Jul 01 22:21:00.242377 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.971228 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7d254b613571f06d3af95550e0820f84-config\") pod \"kube-apiserver-proxy-ip-10-0-129-238.ec2.internal\" (UID: \"7d254b613571f06d3af95550e0820f84\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-238.ec2.internal" Jul 01 22:21:00.242377 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.971248 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9d73a9c7b8912df8d8409030b111886c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-238.ec2.internal\" (UID: \"9d73a9c7b8912df8d8409030b111886c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-238.ec2.internal" Jul 01 22:21:00.242377 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.971264 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d73a9c7b8912df8d8409030b111886c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-238.ec2.internal\" (UID: \"9d73a9c7b8912df8d8409030b111886c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-238.ec2.internal" Jul 01 22:21:00.242377 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.971295 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d73a9c7b8912df8d8409030b111886c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-238.ec2.internal\" (UID: \"9d73a9c7b8912df8d8409030b111886c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-238.ec2.internal" Jul 01 22:21:00.242377 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.971330 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7d254b613571f06d3af95550e0820f84-config\") pod \"kube-apiserver-proxy-ip-10-0-129-238.ec2.internal\" (UID: \"7d254b613571f06d3af95550e0820f84\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-238.ec2.internal" Jul 01 22:21:00.242377 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:20:59.971358 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9d73a9c7b8912df8d8409030b111886c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-238.ec2.internal\" (UID: \"9d73a9c7b8912df8d8409030b111886c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-238.ec2.internal" Jul 01 22:21:00.242377 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:20:59.977259 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-238.ec2.internal\" not found" Jul 01 22:21:00.242377 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:00.078074 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-238.ec2.internal\" not found" Jul 01 22:21:00.242377 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:00.179150 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-238.ec2.internal\" not found" Jul 01 22:21:00.242377 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:00.181289 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-238.ec2.internal" Jul 01 22:21:00.242377 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:00.186137 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-238.ec2.internal" Jul 01 22:21:00.409601 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:00.279242 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-238.ec2.internal\" not found" Jul 01 22:21:00.409601 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:00.379826 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-238.ec2.internal\" not found" Jul 01 22:21:00.595663 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:00.479966 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-238.ec2.internal\" not found" Jul 01 22:21:00.595663 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:00.495165 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 22:21:00.595663 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:00.581041 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-238.ec2.internal\" not found" Jul 01 22:21:00.781148 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:00.617970 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 01 22:21:00.781148 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:00.618084 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 01 22:21:00.781148 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:00.618129 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 01 22:21:00.781148 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:00.681222 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-238.ec2.internal\" not found" Jul 01 22:21:00.781148 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:00.711457 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-30 22:16:00 +0000 UTC" deadline="2028-03-15 11:45:02.563706128 +0000 UTC" Jul 01 22:21:00.781148 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:00.711480 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14941h24m1.852229436s" Jul 01 22:21:00.782685 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:00.782311 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-238.ec2.internal\" not found" Jul 01 22:21:00.939503 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:00.828954 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 22:21:00.939503 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:00.868376 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-238.ec2.internal" Jul 01 22:21:00.939503 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:00.885977 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 22:21:00.939503 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:00.888182 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-238.ec2.internal" Jul 01 22:21:00.939503 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:00.898097 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 22:21:01.134078 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:21:00.984221 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d73a9c7b8912df8d8409030b111886c.slice/crio-49e76d35acd9deb12dfe13e2eb533c61eabe4c60cfffd388e9001615aecc8484 WatchSource:0}: Error finding container 49e76d35acd9deb12dfe13e2eb533c61eabe4c60cfffd388e9001615aecc8484: Status 404 returned error can't find the container with id 49e76d35acd9deb12dfe13e2eb533c61eabe4c60cfffd388e9001615aecc8484 Jul 01 22:21:01.134078 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:21:00.984669 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d254b613571f06d3af95550e0820f84.slice/crio-e34c158e772780d71b958ba56bee877761dcd28713c780440691f391746461f3 WatchSource:0}: Error finding container e34c158e772780d71b958ba56bee877761dcd28713c780440691f391746461f3: Status 404 returned error can't find the container with id e34c158e772780d71b958ba56bee877761dcd28713c780440691f391746461f3 Jul 01 22:21:01.134078 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:00.990034 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 22:21:01.340638 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.152246 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 22:21:01.654599 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.652833 2580 apiserver.go:52] "Watching apiserver" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.661896 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.662327 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-ws2ns","openshift-image-registry/node-ca-wmmbz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-238.ec2.internal","openshift-multus/network-metrics-daemon-kfd7x","kube-system/global-pull-secret-syncer-d5xct","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj","openshift-multus/multus-additional-cni-plugins-p44zj","openshift-multus/multus-d4j26","openshift-network-diagnostics/network-check-target-8tjzj","openshift-network-operator/iptables-alerter-qfqxw","openshift-ovn-kubernetes/ovnkube-node-bwsfz","kube-system/konnectivity-agent-vz9dp","kube-system/kube-apiserver-proxy-ip-10-0-129-238.ec2.internal","openshift-cluster-node-tuning-operator/tuned-jxmpz"] Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.665394 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.667814 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.668720 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.669127 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.669474 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.670258 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.678042 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.678497 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.678757 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.678769 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.679055 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.679234 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8k29c\"" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.679283 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.679925 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.680387 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.680793 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.681049 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4kq7k\"" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.681583 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.681899 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-76dxq\"" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.683729 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.684980 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:01.685064 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfd7x" podUID="246c052d-11a5-4032-9505-a1628e462a19" Jul 01 22:21:01.727738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.685664 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 22:21:01.729338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.685860 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4e8d667b-1ec1-4147-9238-6f016882b971-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.729338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.685894 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxftz\" (UniqueName: \"kubernetes.io/projected/73baab49-4c50-4980-b761-08d3abeb05a8-kube-api-access-hxftz\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.729338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.685919 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpflj\" (UniqueName: \"kubernetes.io/projected/4e8d667b-1ec1-4147-9238-6f016882b971-kube-api-access-xpflj\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.729338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.685961 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-host-slash\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.729338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686006 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4e8d667b-1ec1-4147-9238-6f016882b971-os-release\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.729338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686051 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4e8d667b-1ec1-4147-9238-6f016882b971-cni-binary-copy\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.729338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686086 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4e8d667b-1ec1-4147-9238-6f016882b971-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.729338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686113 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-log-socket\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.729338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686137 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-host-cni-bin\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.729338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686173 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-sys\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.729338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686197 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-host-kubelet\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.729338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686220 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-run-systemd\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.729338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686244 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-var-lib-openvswitch\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.738208 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686269 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-run-openvswitch\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.738208 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686337 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.738208 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686371 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/71d59fe0-f377-4e68-825e-bec682a08714-etc-tuned\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.738208 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686396 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71d59fe0-f377-4e68-825e-bec682a08714-tmp\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.738208 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686420 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e8d667b-1ec1-4147-9238-6f016882b971-system-cni-dir\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.738208 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686444 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4e8d667b-1ec1-4147-9238-6f016882b971-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.738208 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686493 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-etc-sysconfig\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.738208 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686517 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-etc-systemd\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.738208 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686569 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-var-lib-kubelet\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.738208 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686600 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-host-cni-netd\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.738208 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686621 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-run\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.738208 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686642 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-lib-modules\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.738208 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686663 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-host\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.738208 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686688 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.739137 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686741 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-host-run-netns\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.739137 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686777 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/73baab49-4c50-4980-b761-08d3abeb05a8-ovn-node-metrics-cert\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.739137 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686803 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-etc-modprobe-d\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.739137 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686835 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lp7z\" (UniqueName: \"kubernetes.io/projected/71d59fe0-f377-4e68-825e-bec682a08714-kube-api-access-4lp7z\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.739137 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686858 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-etc-openvswitch\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.739137 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686876 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-node-log\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.739137 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686895 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/73baab49-4c50-4980-b761-08d3abeb05a8-env-overrides\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.739137 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686929 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/73baab49-4c50-4980-b761-08d3abeb05a8-ovnkube-script-lib\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.739137 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686947 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-etc-kubernetes\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.739137 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686962 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-etc-sysctl-conf\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.739137 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.686984 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-systemd-units\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.739137 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.687004 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4e8d667b-1ec1-4147-9238-6f016882b971-cnibin\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.739137 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.687018 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-run-ovn\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.739137 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.687033 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/73baab49-4c50-4980-b761-08d3abeb05a8-ovnkube-config\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.687048 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-etc-sysctl-d\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.688028 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:01.688085 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d5xct" podUID="f700798a-ec9b-4ee0-8a62-d5ff885135cd" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.690925 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.693704 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:01.693763 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tjzj" podUID="bebea64f-bddc-4c5a-8c28-837fb09e93c5" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.694583 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.694763 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.694936 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.695174 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mk7ds\"" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.696983 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.699300 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wmmbz" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.699677 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.700310 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-77jt9\"" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.702067 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qfqxw" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.702896 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.703836 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.704009 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.704613 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ws2ns" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.705398 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.705433 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kkgdg\"" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.705582 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jl9qj\"" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.706086 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.706651 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.707075 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.707174 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-p8mdv\"" Jul 01 22:21:01.744524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.707743 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 01 22:21:01.746411 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.709623 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vz9dp" Jul 01 22:21:01.746411 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.710365 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-vqbk2" Jul 01 22:21:01.746411 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.712225 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 01 22:21:01.746411 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.712225 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bqqd9\"" Jul 01 22:21:01.746411 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.712305 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 01 22:21:01.746411 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.719442 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-vqbk2" Jul 01 22:21:01.751282 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.750920 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-238.ec2.internal" event={"ID":"9d73a9c7b8912df8d8409030b111886c","Type":"ContainerStarted","Data":"49e76d35acd9deb12dfe13e2eb533c61eabe4c60cfffd388e9001615aecc8484"} Jul 01 22:21:01.751933 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.751426 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 22:21:01.813963 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.753126 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-238.ec2.internal" event={"ID":"7d254b613571f06d3af95550e0820f84","Type":"ContainerStarted","Data":"e34c158e772780d71b958ba56bee877761dcd28713c780440691f391746461f3"} Jul 01 22:21:01.813963 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.769883 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 01 22:21:01.813963 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787389 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-multus-socket-dir-parent\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.813963 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787426 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-host-var-lib-cni-multus\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.813963 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787460 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgzk2\" (UniqueName: \"kubernetes.io/projected/f7388d18-cc3b-407c-98ee-5543c413b77e-kube-api-access-xgzk2\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.813963 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787486 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret\") pod \"global-pull-secret-syncer-d5xct\" (UID: \"f700798a-ec9b-4ee0-8a62-d5ff885135cd\") " pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:01.813963 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787510 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e0220c87-2759-46c6-aa51-8dcb525f8be1-socket-dir\") pod \"aws-ebs-csi-driver-node-9qzlj\" (UID: \"e0220c87-2759-46c6-aa51-8dcb525f8be1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" Jul 01 22:21:01.813963 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787537 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-cnibin\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.813963 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787566 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-log-socket\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.813963 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787608 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-sys\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.813963 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787631 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-etc-kubernetes\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.813963 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787656 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hdq2\" (UniqueName: \"kubernetes.io/projected/1ae00714-cbd8-4816-b57a-2568b3356575-kube-api-access-5hdq2\") pod \"iptables-alerter-qfqxw\" (UID: \"1ae00714-cbd8-4816-b57a-2568b3356575\") " pod="openshift-network-operator/iptables-alerter-qfqxw" Jul 01 22:21:01.813963 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787683 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-run-systemd\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.813963 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787709 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-var-lib-openvswitch\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.813963 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787733 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-run-openvswitch\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.817338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787745 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-sys\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.817338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787766 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71d59fe0-f377-4e68-825e-bec682a08714-tmp\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.817338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787792 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8694157d-ebfe-44aa-ae98-4ef443d09ec2-konnectivity-ca\") pod \"konnectivity-agent-vz9dp\" (UID: \"8694157d-ebfe-44aa-ae98-4ef443d09ec2\") " pod="kube-system/konnectivity-agent-vz9dp" Jul 01 22:21:01.817338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787815 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f700798a-ec9b-4ee0-8a62-d5ff885135cd-dbus\") pod \"global-pull-secret-syncer-d5xct\" (UID: \"f700798a-ec9b-4ee0-8a62-d5ff885135cd\") " pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:01.817338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787843 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e8d667b-1ec1-4147-9238-6f016882b971-system-cni-dir\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.817338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787894 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-run-systemd\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.817338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787911 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4e8d667b-1ec1-4147-9238-6f016882b971-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.817338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787948 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-var-lib-kubelet\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.817338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787953 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-run-openvswitch\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.817338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787975 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-host-var-lib-cni-bin\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.817338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788002 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-host-run-multus-certs\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.817338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788028 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e0220c87-2759-46c6-aa51-8dcb525f8be1-registration-dir\") pod \"aws-ebs-csi-driver-node-9qzlj\" (UID: \"e0220c87-2759-46c6-aa51-8dcb525f8be1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" Jul 01 22:21:01.817338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788053 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-host\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.817338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788077 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f7388d18-cc3b-407c-98ee-5543c413b77e-multus-daemon-config\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.818172 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788100 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e0220c87-2759-46c6-aa51-8dcb525f8be1-device-dir\") pod \"aws-ebs-csi-driver-node-9qzlj\" (UID: \"e0220c87-2759-46c6-aa51-8dcb525f8be1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" Jul 01 22:21:01.818172 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788123 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fbbd6b24-4540-40b3-b0e8-a9d0b67559f5-host\") pod \"node-ca-wmmbz\" (UID: \"fbbd6b24-4540-40b3-b0e8-a9d0b67559f5\") " pod="openshift-image-registry/node-ca-wmmbz" Jul 01 22:21:01.818172 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788152 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.818172 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788178 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-host-run-netns\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.818172 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788223 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/73baab49-4c50-4980-b761-08d3abeb05a8-ovn-node-metrics-cert\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.818172 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788251 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lp7z\" (UniqueName: \"kubernetes.io/projected/71d59fe0-f377-4e68-825e-bec682a08714-kube-api-access-4lp7z\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.818172 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788252 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 01 22:21:01.818172 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788286 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-etc-openvswitch\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.818172 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788311 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-node-log\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.818172 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788335 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/73baab49-4c50-4980-b761-08d3abeb05a8-ovnkube-script-lib\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.818172 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788364 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-etc-sysctl-conf\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.818172 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788390 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-host-run-k8s-cni-cncf-io\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.818172 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788427 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1ae00714-cbd8-4816-b57a-2568b3356575-iptables-alerter-script\") pod \"iptables-alerter-qfqxw\" (UID: \"1ae00714-cbd8-4816-b57a-2568b3356575\") " pod="openshift-network-operator/iptables-alerter-qfqxw" Jul 01 22:21:01.818172 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788452 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs\") pod \"network-metrics-daemon-kfd7x\" (UID: \"246c052d-11a5-4032-9505-a1628e462a19\") " pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:01.820444 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788478 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztfxg\" (UniqueName: \"kubernetes.io/projected/246c052d-11a5-4032-9505-a1628e462a19-kube-api-access-ztfxg\") pod \"network-metrics-daemon-kfd7x\" (UID: \"246c052d-11a5-4032-9505-a1628e462a19\") " pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:01.820444 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788507 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-systemd-units\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.820444 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788535 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4e8d667b-1ec1-4147-9238-6f016882b971-cnibin\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.820444 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788559 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-run-ovn\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.820444 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788600 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/73baab49-4c50-4980-b761-08d3abeb05a8-ovnkube-config\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.820444 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788626 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slktq\" (UniqueName: \"kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq\") pod \"network-check-target-8tjzj\" (UID: \"bebea64f-bddc-4c5a-8c28-837fb09e93c5\") " pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:01.820444 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788660 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-host-run-netns\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.820444 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788671 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4e8d667b-1ec1-4147-9238-6f016882b971-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.820444 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.787815 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-log-socket\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.820444 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788700 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxftz\" (UniqueName: \"kubernetes.io/projected/73baab49-4c50-4980-b761-08d3abeb05a8-kube-api-access-hxftz\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.820444 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788726 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-system-cni-dir\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.820444 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788759 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8694157d-ebfe-44aa-ae98-4ef443d09ec2-agent-certs\") pod \"konnectivity-agent-vz9dp\" (UID: \"8694157d-ebfe-44aa-ae98-4ef443d09ec2\") " pod="kube-system/konnectivity-agent-vz9dp" Jul 01 22:21:01.820444 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788766 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4e8d667b-1ec1-4147-9238-6f016882b971-system-cni-dir\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.820444 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788784 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/71d59fe0-f377-4e68-825e-bec682a08714-etc-tuned\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.821135 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788811 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpflj\" (UniqueName: \"kubernetes.io/projected/4e8d667b-1ec1-4147-9238-6f016882b971-kube-api-access-xpflj\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.821135 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788840 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-host-slash\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.821135 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788866 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4e8d667b-1ec1-4147-9238-6f016882b971-os-release\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.821135 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788896 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4e8d667b-1ec1-4147-9238-6f016882b971-cni-binary-copy\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.821135 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788912 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4e8d667b-1ec1-4147-9238-6f016882b971-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.821135 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788923 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4e8d667b-1ec1-4147-9238-6f016882b971-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.821135 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788975 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-var-lib-kubelet\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.821135 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789008 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-host-cni-bin\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.821135 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789040 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/33732b5f-89f5-4e70-a763-cfb5cda678e0-hosts-file\") pod \"node-resolver-ws2ns\" (UID: \"33732b5f-89f5-4e70-a763-cfb5cda678e0\") " pod="openshift-dns/node-resolver-ws2ns" Jul 01 22:21:01.821135 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789068 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-host-kubelet\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.821135 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789094 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.821135 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789123 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-multus-cni-dir\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.821135 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789150 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-host-run-netns\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.821135 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789175 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-multus-conf-dir\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.823445 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789201 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33732b5f-89f5-4e70-a763-cfb5cda678e0-tmp-dir\") pod \"node-resolver-ws2ns\" (UID: \"33732b5f-89f5-4e70-a763-cfb5cda678e0\") " pod="openshift-dns/node-resolver-ws2ns" Jul 01 22:21:01.823445 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789227 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f700798a-ec9b-4ee0-8a62-d5ff885135cd-kubelet-config\") pod \"global-pull-secret-syncer-d5xct\" (UID: \"f700798a-ec9b-4ee0-8a62-d5ff885135cd\") " pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:01.823445 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789253 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e0220c87-2759-46c6-aa51-8dcb525f8be1-etc-selinux\") pod \"aws-ebs-csi-driver-node-9qzlj\" (UID: \"e0220c87-2759-46c6-aa51-8dcb525f8be1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" Jul 01 22:21:01.823445 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789281 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-etc-sysconfig\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.823445 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789308 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-etc-systemd\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.823445 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789335 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-os-release\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.823445 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789361 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1ae00714-cbd8-4816-b57a-2568b3356575-host-slash\") pod \"iptables-alerter-qfqxw\" (UID: \"1ae00714-cbd8-4816-b57a-2568b3356575\") " pod="openshift-network-operator/iptables-alerter-qfqxw" Jul 01 22:21:01.823445 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789387 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e0220c87-2759-46c6-aa51-8dcb525f8be1-sys-fs\") pod \"aws-ebs-csi-driver-node-9qzlj\" (UID: \"e0220c87-2759-46c6-aa51-8dcb525f8be1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" Jul 01 22:21:01.823445 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789415 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j224p\" (UniqueName: \"kubernetes.io/projected/fbbd6b24-4540-40b3-b0e8-a9d0b67559f5-kube-api-access-j224p\") pod \"node-ca-wmmbz\" (UID: \"fbbd6b24-4540-40b3-b0e8-a9d0b67559f5\") " pod="openshift-image-registry/node-ca-wmmbz" Jul 01 22:21:01.823445 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789445 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-host-cni-netd\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.823445 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789472 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-run\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.823445 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789499 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-lib-modules\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.823445 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789544 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lc5j\" (UniqueName: \"kubernetes.io/projected/33732b5f-89f5-4e70-a763-cfb5cda678e0-kube-api-access-8lc5j\") pod \"node-resolver-ws2ns\" (UID: \"33732b5f-89f5-4e70-a763-cfb5cda678e0\") " pod="openshift-dns/node-resolver-ws2ns" Jul 01 22:21:01.823445 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789572 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e0220c87-2759-46c6-aa51-8dcb525f8be1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9qzlj\" (UID: \"e0220c87-2759-46c6-aa51-8dcb525f8be1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" Jul 01 22:21:01.824129 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789601 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-etc-modprobe-d\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.824129 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789653 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/73baab49-4c50-4980-b761-08d3abeb05a8-env-overrides\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.824129 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789682 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-etc-kubernetes\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.824129 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789708 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-hostroot\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.824129 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789735 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-etc-sysctl-d\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.824129 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789761 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7388d18-cc3b-407c-98ee-5543c413b77e-cni-binary-copy\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.824129 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789788 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-host-var-lib-kubelet\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.824129 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789815 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f58zh\" (UniqueName: \"kubernetes.io/projected/e0220c87-2759-46c6-aa51-8dcb525f8be1-kube-api-access-f58zh\") pod \"aws-ebs-csi-driver-node-9qzlj\" (UID: \"e0220c87-2759-46c6-aa51-8dcb525f8be1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" Jul 01 22:21:01.824129 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789840 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fbbd6b24-4540-40b3-b0e8-a9d0b67559f5-serviceca\") pod \"node-ca-wmmbz\" (UID: \"fbbd6b24-4540-40b3-b0e8-a9d0b67559f5\") " pod="openshift-image-registry/node-ca-wmmbz" Jul 01 22:21:01.824129 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.790022 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-etc-modprobe-d\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.824129 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.790088 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-etc-systemd\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.824129 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.790155 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-host-cni-netd\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.824129 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.789508 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4e8d667b-1ec1-4147-9238-6f016882b971-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.824129 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.790199 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-etc-kubernetes\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.827389 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.790257 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.827389 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.790268 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-host-kubelet\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.827389 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.790271 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-etc-sysctl-d\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.827389 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.790301 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-host-run-ovn-kubernetes\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.827389 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.790336 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/73baab49-4c50-4980-b761-08d3abeb05a8-env-overrides\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.827389 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.790345 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-host\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.827389 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.790351 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-run\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.827389 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.790343 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-host-cni-bin\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.827389 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.788727 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-var-lib-openvswitch\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.827389 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.790394 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-run-ovn\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.827389 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.790422 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-lib-modules\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.827389 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.790476 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4e8d667b-1ec1-4147-9238-6f016882b971-cnibin\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.827389 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.790926 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4e8d667b-1ec1-4147-9238-6f016882b971-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.827389 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.790930 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/73baab49-4c50-4980-b761-08d3abeb05a8-ovnkube-script-lib\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.827389 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.791012 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/73baab49-4c50-4980-b761-08d3abeb05a8-ovnkube-config\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.828533 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.791067 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-etc-openvswitch\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.828533 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.791094 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-systemd-units\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.828533 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.791124 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4e8d667b-1ec1-4147-9238-6f016882b971-os-release\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.828533 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.791176 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-node-log\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.828533 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.791184 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-etc-sysconfig\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.828533 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.791235 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/73baab49-4c50-4980-b761-08d3abeb05a8-host-slash\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.828533 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.791292 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/71d59fe0-f377-4e68-825e-bec682a08714-etc-sysctl-conf\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.828533 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.791364 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4e8d667b-1ec1-4147-9238-6f016882b971-cni-binary-copy\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.828533 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.794466 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71d59fe0-f377-4e68-825e-bec682a08714-tmp\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.828533 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.794844 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/73baab49-4c50-4980-b761-08d3abeb05a8-ovn-node-metrics-cert\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.828533 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.795611 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/71d59fe0-f377-4e68-825e-bec682a08714-etc-tuned\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.828533 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.803842 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lp7z\" (UniqueName: \"kubernetes.io/projected/71d59fe0-f377-4e68-825e-bec682a08714-kube-api-access-4lp7z\") pod \"tuned-jxmpz\" (UID: \"71d59fe0-f377-4e68-825e-bec682a08714\") " pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:01.828533 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.804167 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpflj\" (UniqueName: \"kubernetes.io/projected/4e8d667b-1ec1-4147-9238-6f016882b971-kube-api-access-xpflj\") pod \"multus-additional-cni-plugins-p44zj\" (UID: \"4e8d667b-1ec1-4147-9238-6f016882b971\") " pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:01.828533 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.805423 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxftz\" (UniqueName: \"kubernetes.io/projected/73baab49-4c50-4980-b761-08d3abeb05a8-kube-api-access-hxftz\") pod \"ovnkube-node-bwsfz\" (UID: \"73baab49-4c50-4980-b761-08d3abeb05a8\") " pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:01.914494 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.890613 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slktq\" (UniqueName: \"kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq\") pod \"network-check-target-8tjzj\" (UID: \"bebea64f-bddc-4c5a-8c28-837fb09e93c5\") " pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:01.914494 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.890653 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-system-cni-dir\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.914494 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.890678 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8694157d-ebfe-44aa-ae98-4ef443d09ec2-agent-certs\") pod \"konnectivity-agent-vz9dp\" (UID: \"8694157d-ebfe-44aa-ae98-4ef443d09ec2\") " pod="kube-system/konnectivity-agent-vz9dp" Jul 01 22:21:01.914494 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.890708 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/33732b5f-89f5-4e70-a763-cfb5cda678e0-hosts-file\") pod \"node-resolver-ws2ns\" (UID: \"33732b5f-89f5-4e70-a763-cfb5cda678e0\") " pod="openshift-dns/node-resolver-ws2ns" Jul 01 22:21:01.914494 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.890735 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-multus-cni-dir\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.914494 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.890761 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-host-run-netns\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.914494 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.890785 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-multus-conf-dir\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.914494 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.890809 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33732b5f-89f5-4e70-a763-cfb5cda678e0-tmp-dir\") pod \"node-resolver-ws2ns\" (UID: \"33732b5f-89f5-4e70-a763-cfb5cda678e0\") " pod="openshift-dns/node-resolver-ws2ns" Jul 01 22:21:01.914494 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.890858 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f700798a-ec9b-4ee0-8a62-d5ff885135cd-kubelet-config\") pod \"global-pull-secret-syncer-d5xct\" (UID: \"f700798a-ec9b-4ee0-8a62-d5ff885135cd\") " pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:01.914494 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.890885 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e0220c87-2759-46c6-aa51-8dcb525f8be1-etc-selinux\") pod \"aws-ebs-csi-driver-node-9qzlj\" (UID: \"e0220c87-2759-46c6-aa51-8dcb525f8be1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" Jul 01 22:21:01.914494 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.890910 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-os-release\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.914494 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.890933 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1ae00714-cbd8-4816-b57a-2568b3356575-host-slash\") pod \"iptables-alerter-qfqxw\" (UID: \"1ae00714-cbd8-4816-b57a-2568b3356575\") " pod="openshift-network-operator/iptables-alerter-qfqxw" Jul 01 22:21:01.914494 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.890957 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e0220c87-2759-46c6-aa51-8dcb525f8be1-sys-fs\") pod \"aws-ebs-csi-driver-node-9qzlj\" (UID: \"e0220c87-2759-46c6-aa51-8dcb525f8be1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" Jul 01 22:21:01.914494 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.890979 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-host-run-netns\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.914494 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.890981 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j224p\" (UniqueName: \"kubernetes.io/projected/fbbd6b24-4540-40b3-b0e8-a9d0b67559f5-kube-api-access-j224p\") pod \"node-ca-wmmbz\" (UID: \"fbbd6b24-4540-40b3-b0e8-a9d0b67559f5\") " pod="openshift-image-registry/node-ca-wmmbz" Jul 01 22:21:01.915420 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891040 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lc5j\" (UniqueName: \"kubernetes.io/projected/33732b5f-89f5-4e70-a763-cfb5cda678e0-kube-api-access-8lc5j\") pod \"node-resolver-ws2ns\" (UID: \"33732b5f-89f5-4e70-a763-cfb5cda678e0\") " pod="openshift-dns/node-resolver-ws2ns" Jul 01 22:21:01.915420 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891091 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-multus-conf-dir\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.915420 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891095 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e0220c87-2759-46c6-aa51-8dcb525f8be1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9qzlj\" (UID: \"e0220c87-2759-46c6-aa51-8dcb525f8be1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" Jul 01 22:21:01.915420 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891143 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-hostroot\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.915420 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891170 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7388d18-cc3b-407c-98ee-5543c413b77e-cni-binary-copy\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.915420 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891176 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-multus-cni-dir\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.915420 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891192 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-host-var-lib-kubelet\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.915420 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891220 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f58zh\" (UniqueName: \"kubernetes.io/projected/e0220c87-2759-46c6-aa51-8dcb525f8be1-kube-api-access-f58zh\") pod \"aws-ebs-csi-driver-node-9qzlj\" (UID: \"e0220c87-2759-46c6-aa51-8dcb525f8be1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" Jul 01 22:21:01.915420 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891245 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fbbd6b24-4540-40b3-b0e8-a9d0b67559f5-serviceca\") pod \"node-ca-wmmbz\" (UID: \"fbbd6b24-4540-40b3-b0e8-a9d0b67559f5\") " pod="openshift-image-registry/node-ca-wmmbz" Jul 01 22:21:01.915420 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891262 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1ae00714-cbd8-4816-b57a-2568b3356575-host-slash\") pod \"iptables-alerter-qfqxw\" (UID: \"1ae00714-cbd8-4816-b57a-2568b3356575\") " pod="openshift-network-operator/iptables-alerter-qfqxw" Jul 01 22:21:01.915420 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891269 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-multus-socket-dir-parent\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.915420 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891292 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-host-var-lib-cni-multus\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.915420 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891315 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgzk2\" (UniqueName: \"kubernetes.io/projected/f7388d18-cc3b-407c-98ee-5543c413b77e-kube-api-access-xgzk2\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.915420 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891332 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e0220c87-2759-46c6-aa51-8dcb525f8be1-sys-fs\") pod \"aws-ebs-csi-driver-node-9qzlj\" (UID: \"e0220c87-2759-46c6-aa51-8dcb525f8be1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" Jul 01 22:21:01.915420 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891340 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret\") pod \"global-pull-secret-syncer-d5xct\" (UID: \"f700798a-ec9b-4ee0-8a62-d5ff885135cd\") " pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:01.919368 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891365 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e0220c87-2759-46c6-aa51-8dcb525f8be1-socket-dir\") pod \"aws-ebs-csi-driver-node-9qzlj\" (UID: \"e0220c87-2759-46c6-aa51-8dcb525f8be1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" Jul 01 22:21:01.919368 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891373 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-host-var-lib-kubelet\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.919368 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891387 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-cnibin\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.919368 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891408 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/33732b5f-89f5-4e70-a763-cfb5cda678e0-tmp-dir\") pod \"node-resolver-ws2ns\" (UID: \"33732b5f-89f5-4e70-a763-cfb5cda678e0\") " pod="openshift-dns/node-resolver-ws2ns" Jul 01 22:21:01.919368 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891417 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-system-cni-dir\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.919368 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891456 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-etc-kubernetes\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.919368 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891412 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-etc-kubernetes\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.919368 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891498 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f700798a-ec9b-4ee0-8a62-d5ff885135cd-kubelet-config\") pod \"global-pull-secret-syncer-d5xct\" (UID: \"f700798a-ec9b-4ee0-8a62-d5ff885135cd\") " pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:01.919368 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891503 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hdq2\" (UniqueName: \"kubernetes.io/projected/1ae00714-cbd8-4816-b57a-2568b3356575-kube-api-access-5hdq2\") pod \"iptables-alerter-qfqxw\" (UID: \"1ae00714-cbd8-4816-b57a-2568b3356575\") " pod="openshift-network-operator/iptables-alerter-qfqxw" Jul 01 22:21:01.919368 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891500 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e0220c87-2759-46c6-aa51-8dcb525f8be1-etc-selinux\") pod \"aws-ebs-csi-driver-node-9qzlj\" (UID: \"e0220c87-2759-46c6-aa51-8dcb525f8be1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" Jul 01 22:21:01.919368 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891520 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-os-release\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.919368 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891539 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/33732b5f-89f5-4e70-a763-cfb5cda678e0-hosts-file\") pod \"node-resolver-ws2ns\" (UID: \"33732b5f-89f5-4e70-a763-cfb5cda678e0\") " pod="openshift-dns/node-resolver-ws2ns" Jul 01 22:21:01.919368 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891547 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8694157d-ebfe-44aa-ae98-4ef443d09ec2-konnectivity-ca\") pod \"konnectivity-agent-vz9dp\" (UID: \"8694157d-ebfe-44aa-ae98-4ef443d09ec2\") " pod="kube-system/konnectivity-agent-vz9dp" Jul 01 22:21:01.919368 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891625 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-multus-socket-dir-parent\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.919368 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891656 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-cnibin\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.920518 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891663 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-hostroot\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.920518 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891698 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f700798a-ec9b-4ee0-8a62-d5ff885135cd-dbus\") pod \"global-pull-secret-syncer-d5xct\" (UID: \"f700798a-ec9b-4ee0-8a62-d5ff885135cd\") " pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:01.920518 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891711 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-host-var-lib-cni-multus\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.920518 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891757 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-host-var-lib-cni-bin\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.920518 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891767 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f700798a-ec9b-4ee0-8a62-d5ff885135cd-dbus\") pod \"global-pull-secret-syncer-d5xct\" (UID: \"f700798a-ec9b-4ee0-8a62-d5ff885135cd\") " pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:01.920518 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891802 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-host-run-multus-certs\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.920518 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891809 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-host-var-lib-cni-bin\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.920518 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891828 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e0220c87-2759-46c6-aa51-8dcb525f8be1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-9qzlj\" (UID: \"e0220c87-2759-46c6-aa51-8dcb525f8be1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" Jul 01 22:21:01.920518 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891867 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e0220c87-2759-46c6-aa51-8dcb525f8be1-registration-dir\") pod \"aws-ebs-csi-driver-node-9qzlj\" (UID: \"e0220c87-2759-46c6-aa51-8dcb525f8be1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" Jul 01 22:21:01.920518 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891890 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e0220c87-2759-46c6-aa51-8dcb525f8be1-socket-dir\") pod \"aws-ebs-csi-driver-node-9qzlj\" (UID: \"e0220c87-2759-46c6-aa51-8dcb525f8be1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" Jul 01 22:21:01.920518 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891918 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e0220c87-2759-46c6-aa51-8dcb525f8be1-registration-dir\") pod \"aws-ebs-csi-driver-node-9qzlj\" (UID: \"e0220c87-2759-46c6-aa51-8dcb525f8be1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" Jul 01 22:21:01.920518 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891947 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f7388d18-cc3b-407c-98ee-5543c413b77e-multus-daemon-config\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.920518 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.891993 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e0220c87-2759-46c6-aa51-8dcb525f8be1-device-dir\") pod \"aws-ebs-csi-driver-node-9qzlj\" (UID: \"e0220c87-2759-46c6-aa51-8dcb525f8be1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" Jul 01 22:21:01.920518 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.892020 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fbbd6b24-4540-40b3-b0e8-a9d0b67559f5-host\") pod \"node-ca-wmmbz\" (UID: \"fbbd6b24-4540-40b3-b0e8-a9d0b67559f5\") " pod="openshift-image-registry/node-ca-wmmbz" Jul 01 22:21:01.920518 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.892051 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-host-run-multus-certs\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.921700 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.892117 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fbbd6b24-4540-40b3-b0e8-a9d0b67559f5-host\") pod \"node-ca-wmmbz\" (UID: \"fbbd6b24-4540-40b3-b0e8-a9d0b67559f5\") " pod="openshift-image-registry/node-ca-wmmbz" Jul 01 22:21:01.921700 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.892155 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8694157d-ebfe-44aa-ae98-4ef443d09ec2-konnectivity-ca\") pod \"konnectivity-agent-vz9dp\" (UID: \"8694157d-ebfe-44aa-ae98-4ef443d09ec2\") " pod="kube-system/konnectivity-agent-vz9dp" Jul 01 22:21:01.921700 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:01.892233 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 22:21:01.921700 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:01.892305 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret podName:f700798a-ec9b-4ee0-8a62-d5ff885135cd nodeName:}" failed. No retries permitted until 2026-07-01 22:21:02.392284086 +0000 UTC m=+3.139724668 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret") pod "global-pull-secret-syncer-d5xct" (UID: "f700798a-ec9b-4ee0-8a62-d5ff885135cd") : object "kube-system"/"original-pull-secret" not registered Jul 01 22:21:01.921700 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.892303 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/fbbd6b24-4540-40b3-b0e8-a9d0b67559f5-serviceca\") pod \"node-ca-wmmbz\" (UID: \"fbbd6b24-4540-40b3-b0e8-a9d0b67559f5\") " pod="openshift-image-registry/node-ca-wmmbz" Jul 01 22:21:01.921700 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.892370 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e0220c87-2759-46c6-aa51-8dcb525f8be1-device-dir\") pod \"aws-ebs-csi-driver-node-9qzlj\" (UID: \"e0220c87-2759-46c6-aa51-8dcb525f8be1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" Jul 01 22:21:01.921700 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.892376 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7388d18-cc3b-407c-98ee-5543c413b77e-cni-binary-copy\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.921700 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.892381 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-host-run-k8s-cni-cncf-io\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.921700 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.892416 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f7388d18-cc3b-407c-98ee-5543c413b77e-host-run-k8s-cni-cncf-io\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.921700 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.892422 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f7388d18-cc3b-407c-98ee-5543c413b77e-multus-daemon-config\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.921700 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.892445 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1ae00714-cbd8-4816-b57a-2568b3356575-iptables-alerter-script\") pod \"iptables-alerter-qfqxw\" (UID: \"1ae00714-cbd8-4816-b57a-2568b3356575\") " pod="openshift-network-operator/iptables-alerter-qfqxw" Jul 01 22:21:01.921700 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.892475 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs\") pod \"network-metrics-daemon-kfd7x\" (UID: \"246c052d-11a5-4032-9505-a1628e462a19\") " pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:01.921700 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.892502 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztfxg\" (UniqueName: \"kubernetes.io/projected/246c052d-11a5-4032-9505-a1628e462a19-kube-api-access-ztfxg\") pod \"network-metrics-daemon-kfd7x\" (UID: \"246c052d-11a5-4032-9505-a1628e462a19\") " pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:01.921700 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:01.892580 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:21:01.924100 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:01.892666 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs podName:246c052d-11a5-4032-9505-a1628e462a19 nodeName:}" failed. No retries permitted until 2026-07-01 22:21:02.392649328 +0000 UTC m=+3.140089909 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs") pod "network-metrics-daemon-kfd7x" (UID: "246c052d-11a5-4032-9505-a1628e462a19") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:21:01.924100 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.893000 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1ae00714-cbd8-4816-b57a-2568b3356575-iptables-alerter-script\") pod \"iptables-alerter-qfqxw\" (UID: \"1ae00714-cbd8-4816-b57a-2568b3356575\") " pod="openshift-network-operator/iptables-alerter-qfqxw" Jul 01 22:21:01.924100 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.895248 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8694157d-ebfe-44aa-ae98-4ef443d09ec2-agent-certs\") pod \"konnectivity-agent-vz9dp\" (UID: \"8694157d-ebfe-44aa-ae98-4ef443d09ec2\") " pod="kube-system/konnectivity-agent-vz9dp" Jul 01 22:21:01.924100 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:01.897451 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 22:21:01.924100 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:01.897470 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 22:21:01.924100 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:01.897485 2580 projected.go:196] Error preparing data for projected volume kube-api-access-slktq for pod openshift-network-diagnostics/network-check-target-8tjzj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:21:01.924100 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:01.897544 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq podName:bebea64f-bddc-4c5a-8c28-837fb09e93c5 nodeName:}" failed. No retries permitted until 2026-07-01 22:21:02.397528939 +0000 UTC m=+3.144969511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-slktq" (UniqueName: "kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq") pod "network-check-target-8tjzj" (UID: "bebea64f-bddc-4c5a-8c28-837fb09e93c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:21:01.924100 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.899937 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j224p\" (UniqueName: \"kubernetes.io/projected/fbbd6b24-4540-40b3-b0e8-a9d0b67559f5-kube-api-access-j224p\") pod \"node-ca-wmmbz\" (UID: \"fbbd6b24-4540-40b3-b0e8-a9d0b67559f5\") " pod="openshift-image-registry/node-ca-wmmbz" Jul 01 22:21:01.924100 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.901778 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f58zh\" (UniqueName: \"kubernetes.io/projected/e0220c87-2759-46c6-aa51-8dcb525f8be1-kube-api-access-f58zh\") pod \"aws-ebs-csi-driver-node-9qzlj\" (UID: \"e0220c87-2759-46c6-aa51-8dcb525f8be1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" Jul 01 22:21:01.924100 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.903265 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lc5j\" (UniqueName: \"kubernetes.io/projected/33732b5f-89f5-4e70-a763-cfb5cda678e0-kube-api-access-8lc5j\") pod \"node-resolver-ws2ns\" (UID: \"33732b5f-89f5-4e70-a763-cfb5cda678e0\") " pod="openshift-dns/node-resolver-ws2ns" Jul 01 22:21:01.924100 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.903757 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hdq2\" (UniqueName: \"kubernetes.io/projected/1ae00714-cbd8-4816-b57a-2568b3356575-kube-api-access-5hdq2\") pod \"iptables-alerter-qfqxw\" (UID: \"1ae00714-cbd8-4816-b57a-2568b3356575\") " pod="openshift-network-operator/iptables-alerter-qfqxw" Jul 01 22:21:01.924100 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.903998 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztfxg\" (UniqueName: \"kubernetes.io/projected/246c052d-11a5-4032-9505-a1628e462a19-kube-api-access-ztfxg\") pod \"network-metrics-daemon-kfd7x\" (UID: \"246c052d-11a5-4032-9505-a1628e462a19\") " pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:01.924100 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.906058 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgzk2\" (UniqueName: \"kubernetes.io/projected/f7388d18-cc3b-407c-98ee-5543c413b77e-kube-api-access-xgzk2\") pod \"multus-d4j26\" (UID: \"f7388d18-cc3b-407c-98ee-5543c413b77e\") " pod="openshift-multus/multus-d4j26" Jul 01 22:21:01.994891 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:01.988349 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p44zj" Jul 01 22:21:02.072878 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:02.001395 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:02.072878 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:02.010234 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" Jul 01 22:21:02.072878 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:02.016659 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" Jul 01 22:21:02.072878 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:02.025859 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d4j26" Jul 01 22:21:02.072878 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:02.036427 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wmmbz" Jul 01 22:21:02.072878 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:02.051713 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qfqxw" Jul 01 22:21:02.072878 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:02.063991 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ws2ns" Jul 01 22:21:02.083718 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:02.083699 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vz9dp" Jul 01 22:21:02.118475 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:21:02.118438 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbbd6b24_4540_40b3_b0e8_a9d0b67559f5.slice/crio-1674cb4e8a22c5229517cb7da5190791f4e36dbc25ebed1f1b052abaf91b82e8 WatchSource:0}: Error finding container 1674cb4e8a22c5229517cb7da5190791f4e36dbc25ebed1f1b052abaf91b82e8: Status 404 returned error can't find the container with id 1674cb4e8a22c5229517cb7da5190791f4e36dbc25ebed1f1b052abaf91b82e8 Jul 01 22:21:02.122578 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:21:02.122540 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33732b5f_89f5_4e70_a763_cfb5cda678e0.slice/crio-e81fc825c1431299a9e64ffaba7807f4bc8cf2e43ad9065372ea74be9e4cb640 WatchSource:0}: Error finding container e81fc825c1431299a9e64ffaba7807f4bc8cf2e43ad9065372ea74be9e4cb640: Status 404 returned error can't find the container with id e81fc825c1431299a9e64ffaba7807f4bc8cf2e43ad9065372ea74be9e4cb640 Jul 01 22:21:02.123663 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:21:02.123625 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8694157d_ebfe_44aa_ae98_4ef443d09ec2.slice/crio-485e993a6c0db7f67acd9efb124699757bc28ce5f9096e0dddd1d948688ef6af WatchSource:0}: Error finding container 485e993a6c0db7f67acd9efb124699757bc28ce5f9096e0dddd1d948688ef6af: Status 404 returned error can't find the container with id 485e993a6c0db7f67acd9efb124699757bc28ce5f9096e0dddd1d948688ef6af Jul 01 22:21:02.128014 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:21:02.127985 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ae00714_cbd8_4816_b57a_2568b3356575.slice/crio-3fced08aa6afc9f5f614ea35a860c30df641dcb1ad81ffcd4fd8b2d936aeb724 WatchSource:0}: Error finding container 3fced08aa6afc9f5f614ea35a860c30df641dcb1ad81ffcd4fd8b2d936aeb724: Status 404 returned error can't find the container with id 3fced08aa6afc9f5f614ea35a860c30df641dcb1ad81ffcd4fd8b2d936aeb724 Jul 01 22:21:02.395450 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:02.395361 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret\") pod \"global-pull-secret-syncer-d5xct\" (UID: \"f700798a-ec9b-4ee0-8a62-d5ff885135cd\") " pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:02.395450 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:02.395422 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs\") pod \"network-metrics-daemon-kfd7x\" (UID: \"246c052d-11a5-4032-9505-a1628e462a19\") " pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:02.395685 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:02.395528 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 22:21:02.395685 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:02.395550 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:21:02.395685 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:02.395612 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret podName:f700798a-ec9b-4ee0-8a62-d5ff885135cd nodeName:}" failed. No retries permitted until 2026-07-01 22:21:03.395589395 +0000 UTC m=+4.143029982 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret") pod "global-pull-secret-syncer-d5xct" (UID: "f700798a-ec9b-4ee0-8a62-d5ff885135cd") : object "kube-system"/"original-pull-secret" not registered Jul 01 22:21:02.395685 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:02.395637 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs podName:246c052d-11a5-4032-9505-a1628e462a19 nodeName:}" failed. No retries permitted until 2026-07-01 22:21:03.395624931 +0000 UTC m=+4.143065501 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs") pod "network-metrics-daemon-kfd7x" (UID: "246c052d-11a5-4032-9505-a1628e462a19") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:21:02.496465 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:02.496433 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slktq\" (UniqueName: \"kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq\") pod \"network-check-target-8tjzj\" (UID: \"bebea64f-bddc-4c5a-8c28-837fb09e93c5\") " pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:02.496664 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:02.496634 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 22:21:02.496664 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:02.496654 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 22:21:02.496664 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:02.496665 2580 projected.go:196] Error preparing data for projected volume kube-api-access-slktq for pod openshift-network-diagnostics/network-check-target-8tjzj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:21:02.496836 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:02.496729 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq podName:bebea64f-bddc-4c5a-8c28-837fb09e93c5 nodeName:}" failed. No retries permitted until 2026-07-01 22:21:03.49670839 +0000 UTC m=+4.244148984 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-slktq" (UniqueName: "kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq") pod "network-check-target-8tjzj" (UID: "bebea64f-bddc-4c5a-8c28-837fb09e93c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:21:02.723849 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:02.723722 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 22:16:02 +0000 UTC" deadline="2028-02-06 14:23:41.773809892 +0000 UTC" Jul 01 22:21:02.723849 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:02.723802 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14032h2m39.050012344s" Jul 01 22:21:02.756220 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:02.756181 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d4j26" event={"ID":"f7388d18-cc3b-407c-98ee-5543c413b77e","Type":"ContainerStarted","Data":"9e222cefeeacd702732984823c697b992ce17378abb3abb999e092e5f06e1eb5"} Jul 01 22:21:02.757822 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:02.757793 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p44zj" event={"ID":"4e8d667b-1ec1-4147-9238-6f016882b971","Type":"ContainerStarted","Data":"e6d5d8da2c1f81c3a4d438fd6c6eacb2d2b796133e9223f08d3521408166fd71"} Jul 01 22:21:02.759350 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:02.759319 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" event={"ID":"73baab49-4c50-4980-b761-08d3abeb05a8","Type":"ContainerStarted","Data":"a733aafd13627fed5dba8f096019c4144b06e4f82a0f8a2f78c52d8aa889b27e"} Jul 01 22:21:02.760875 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:02.760824 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wmmbz" event={"ID":"fbbd6b24-4540-40b3-b0e8-a9d0b67559f5","Type":"ContainerStarted","Data":"1674cb4e8a22c5229517cb7da5190791f4e36dbc25ebed1f1b052abaf91b82e8"} Jul 01 22:21:02.762306 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:02.762270 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" event={"ID":"71d59fe0-f377-4e68-825e-bec682a08714","Type":"ContainerStarted","Data":"37942df8a91b175c816f7d0cbf8968f0987e3cc40ad2fbd2805a9a8a9e0a7408"} Jul 01 22:21:02.763628 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:02.763600 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" event={"ID":"e0220c87-2759-46c6-aa51-8dcb525f8be1","Type":"ContainerStarted","Data":"9bbbc94140ed8c05fb926d34cb916f57232398c92bd495f61218c8bbe87a479c"} Jul 01 22:21:02.765126 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:02.765097 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vz9dp" event={"ID":"8694157d-ebfe-44aa-ae98-4ef443d09ec2","Type":"ContainerStarted","Data":"485e993a6c0db7f67acd9efb124699757bc28ce5f9096e0dddd1d948688ef6af"} Jul 01 22:21:02.766384 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:02.766347 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ws2ns" event={"ID":"33732b5f-89f5-4e70-a763-cfb5cda678e0","Type":"ContainerStarted","Data":"e81fc825c1431299a9e64ffaba7807f4bc8cf2e43ad9065372ea74be9e4cb640"} Jul 01 22:21:02.767671 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:02.767643 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qfqxw" event={"ID":"1ae00714-cbd8-4816-b57a-2568b3356575","Type":"ContainerStarted","Data":"3fced08aa6afc9f5f614ea35a860c30df641dcb1ad81ffcd4fd8b2d936aeb724"} Jul 01 22:21:03.245757 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:03.245503 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 22:21:03.404303 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:03.404262 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret\") pod \"global-pull-secret-syncer-d5xct\" (UID: \"f700798a-ec9b-4ee0-8a62-d5ff885135cd\") " pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:03.404491 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:03.404336 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs\") pod \"network-metrics-daemon-kfd7x\" (UID: \"246c052d-11a5-4032-9505-a1628e462a19\") " pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:03.404491 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:03.404412 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 22:21:03.404491 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:03.404486 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:21:03.404697 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:03.404502 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret podName:f700798a-ec9b-4ee0-8a62-d5ff885135cd nodeName:}" failed. No retries permitted until 2026-07-01 22:21:05.404479083 +0000 UTC m=+6.151919665 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret") pod "global-pull-secret-syncer-d5xct" (UID: "f700798a-ec9b-4ee0-8a62-d5ff885135cd") : object "kube-system"/"original-pull-secret" not registered Jul 01 22:21:03.404697 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:03.404540 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs podName:246c052d-11a5-4032-9505-a1628e462a19 nodeName:}" failed. No retries permitted until 2026-07-01 22:21:05.404523741 +0000 UTC m=+6.151964314 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs") pod "network-metrics-daemon-kfd7x" (UID: "246c052d-11a5-4032-9505-a1628e462a19") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:21:03.504916 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:03.504885 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slktq\" (UniqueName: \"kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq\") pod \"network-check-target-8tjzj\" (UID: \"bebea64f-bddc-4c5a-8c28-837fb09e93c5\") " pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:03.505017 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:03.505000 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 22:21:03.505074 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:03.505023 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 22:21:03.505074 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:03.505050 2580 projected.go:196] Error preparing data for projected volume kube-api-access-slktq for pod openshift-network-diagnostics/network-check-target-8tjzj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:21:03.505187 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:03.505109 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq podName:bebea64f-bddc-4c5a-8c28-837fb09e93c5 nodeName:}" failed. No retries permitted until 2026-07-01 22:21:05.505087845 +0000 UTC m=+6.252528431 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-slktq" (UniqueName: "kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq") pod "network-check-target-8tjzj" (UID: "bebea64f-bddc-4c5a-8c28-837fb09e93c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:21:03.724876 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:03.724588 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 22:16:02 +0000 UTC" deadline="2027-11-29 03:01:47.403838021 +0000 UTC" Jul 01 22:21:03.724876 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:03.724837 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12364h40m43.679008708s" Jul 01 22:21:03.742509 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:03.741476 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:03.742509 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:03.741646 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfd7x" podUID="246c052d-11a5-4032-9505-a1628e462a19" Jul 01 22:21:03.742509 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:03.742093 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:03.742509 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:03.742197 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d5xct" podUID="f700798a-ec9b-4ee0-8a62-d5ff885135cd" Jul 01 22:21:03.742509 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:03.742366 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:03.742509 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:03.742452 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tjzj" podUID="bebea64f-bddc-4c5a-8c28-837fb09e93c5" Jul 01 22:21:03.777388 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:03.777305 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-238.ec2.internal" event={"ID":"7d254b613571f06d3af95550e0820f84","Type":"ContainerStarted","Data":"e5dfa63dcdfa862c246d1ea21f78031dbef74d0a6d31efaf9b093240b4a46797"} Jul 01 22:21:04.787972 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:04.787911 2580 generic.go:356] "Generic (PLEG): container finished" podID="9d73a9c7b8912df8d8409030b111886c" containerID="cd46279aa0671fd863c65287562401c89c8843e5f8172a5e02450d5e292bc43f" exitCode=0 Jul 01 22:21:04.788975 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:04.788944 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-238.ec2.internal" event={"ID":"9d73a9c7b8912df8d8409030b111886c","Type":"ContainerDied","Data":"cd46279aa0671fd863c65287562401c89c8843e5f8172a5e02450d5e292bc43f"} Jul 01 22:21:04.808736 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:04.808682 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-238.ec2.internal" podStartSLOduration=3.808660059 podStartE2EDuration="3.808660059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:21:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:21:03.797111652 +0000 UTC m=+4.544552249" watchObservedRunningTime="2026-07-01 22:21:04.808660059 +0000 UTC m=+5.556100657" Jul 01 22:21:05.424962 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:05.424917 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret\") pod \"global-pull-secret-syncer-d5xct\" (UID: \"f700798a-ec9b-4ee0-8a62-d5ff885135cd\") " pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:05.425164 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:05.425000 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs\") pod \"network-metrics-daemon-kfd7x\" (UID: \"246c052d-11a5-4032-9505-a1628e462a19\") " pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:05.425232 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:05.425179 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:21:05.425283 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:05.425245 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs podName:246c052d-11a5-4032-9505-a1628e462a19 nodeName:}" failed. No retries permitted until 2026-07-01 22:21:09.425227306 +0000 UTC m=+10.172667878 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs") pod "network-metrics-daemon-kfd7x" (UID: "246c052d-11a5-4032-9505-a1628e462a19") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:21:05.425698 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:05.425678 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 22:21:05.425861 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:05.425732 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret podName:f700798a-ec9b-4ee0-8a62-d5ff885135cd nodeName:}" failed. No retries permitted until 2026-07-01 22:21:09.425716055 +0000 UTC m=+10.173156624 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret") pod "global-pull-secret-syncer-d5xct" (UID: "f700798a-ec9b-4ee0-8a62-d5ff885135cd") : object "kube-system"/"original-pull-secret" not registered Jul 01 22:21:05.526153 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:05.526111 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slktq\" (UniqueName: \"kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq\") pod \"network-check-target-8tjzj\" (UID: \"bebea64f-bddc-4c5a-8c28-837fb09e93c5\") " pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:05.526829 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:05.526335 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 22:21:05.526829 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:05.526357 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 22:21:05.526829 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:05.526371 2580 projected.go:196] Error preparing data for projected volume kube-api-access-slktq for pod openshift-network-diagnostics/network-check-target-8tjzj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:21:05.526829 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:05.526432 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq podName:bebea64f-bddc-4c5a-8c28-837fb09e93c5 nodeName:}" failed. No retries permitted until 2026-07-01 22:21:09.526412865 +0000 UTC m=+10.273853438 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-slktq" (UniqueName: "kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq") pod "network-check-target-8tjzj" (UID: "bebea64f-bddc-4c5a-8c28-837fb09e93c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:21:05.744667 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:05.743762 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:05.744667 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:05.743929 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfd7x" podUID="246c052d-11a5-4032-9505-a1628e462a19" Jul 01 22:21:05.744667 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:05.744348 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:05.744667 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:05.744457 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d5xct" podUID="f700798a-ec9b-4ee0-8a62-d5ff885135cd" Jul 01 22:21:05.745388 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:05.745154 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:05.745388 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:05.745264 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tjzj" podUID="bebea64f-bddc-4c5a-8c28-837fb09e93c5" Jul 01 22:21:07.742210 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:07.742131 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:07.742210 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:07.742165 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:07.742793 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:07.742262 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d5xct" podUID="f700798a-ec9b-4ee0-8a62-d5ff885135cd" Jul 01 22:21:07.742793 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:07.742130 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:07.742793 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:07.742713 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tjzj" podUID="bebea64f-bddc-4c5a-8c28-837fb09e93c5" Jul 01 22:21:07.742943 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:07.742830 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfd7x" podUID="246c052d-11a5-4032-9505-a1628e462a19" Jul 01 22:21:09.466117 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:09.466078 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret\") pod \"global-pull-secret-syncer-d5xct\" (UID: \"f700798a-ec9b-4ee0-8a62-d5ff885135cd\") " pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:09.466522 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:09.466140 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs\") pod \"network-metrics-daemon-kfd7x\" (UID: \"246c052d-11a5-4032-9505-a1628e462a19\") " pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:09.466522 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:09.466277 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:21:09.466522 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:09.466332 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs podName:246c052d-11a5-4032-9505-a1628e462a19 nodeName:}" failed. No retries permitted until 2026-07-01 22:21:17.466313686 +0000 UTC m=+18.213754258 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs") pod "network-metrics-daemon-kfd7x" (UID: "246c052d-11a5-4032-9505-a1628e462a19") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:21:09.466725 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:09.466709 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 22:21:09.466770 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:09.466760 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret podName:f700798a-ec9b-4ee0-8a62-d5ff885135cd nodeName:}" failed. No retries permitted until 2026-07-01 22:21:17.466745719 +0000 UTC m=+18.214186290 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret") pod "global-pull-secret-syncer-d5xct" (UID: "f700798a-ec9b-4ee0-8a62-d5ff885135cd") : object "kube-system"/"original-pull-secret" not registered Jul 01 22:21:09.567290 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:09.567258 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slktq\" (UniqueName: \"kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq\") pod \"network-check-target-8tjzj\" (UID: \"bebea64f-bddc-4c5a-8c28-837fb09e93c5\") " pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:09.567476 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:09.567435 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 22:21:09.567476 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:09.567457 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 22:21:09.567476 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:09.567470 2580 projected.go:196] Error preparing data for projected volume kube-api-access-slktq for pod openshift-network-diagnostics/network-check-target-8tjzj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:21:09.567658 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:09.567525 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq podName:bebea64f-bddc-4c5a-8c28-837fb09e93c5 nodeName:}" failed. No retries permitted until 2026-07-01 22:21:17.567506246 +0000 UTC m=+18.314946838 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-slktq" (UniqueName: "kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq") pod "network-check-target-8tjzj" (UID: "bebea64f-bddc-4c5a-8c28-837fb09e93c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:21:09.743221 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:09.743142 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:09.743365 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:09.743255 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfd7x" podUID="246c052d-11a5-4032-9505-a1628e462a19" Jul 01 22:21:09.743716 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:09.743513 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:09.743716 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:09.743636 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d5xct" podUID="f700798a-ec9b-4ee0-8a62-d5ff885135cd" Jul 01 22:21:09.743716 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:09.743687 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:09.743939 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:09.743791 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tjzj" podUID="bebea64f-bddc-4c5a-8c28-837fb09e93c5" Jul 01 22:21:11.741869 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:11.741535 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:11.741869 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:11.741567 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:11.741869 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:11.741578 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:11.741869 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:11.741676 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tjzj" podUID="bebea64f-bddc-4c5a-8c28-837fb09e93c5" Jul 01 22:21:11.741869 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:11.741750 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfd7x" podUID="246c052d-11a5-4032-9505-a1628e462a19" Jul 01 22:21:11.741869 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:11.741827 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d5xct" podUID="f700798a-ec9b-4ee0-8a62-d5ff885135cd" Jul 01 22:21:13.741030 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:13.741004 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:13.741030 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:13.741022 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:13.741336 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:13.741012 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:13.741336 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:13.741120 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d5xct" podUID="f700798a-ec9b-4ee0-8a62-d5ff885135cd" Jul 01 22:21:13.741336 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:13.741181 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfd7x" podUID="246c052d-11a5-4032-9505-a1628e462a19" Jul 01 22:21:13.741336 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:13.741222 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tjzj" podUID="bebea64f-bddc-4c5a-8c28-837fb09e93c5" Jul 01 22:21:15.741960 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:15.741926 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:15.742409 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:15.741965 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:15.742409 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:15.741924 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:15.742409 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:15.742058 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d5xct" podUID="f700798a-ec9b-4ee0-8a62-d5ff885135cd" Jul 01 22:21:15.742409 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:15.742147 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfd7x" podUID="246c052d-11a5-4032-9505-a1628e462a19" Jul 01 22:21:15.742409 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:15.742216 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tjzj" podUID="bebea64f-bddc-4c5a-8c28-837fb09e93c5" Jul 01 22:21:17.533216 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:17.533192 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret\") pod \"global-pull-secret-syncer-d5xct\" (UID: \"f700798a-ec9b-4ee0-8a62-d5ff885135cd\") " pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:17.533479 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:17.533235 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs\") pod \"network-metrics-daemon-kfd7x\" (UID: \"246c052d-11a5-4032-9505-a1628e462a19\") " pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:17.533479 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:17.533316 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:21:17.533479 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:17.533322 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 22:21:17.533479 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:17.533358 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs podName:246c052d-11a5-4032-9505-a1628e462a19 nodeName:}" failed. No retries permitted until 2026-07-01 22:21:33.533346857 +0000 UTC m=+34.280787424 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs") pod "network-metrics-daemon-kfd7x" (UID: "246c052d-11a5-4032-9505-a1628e462a19") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:21:17.533479 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:17.533370 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret podName:f700798a-ec9b-4ee0-8a62-d5ff885135cd nodeName:}" failed. No retries permitted until 2026-07-01 22:21:33.533364454 +0000 UTC m=+34.280805022 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret") pod "global-pull-secret-syncer-d5xct" (UID: "f700798a-ec9b-4ee0-8a62-d5ff885135cd") : object "kube-system"/"original-pull-secret" not registered Jul 01 22:21:17.634500 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:17.634471 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slktq\" (UniqueName: \"kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq\") pod \"network-check-target-8tjzj\" (UID: \"bebea64f-bddc-4c5a-8c28-837fb09e93c5\") " pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:17.634687 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:17.634666 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 22:21:17.634735 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:17.634697 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 22:21:17.634735 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:17.634710 2580 projected.go:196] Error preparing data for projected volume kube-api-access-slktq for pod openshift-network-diagnostics/network-check-target-8tjzj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:21:17.634816 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:17.634789 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq podName:bebea64f-bddc-4c5a-8c28-837fb09e93c5 nodeName:}" failed. No retries permitted until 2026-07-01 22:21:33.634748641 +0000 UTC m=+34.382189222 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-slktq" (UniqueName: "kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq") pod "network-check-target-8tjzj" (UID: "bebea64f-bddc-4c5a-8c28-837fb09e93c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:21:17.741822 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:17.741797 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:17.741986 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:17.741798 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:17.741986 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:17.741908 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d5xct" podUID="f700798a-ec9b-4ee0-8a62-d5ff885135cd" Jul 01 22:21:17.742089 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:17.742012 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfd7x" podUID="246c052d-11a5-4032-9505-a1628e462a19" Jul 01 22:21:17.742089 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:17.741801 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:17.742183 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:17.742093 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tjzj" podUID="bebea64f-bddc-4c5a-8c28-837fb09e93c5" Jul 01 22:21:19.742067 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:19.742034 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:19.742507 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:19.742157 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:19.742507 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:19.742168 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfd7x" podUID="246c052d-11a5-4032-9505-a1628e462a19" Jul 01 22:21:19.742507 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:19.742184 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:19.742507 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:19.742256 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d5xct" podUID="f700798a-ec9b-4ee0-8a62-d5ff885135cd" Jul 01 22:21:19.742507 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:19.742326 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tjzj" podUID="bebea64f-bddc-4c5a-8c28-837fb09e93c5" Jul 01 22:21:21.743703 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:21.743676 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:21.744051 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:21.743807 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfd7x" podUID="246c052d-11a5-4032-9505-a1628e462a19" Jul 01 22:21:21.744183 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:21.744165 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:21.744307 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:21.744285 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d5xct" podUID="f700798a-ec9b-4ee0-8a62-d5ff885135cd" Jul 01 22:21:21.744580 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:21.744523 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:21.744817 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:21.744773 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tjzj" podUID="bebea64f-bddc-4c5a-8c28-837fb09e93c5" Jul 01 22:21:21.835578 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:21.835109 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" event={"ID":"71d59fe0-f377-4e68-825e-bec682a08714","Type":"ContainerStarted","Data":"edbd2b5e19cb2edf092e2217ccc7b79523c6e50ceac0dcb444d2387e2db3fad3"} Jul 01 22:21:22.838794 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:22.838602 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" event={"ID":"e0220c87-2759-46c6-aa51-8dcb525f8be1","Type":"ContainerStarted","Data":"ebfc3fed58e3c998103f779fd5dbf77b7768c401388d6143db0c532411aaedba"} Jul 01 22:21:22.840321 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:22.840298 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-238.ec2.internal" event={"ID":"9d73a9c7b8912df8d8409030b111886c","Type":"ContainerStarted","Data":"08a9b745c2bb96ac2c1e4e78a63c7fa2bef4ca82b4eb1ee96bcac2249b358d8d"} Jul 01 22:21:22.841487 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:22.841467 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vz9dp" event={"ID":"8694157d-ebfe-44aa-ae98-4ef443d09ec2","Type":"ContainerStarted","Data":"f4563aeb2b25b8690fcd7ad4f1c2a187d26d18a02c51961c45a147c6e1b5eab4"} Jul 01 22:21:22.842823 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:22.842797 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ws2ns" event={"ID":"33732b5f-89f5-4e70-a763-cfb5cda678e0","Type":"ContainerStarted","Data":"fd787b767b76e7d38305f076b30c40abf706b053a5e717cb7b97f967087eba60"} Jul 01 22:21:22.844062 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:22.844043 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d4j26" event={"ID":"f7388d18-cc3b-407c-98ee-5543c413b77e","Type":"ContainerStarted","Data":"c9ad649c19b2b74f07ab198bf2306b08e88b2e57513253cc8e81545878075ebe"} Jul 01 22:21:22.845314 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:22.845293 2580 generic.go:356] "Generic (PLEG): container finished" podID="4e8d667b-1ec1-4147-9238-6f016882b971" containerID="b266df931499e99e909432ceff603186b1ed80f986f350041d819660a14e9636" exitCode=0 Jul 01 22:21:22.845406 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:22.845373 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p44zj" event={"ID":"4e8d667b-1ec1-4147-9238-6f016882b971","Type":"ContainerDied","Data":"b266df931499e99e909432ceff603186b1ed80f986f350041d819660a14e9636"} Jul 01 22:21:22.848308 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:22.848288 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" event={"ID":"73baab49-4c50-4980-b761-08d3abeb05a8","Type":"ContainerStarted","Data":"c62dc75ac6a876ec4e55231fb5a6821d2e218c3641b984d1231b704d2b27c010"} Jul 01 22:21:22.848387 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:22.848351 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" event={"ID":"73baab49-4c50-4980-b761-08d3abeb05a8","Type":"ContainerStarted","Data":"015abce2ac0cf27f588a6552ff496d3cebb9acb16124c9a3ede0e1b50ba7c1f3"} Jul 01 22:21:22.848436 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:22.848393 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" event={"ID":"73baab49-4c50-4980-b761-08d3abeb05a8","Type":"ContainerStarted","Data":"9de4ec1c2fbdeb64911f3867ef8cec6081b524bd5fcb5377a74c76c291aa744b"} Jul 01 22:21:22.848436 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:22.848426 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" event={"ID":"73baab49-4c50-4980-b761-08d3abeb05a8","Type":"ContainerStarted","Data":"5650b577290d13568699a7a3158f468e5c95652c224dc62110b56b8b690a6823"} Jul 01 22:21:22.848507 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:22.848470 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" event={"ID":"73baab49-4c50-4980-b761-08d3abeb05a8","Type":"ContainerStarted","Data":"d9565e672cf1a863f9000a17e8faab12b8245e1396149e8d2842bd3463362ea4"} Jul 01 22:21:22.848563 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:22.848516 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" event={"ID":"73baab49-4c50-4980-b761-08d3abeb05a8","Type":"ContainerStarted","Data":"dc6c1d561dd17ca1603ff1be60b574d510b9d4a4855e42f5923fbe3ad7da2908"} Jul 01 22:21:22.849592 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:22.849570 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wmmbz" event={"ID":"fbbd6b24-4540-40b3-b0e8-a9d0b67559f5","Type":"ContainerStarted","Data":"3a79e3731a1098b6b18d4924558161048a649793d28b2c3a4aa76713e122c2b8"} Jul 01 22:21:22.856514 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:22.856475 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jxmpz" podStartSLOduration=3.316937541 podStartE2EDuration="22.856459471s" totalImagesPullingTime="19.53952193s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:21:21.856324831 +0000 UTC m=+22.603765423" watchObservedRunningTime="2026-07-01 22:21:22.856459471 +0000 UTC m=+23.603900063" Jul 01 22:21:22.856700 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:22.856681 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-238.ec2.internal" podStartSLOduration=21.856673194 podStartE2EDuration="21.856673194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:21:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:21:22.856399469 +0000 UTC m=+23.603840060" watchObservedRunningTime="2026-07-01 22:21:22.856673194 +0000 UTC m=+23.604113783" Jul 01 22:21:22.871877 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:22.871842 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vz9dp" podStartSLOduration=7.474031689 podStartE2EDuration="22.871831436s" totalImagesPullingTime="15.397799747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:21:22.871512821 +0000 UTC m=+23.618953411" watchObservedRunningTime="2026-07-01 22:21:22.871831436 +0000 UTC m=+23.619272026" Jul 01 22:21:22.911904 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:22.911864 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-d4j26" podStartSLOduration=3.325244741 podStartE2EDuration="22.911847785s" totalImagesPullingTime="19.586603044s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:21:22.911830035 +0000 UTC m=+23.659270627" watchObservedRunningTime="2026-07-01 22:21:22.911847785 +0000 UTC m=+23.659288358" Jul 01 22:21:22.926665 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:22.926597 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wmmbz" podStartSLOduration=3.399506268 podStartE2EDuration="22.926587442s" totalImagesPullingTime="19.527081174s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:21:22.926096155 +0000 UTC m=+23.673536744" watchObservedRunningTime="2026-07-01 22:21:22.926587442 +0000 UTC m=+23.674028032" Jul 01 22:21:22.941812 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:22.941773 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ws2ns" podStartSLOduration=3.419514919 podStartE2EDuration="22.941761524s" totalImagesPullingTime="19.522246605s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:21:22.941422722 +0000 UTC m=+23.688863313" watchObservedRunningTime="2026-07-01 22:21:22.941761524 +0000 UTC m=+23.689202114" Jul 01 22:21:23.170858 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:23.170842 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 01 22:21:23.741800 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:23.741771 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:23.742160 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:23.741822 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:23.742160 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:23.741776 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:23.742160 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:23.741909 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d5xct" podUID="f700798a-ec9b-4ee0-8a62-d5ff885135cd" Jul 01 22:21:23.742494 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:23.742467 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tjzj" podUID="bebea64f-bddc-4c5a-8c28-837fb09e93c5" Jul 01 22:21:23.745321 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:23.743150 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfd7x" podUID="246c052d-11a5-4032-9505-a1628e462a19" Jul 01 22:21:23.753850 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:23.753763 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-01T22:21:23.170855678Z","UUID":"3d22aa2b-07eb-49f7-9d77-144e5661c682","Handler":null,"Name":"","Endpoint":""} Jul 01 22:21:23.755464 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:23.755445 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 01 22:21:23.755590 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:23.755485 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 01 22:21:23.853993 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:23.853962 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qfqxw" event={"ID":"1ae00714-cbd8-4816-b57a-2568b3356575","Type":"ContainerStarted","Data":"d28d52f50d5324cf49e7c8d6d394d0facd5b458420d0c5ed77c6842fee55cf57"} Jul 01 22:21:23.856913 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:23.856876 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" event={"ID":"e0220c87-2759-46c6-aa51-8dcb525f8be1","Type":"ContainerStarted","Data":"d1a746dbd92930dff4b78b1e521e62d0478ed5537d64031717308e8dce53ad6a"} Jul 01 22:21:23.870231 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:23.870189 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qfqxw" podStartSLOduration=4.352930883 podStartE2EDuration="23.870174894s" totalImagesPullingTime="19.517244011s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:21:23.869738276 +0000 UTC m=+24.617178867" watchObservedRunningTime="2026-07-01 22:21:23.870174894 +0000 UTC m=+24.617615507" Jul 01 22:21:24.863078 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:24.862910 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" event={"ID":"73baab49-4c50-4980-b761-08d3abeb05a8","Type":"ContainerStarted","Data":"5e5553815e7af4ea9ffecae431bf754572277d332a0a2139510de749524a4532"} Jul 01 22:21:24.865179 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:24.865150 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" event={"ID":"e0220c87-2759-46c6-aa51-8dcb525f8be1","Type":"ContainerStarted","Data":"ddf9e343b66d049a18d3ab271e9e1ce3add158e605442f1db6238d7ef5ee946e"} Jul 01 22:21:24.884733 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:24.884687 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-9qzlj" podStartSLOduration=2.754445892 podStartE2EDuration="24.884672186s" totalImagesPullingTime="22.130226294s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:21:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:21:24.884271763 +0000 UTC m=+25.631712352" watchObservedRunningTime="2026-07-01 22:21:24.884672186 +0000 UTC m=+25.632112775" Jul 01 22:21:25.741573 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:25.741527 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:25.741780 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:25.741527 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:25.741780 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:25.741669 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfd7x" podUID="246c052d-11a5-4032-9505-a1628e462a19" Jul 01 22:21:25.741780 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:25.741537 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:25.741780 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:25.741759 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tjzj" podUID="bebea64f-bddc-4c5a-8c28-837fb09e93c5" Jul 01 22:21:25.741995 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:25.741805 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d5xct" podUID="f700798a-ec9b-4ee0-8a62-d5ff885135cd" Jul 01 22:21:26.976822 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:26.976787 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vz9dp" Jul 01 22:21:26.977588 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:26.977571 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vz9dp" Jul 01 22:21:27.742099 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:27.741905 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:27.742212 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:27.741917 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:27.742212 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:27.742141 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfd7x" podUID="246c052d-11a5-4032-9505-a1628e462a19" Jul 01 22:21:27.742288 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:27.742245 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tjzj" podUID="bebea64f-bddc-4c5a-8c28-837fb09e93c5" Jul 01 22:21:27.742288 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:27.741917 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:27.742365 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:27.742346 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d5xct" podUID="f700798a-ec9b-4ee0-8a62-d5ff885135cd" Jul 01 22:21:27.873767 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:27.873744 2580 generic.go:356] "Generic (PLEG): container finished" podID="4e8d667b-1ec1-4147-9238-6f016882b971" containerID="8251feb0a7dca28b410e396592b99c1dc979f59614c78dab808c3bb232f24b34" exitCode=0 Jul 01 22:21:27.873887 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:27.873822 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p44zj" event={"ID":"4e8d667b-1ec1-4147-9238-6f016882b971","Type":"ContainerDied","Data":"8251feb0a7dca28b410e396592b99c1dc979f59614c78dab808c3bb232f24b34"} Jul 01 22:21:27.877764 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:27.877745 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" event={"ID":"73baab49-4c50-4980-b761-08d3abeb05a8","Type":"ContainerStarted","Data":"5cbb8d47f402a366a68f853e5b90a9acd2f73f010c48eb12b68d8f0459407ae9"} Jul 01 22:21:27.878097 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:27.878079 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vz9dp" Jul 01 22:21:27.878422 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:27.878402 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vz9dp" Jul 01 22:21:27.926085 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:27.926048 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" podStartSLOduration=8.322098331 podStartE2EDuration="27.926036187s" totalImagesPullingTime="19.603937856s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:21:27.924339061 +0000 UTC m=+28.671779645" watchObservedRunningTime="2026-07-01 22:21:27.926036187 +0000 UTC m=+28.673476777" Jul 01 22:21:28.880254 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:28.880224 2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 01 22:21:28.880988 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:28.880692 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:28.880988 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:28.880816 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:28.897394 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:28.897368 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:28.897538 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:28.897501 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:29.356998 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:29.356964 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kfd7x"] Jul 01 22:21:29.357115 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:29.357068 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:29.357175 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:29.357157 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfd7x" podUID="246c052d-11a5-4032-9505-a1628e462a19" Jul 01 22:21:29.359770 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:29.359742 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8tjzj"] Jul 01 22:21:29.359868 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:29.359845 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:29.359933 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:29.359914 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tjzj" podUID="bebea64f-bddc-4c5a-8c28-837fb09e93c5" Jul 01 22:21:29.370023 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:29.370001 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d5xct"] Jul 01 22:21:29.370161 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:29.370090 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:29.370222 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:29.370174 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d5xct" podUID="f700798a-ec9b-4ee0-8a62-d5ff885135cd" Jul 01 22:21:29.454145 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:29.454088 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:21:29.883521 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:29.883453 2580 generic.go:356] "Generic (PLEG): container finished" podID="4e8d667b-1ec1-4147-9238-6f016882b971" containerID="c045aefed56286d462affb8031453648bdaf5c907c21b49a5b0b96c06b8f4739" exitCode=0 Jul 01 22:21:29.883914 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:29.883539 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p44zj" event={"ID":"4e8d667b-1ec1-4147-9238-6f016882b971","Type":"ContainerDied","Data":"c045aefed56286d462affb8031453648bdaf5c907c21b49a5b0b96c06b8f4739"} Jul 01 22:21:30.741195 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:30.741163 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:30.741345 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:30.741201 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:30.741345 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:30.741271 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tjzj" podUID="bebea64f-bddc-4c5a-8c28-837fb09e93c5" Jul 01 22:21:30.741423 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:30.741391 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfd7x" podUID="246c052d-11a5-4032-9505-a1628e462a19" Jul 01 22:21:30.888655 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:30.888274 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p44zj" event={"ID":"4e8d667b-1ec1-4147-9238-6f016882b971","Type":"ContainerStarted","Data":"a02a5ec5de2d7e7ee8f10b0bc359117df5d698d19ab55a145eddc9205032ffb5"} Jul 01 22:21:31.741650 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:31.741622 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:31.741810 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:31.741716 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d5xct" podUID="f700798a-ec9b-4ee0-8a62-d5ff885135cd" Jul 01 22:21:31.892089 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:31.892063 2580 generic.go:356] "Generic (PLEG): container finished" podID="4e8d667b-1ec1-4147-9238-6f016882b971" containerID="a02a5ec5de2d7e7ee8f10b0bc359117df5d698d19ab55a145eddc9205032ffb5" exitCode=0 Jul 01 22:21:31.892514 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:31.892101 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p44zj" event={"ID":"4e8d667b-1ec1-4147-9238-6f016882b971","Type":"ContainerDied","Data":"a02a5ec5de2d7e7ee8f10b0bc359117df5d698d19ab55a145eddc9205032ffb5"} Jul 01 22:21:32.741679 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:32.741650 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:32.741679 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:32.741671 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:32.741889 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:32.741760 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tjzj" podUID="bebea64f-bddc-4c5a-8c28-837fb09e93c5" Jul 01 22:21:32.741889 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:32.741872 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfd7x" podUID="246c052d-11a5-4032-9505-a1628e462a19" Jul 01 22:21:33.571281 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:33.571196 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs\") pod \"network-metrics-daemon-kfd7x\" (UID: \"246c052d-11a5-4032-9505-a1628e462a19\") " pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:33.571281 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:33.571274 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret\") pod \"global-pull-secret-syncer-d5xct\" (UID: \"f700798a-ec9b-4ee0-8a62-d5ff885135cd\") " pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:33.571835 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:33.571378 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 22:21:33.571835 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:33.571382 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:21:33.571835 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:33.571436 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret podName:f700798a-ec9b-4ee0-8a62-d5ff885135cd nodeName:}" failed. No retries permitted until 2026-07-01 22:22:05.571422493 +0000 UTC m=+66.318863063 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret") pod "global-pull-secret-syncer-d5xct" (UID: "f700798a-ec9b-4ee0-8a62-d5ff885135cd") : object "kube-system"/"original-pull-secret" not registered Jul 01 22:21:33.571835 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:33.571457 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs podName:246c052d-11a5-4032-9505-a1628e462a19 nodeName:}" failed. No retries permitted until 2026-07-01 22:22:05.571445991 +0000 UTC m=+66.318886564 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs") pod "network-metrics-daemon-kfd7x" (UID: "246c052d-11a5-4032-9505-a1628e462a19") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 22:21:33.671827 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:33.671799 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slktq\" (UniqueName: \"kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq\") pod \"network-check-target-8tjzj\" (UID: \"bebea64f-bddc-4c5a-8c28-837fb09e93c5\") " pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:33.671996 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:33.671942 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 22:21:33.671996 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:33.671962 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 22:21:33.671996 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:33.671976 2580 projected.go:196] Error preparing data for projected volume kube-api-access-slktq for pod openshift-network-diagnostics/network-check-target-8tjzj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:21:33.672122 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:33.672028 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq podName:bebea64f-bddc-4c5a-8c28-837fb09e93c5 nodeName:}" failed. No retries permitted until 2026-07-01 22:22:05.672011169 +0000 UTC m=+66.419451747 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-slktq" (UniqueName: "kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq") pod "network-check-target-8tjzj" (UID: "bebea64f-bddc-4c5a-8c28-837fb09e93c5") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 22:21:33.741587 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:33.741506 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:33.741736 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:33.741648 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-d5xct" podUID="f700798a-ec9b-4ee0-8a62-d5ff885135cd" Jul 01 22:21:34.741813 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:34.741778 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:34.742246 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:34.741823 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:34.742246 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:34.741913 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kfd7x" podUID="246c052d-11a5-4032-9505-a1628e462a19" Jul 01 22:21:34.742246 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:34.742017 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8tjzj" podUID="bebea64f-bddc-4c5a-8c28-837fb09e93c5" Jul 01 22:21:35.558469 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.558390 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-238.ec2.internal" event="NodeReady" Jul 01 22:21:35.558648 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.558531 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 01 22:21:35.593716 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.593690 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-77d8dfbf5c-c4b7k"] Jul 01 22:21:35.598922 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.598901 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.601726 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.601583 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 01 22:21:35.601726 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.601613 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5b2tg\"" Jul 01 22:21:35.601726 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.601586 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 01 22:21:35.601936 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.601922 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 01 22:21:35.609390 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.609370 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 01 22:21:35.610694 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.610676 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7zjsw"] Jul 01 22:21:35.614203 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.614140 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77d8dfbf5c-c4b7k"] Jul 01 22:21:35.614316 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.614173 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5sqqv"] Jul 01 22:21:35.614540 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.614252 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7zjsw" Jul 01 22:21:35.616886 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.616867 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tjcnx\"" Jul 01 22:21:35.616984 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.616871 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 01 22:21:35.617151 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.617135 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 01 22:21:35.617860 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.617845 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5sqqv" Jul 01 22:21:35.620440 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.620419 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 01 22:21:35.620440 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.620448 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 01 22:21:35.620624 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.620613 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 01 22:21:35.620742 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.620725 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9nw5d\"" Jul 01 22:21:35.627123 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.626620 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7zjsw"] Jul 01 22:21:35.627352 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.627323 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5sqqv"] Jul 01 22:21:35.685788 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.685761 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3740f848-7757-4f34-aac1-5318ff61e81d-trusted-ca\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.685926 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.685800 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcvfg\" (UniqueName: \"kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-kube-api-access-bcvfg\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.685926 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.685828 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ct89\" (UniqueName: \"kubernetes.io/projected/e405fa2c-a5db-47a7-9de0-909629fafdd8-kube-api-access-5ct89\") pod \"ingress-canary-5sqqv\" (UID: \"e405fa2c-a5db-47a7-9de0-909629fafdd8\") " pod="openshift-ingress-canary/ingress-canary-5sqqv" Jul 01 22:21:35.685926 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.685843 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3740f848-7757-4f34-aac1-5318ff61e81d-registry-certificates\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.685926 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.685872 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-bound-sa-token\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.686142 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.685960 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-config-volume\") pod \"dns-default-7zjsw\" (UID: \"0ed3959f-efbe-43a9-a6f3-d7516f19dcd0\") " pod="openshift-dns/dns-default-7zjsw" Jul 01 22:21:35.686142 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.685994 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls\") pod \"dns-default-7zjsw\" (UID: \"0ed3959f-efbe-43a9-a6f3-d7516f19dcd0\") " pod="openshift-dns/dns-default-7zjsw" Jul 01 22:21:35.686142 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.686041 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3740f848-7757-4f34-aac1-5318ff61e81d-ca-trust-extracted\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.686142 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.686068 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3740f848-7757-4f34-aac1-5318ff61e81d-installation-pull-secrets\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.686142 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.686096 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-tmp-dir\") pod \"dns-default-7zjsw\" (UID: \"0ed3959f-efbe-43a9-a6f3-d7516f19dcd0\") " pod="openshift-dns/dns-default-7zjsw" Jul 01 22:21:35.686142 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.686118 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl85g\" (UniqueName: \"kubernetes.io/projected/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-kube-api-access-xl85g\") pod \"dns-default-7zjsw\" (UID: \"0ed3959f-efbe-43a9-a6f3-d7516f19dcd0\") " pod="openshift-dns/dns-default-7zjsw" Jul 01 22:21:35.686361 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.686144 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.686361 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.686184 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e405fa2c-a5db-47a7-9de0-909629fafdd8-cert\") pod \"ingress-canary-5sqqv\" (UID: \"e405fa2c-a5db-47a7-9de0-909629fafdd8\") " pod="openshift-ingress-canary/ingress-canary-5sqqv" Jul 01 22:21:35.686361 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.686209 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3740f848-7757-4f34-aac1-5318ff61e81d-image-registry-private-configuration\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.741627 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.741606 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:21:35.744436 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.744401 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 01 22:21:35.786597 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.786576 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-config-volume\") pod \"dns-default-7zjsw\" (UID: \"0ed3959f-efbe-43a9-a6f3-d7516f19dcd0\") " pod="openshift-dns/dns-default-7zjsw" Jul 01 22:21:35.786705 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.786604 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls\") pod \"dns-default-7zjsw\" (UID: \"0ed3959f-efbe-43a9-a6f3-d7516f19dcd0\") " pod="openshift-dns/dns-default-7zjsw" Jul 01 22:21:35.786705 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.786644 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3740f848-7757-4f34-aac1-5318ff61e81d-ca-trust-extracted\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.786705 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.786662 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3740f848-7757-4f34-aac1-5318ff61e81d-installation-pull-secrets\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.786705 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.786686 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-tmp-dir\") pod \"dns-default-7zjsw\" (UID: \"0ed3959f-efbe-43a9-a6f3-d7516f19dcd0\") " pod="openshift-dns/dns-default-7zjsw" Jul 01 22:21:35.786901 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.786709 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xl85g\" (UniqueName: \"kubernetes.io/projected/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-kube-api-access-xl85g\") pod \"dns-default-7zjsw\" (UID: \"0ed3959f-efbe-43a9-a6f3-d7516f19dcd0\") " pod="openshift-dns/dns-default-7zjsw" Jul 01 22:21:35.786901 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.786736 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.786901 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.786779 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e405fa2c-a5db-47a7-9de0-909629fafdd8-cert\") pod \"ingress-canary-5sqqv\" (UID: \"e405fa2c-a5db-47a7-9de0-909629fafdd8\") " pod="openshift-ingress-canary/ingress-canary-5sqqv" Jul 01 22:21:35.786901 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.786802 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3740f848-7757-4f34-aac1-5318ff61e81d-image-registry-private-configuration\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.786901 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:35.786821 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 22:21:35.786901 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.786831 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3740f848-7757-4f34-aac1-5318ff61e81d-trusted-ca\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.786901 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.786855 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcvfg\" (UniqueName: \"kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-kube-api-access-bcvfg\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.786901 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:35.786875 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls podName:0ed3959f-efbe-43a9-a6f3-d7516f19dcd0 nodeName:}" failed. No retries permitted until 2026-07-01 22:21:36.286858037 +0000 UTC m=+37.034298616 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls") pod "dns-default-7zjsw" (UID: "0ed3959f-efbe-43a9-a6f3-d7516f19dcd0") : secret "dns-default-metrics-tls" not found Jul 01 22:21:35.787347 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.786908 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ct89\" (UniqueName: \"kubernetes.io/projected/e405fa2c-a5db-47a7-9de0-909629fafdd8-kube-api-access-5ct89\") pod \"ingress-canary-5sqqv\" (UID: \"e405fa2c-a5db-47a7-9de0-909629fafdd8\") " pod="openshift-ingress-canary/ingress-canary-5sqqv" Jul 01 22:21:35.787347 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.786933 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3740f848-7757-4f34-aac1-5318ff61e81d-registry-certificates\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.787347 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.786964 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-bound-sa-token\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.787347 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.787060 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3740f848-7757-4f34-aac1-5318ff61e81d-ca-trust-extracted\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.787347 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.787110 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-tmp-dir\") pod \"dns-default-7zjsw\" (UID: \"0ed3959f-efbe-43a9-a6f3-d7516f19dcd0\") " pod="openshift-dns/dns-default-7zjsw" Jul 01 22:21:35.787347 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.787169 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-config-volume\") pod \"dns-default-7zjsw\" (UID: \"0ed3959f-efbe-43a9-a6f3-d7516f19dcd0\") " pod="openshift-dns/dns-default-7zjsw" Jul 01 22:21:35.787671 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:35.787497 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 22:21:35.787671 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:35.787513 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77d8dfbf5c-c4b7k: secret "image-registry-tls" not found Jul 01 22:21:35.787671 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:35.787584 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls podName:3740f848-7757-4f34-aac1-5318ff61e81d nodeName:}" failed. No retries permitted until 2026-07-01 22:21:36.287567469 +0000 UTC m=+37.035008054 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls") pod "image-registry-77d8dfbf5c-c4b7k" (UID: "3740f848-7757-4f34-aac1-5318ff61e81d") : secret "image-registry-tls" not found Jul 01 22:21:35.787671 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.787661 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3740f848-7757-4f34-aac1-5318ff61e81d-registry-certificates\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.788203 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.788094 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3740f848-7757-4f34-aac1-5318ff61e81d-trusted-ca\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.791365 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.791345 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3740f848-7757-4f34-aac1-5318ff61e81d-installation-pull-secrets\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.791452 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.791345 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3740f848-7757-4f34-aac1-5318ff61e81d-image-registry-private-configuration\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.792011 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.791991 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e405fa2c-a5db-47a7-9de0-909629fafdd8-cert\") pod \"ingress-canary-5sqqv\" (UID: \"e405fa2c-a5db-47a7-9de0-909629fafdd8\") " pod="openshift-ingress-canary/ingress-canary-5sqqv" Jul 01 22:21:35.798353 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.798129 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-bound-sa-token\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.798353 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.798185 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcvfg\" (UniqueName: \"kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-kube-api-access-bcvfg\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:35.799221 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.799176 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl85g\" (UniqueName: \"kubernetes.io/projected/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-kube-api-access-xl85g\") pod \"dns-default-7zjsw\" (UID: \"0ed3959f-efbe-43a9-a6f3-d7516f19dcd0\") " pod="openshift-dns/dns-default-7zjsw" Jul 01 22:21:35.800029 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.800010 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ct89\" (UniqueName: \"kubernetes.io/projected/e405fa2c-a5db-47a7-9de0-909629fafdd8-kube-api-access-5ct89\") pod \"ingress-canary-5sqqv\" (UID: \"e405fa2c-a5db-47a7-9de0-909629fafdd8\") " pod="openshift-ingress-canary/ingress-canary-5sqqv" Jul 01 22:21:35.836814 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.836751 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vz9dp_8694157d-ebfe-44aa-ae98-4ef443d09ec2/konnectivity-agent/0.log" limit=1024 Jul 01 22:21:35.933236 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.933205 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5sqqv" Jul 01 22:21:35.965102 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:35.965075 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vz9dp_8694157d-ebfe-44aa-ae98-4ef443d09ec2/konnectivity-agent/0.log" limit=1024 Jul 01 22:21:36.055048 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:36.054891 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vz9dp_8694157d-ebfe-44aa-ae98-4ef443d09ec2/konnectivity-agent/0.log" limit=1024 Jul 01 22:21:36.111515 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:36.111488 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5sqqv"] Jul 01 22:21:36.119102 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:21:36.119065 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode405fa2c_a5db_47a7_9de0_909629fafdd8.slice/crio-d6ef2f14119aa9f52892ef7a0d005811df5bd27b39ef0afa8ba0d4fb3dd46a73 WatchSource:0}: Error finding container d6ef2f14119aa9f52892ef7a0d005811df5bd27b39ef0afa8ba0d4fb3dd46a73: Status 404 returned error can't find the container with id d6ef2f14119aa9f52892ef7a0d005811df5bd27b39ef0afa8ba0d4fb3dd46a73 Jul 01 22:21:36.226032 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:36.226005 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vz9dp_8694157d-ebfe-44aa-ae98-4ef443d09ec2/konnectivity-agent/0.log" limit=1024 Jul 01 22:21:36.290679 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:36.290646 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vz9dp_8694157d-ebfe-44aa-ae98-4ef443d09ec2/konnectivity-agent/0.log" limit=1024 Jul 01 22:21:36.291544 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:36.291526 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls\") pod \"dns-default-7zjsw\" (UID: \"0ed3959f-efbe-43a9-a6f3-d7516f19dcd0\") " pod="openshift-dns/dns-default-7zjsw" Jul 01 22:21:36.291614 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:36.291581 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:36.291665 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:36.291650 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 22:21:36.291699 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:36.291678 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 22:21:36.291699 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:36.291688 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77d8dfbf5c-c4b7k: secret "image-registry-tls" not found Jul 01 22:21:36.291758 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:36.291708 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls podName:0ed3959f-efbe-43a9-a6f3-d7516f19dcd0 nodeName:}" failed. No retries permitted until 2026-07-01 22:21:37.291691514 +0000 UTC m=+38.039132082 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls") pod "dns-default-7zjsw" (UID: "0ed3959f-efbe-43a9-a6f3-d7516f19dcd0") : secret "dns-default-metrics-tls" not found Jul 01 22:21:36.291758 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:36.291724 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls podName:3740f848-7757-4f34-aac1-5318ff61e81d nodeName:}" failed. No retries permitted until 2026-07-01 22:21:37.291713158 +0000 UTC m=+38.039153730 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls") pod "image-registry-77d8dfbf5c-c4b7k" (UID: "3740f848-7757-4f34-aac1-5318ff61e81d") : secret "image-registry-tls" not found Jul 01 22:21:36.741467 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:36.741436 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:21:36.741650 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:36.741632 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:21:36.744700 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:36.744683 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 01 22:21:36.745119 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:36.744701 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f7p2h\"" Jul 01 22:21:36.745748 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:36.745725 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 01 22:21:36.745855 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:36.745788 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 01 22:21:36.745937 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:36.745921 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-smgl4\"" Jul 01 22:21:36.786327 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:36.786302 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vz9dp_8694157d-ebfe-44aa-ae98-4ef443d09ec2/konnectivity-agent/0.log" limit=1024 Jul 01 22:21:36.885666 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:36.885641 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vz9dp_8694157d-ebfe-44aa-ae98-4ef443d09ec2/konnectivity-agent/0.log" limit=1024 Jul 01 22:21:36.907910 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:36.907889 2580 generic.go:356] "Generic (PLEG): container finished" podID="4e8d667b-1ec1-4147-9238-6f016882b971" containerID="2e1f8d0807132c4fbf0a92d51c091032adebe95071b5049ed03a54685111bc0d" exitCode=0 Jul 01 22:21:36.908016 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:36.907923 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p44zj" event={"ID":"4e8d667b-1ec1-4147-9238-6f016882b971","Type":"ContainerDied","Data":"2e1f8d0807132c4fbf0a92d51c091032adebe95071b5049ed03a54685111bc0d"} Jul 01 22:21:36.909272 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:36.909251 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5sqqv" event={"ID":"e405fa2c-a5db-47a7-9de0-909629fafdd8","Type":"ContainerStarted","Data":"d6ef2f14119aa9f52892ef7a0d005811df5bd27b39ef0afa8ba0d4fb3dd46a73"} Jul 01 22:21:36.957925 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:36.955200 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vz9dp_8694157d-ebfe-44aa-ae98-4ef443d09ec2/konnectivity-agent/0.log" limit=1024 Jul 01 22:21:37.026318 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:37.026284 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vz9dp_8694157d-ebfe-44aa-ae98-4ef443d09ec2/konnectivity-agent/0.log" limit=1024 Jul 01 22:21:37.184079 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:37.184027 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vz9dp_8694157d-ebfe-44aa-ae98-4ef443d09ec2/konnectivity-agent/0.log" limit=1024 Jul 01 22:21:37.284539 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:37.284460 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vz9dp_8694157d-ebfe-44aa-ae98-4ef443d09ec2/konnectivity-agent/0.log" limit=1024 Jul 01 22:21:37.298596 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:37.298545 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls\") pod \"dns-default-7zjsw\" (UID: \"0ed3959f-efbe-43a9-a6f3-d7516f19dcd0\") " pod="openshift-dns/dns-default-7zjsw" Jul 01 22:21:37.298712 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:37.298617 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:37.298712 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:37.298685 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 22:21:37.298802 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:37.298725 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 22:21:37.298802 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:37.298739 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77d8dfbf5c-c4b7k: secret "image-registry-tls" not found Jul 01 22:21:37.298802 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:37.298754 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls podName:0ed3959f-efbe-43a9-a6f3-d7516f19dcd0 nodeName:}" failed. No retries permitted until 2026-07-01 22:21:39.298735373 +0000 UTC m=+40.046175970 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls") pod "dns-default-7zjsw" (UID: "0ed3959f-efbe-43a9-a6f3-d7516f19dcd0") : secret "dns-default-metrics-tls" not found Jul 01 22:21:37.298802 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:37.298787 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls podName:3740f848-7757-4f34-aac1-5318ff61e81d nodeName:}" failed. No retries permitted until 2026-07-01 22:21:39.298772665 +0000 UTC m=+40.046213236 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls") pod "image-registry-77d8dfbf5c-c4b7k" (UID: "3740f848-7757-4f34-aac1-5318ff61e81d") : secret "image-registry-tls" not found Jul 01 22:21:37.913869 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:37.913840 2580 generic.go:356] "Generic (PLEG): container finished" podID="4e8d667b-1ec1-4147-9238-6f016882b971" containerID="e85c38c33770e833a9692ce79f947007e4ca74bf2d54a914b7043b09d0817ed6" exitCode=0 Jul 01 22:21:37.914271 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:37.913888 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p44zj" event={"ID":"4e8d667b-1ec1-4147-9238-6f016882b971","Type":"ContainerDied","Data":"e85c38c33770e833a9692ce79f947007e4ca74bf2d54a914b7043b09d0817ed6"} Jul 01 22:21:38.835763 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:38.835600 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vz9dp_8694157d-ebfe-44aa-ae98-4ef443d09ec2/konnectivity-agent/0.log" limit=1024 Jul 01 22:21:38.919276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:38.919246 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p44zj" event={"ID":"4e8d667b-1ec1-4147-9238-6f016882b971","Type":"ContainerStarted","Data":"f1db806de89dfa800ff0c68d27dac5c063330ad4728fcf822d799bc02e62390c"} Jul 01 22:21:38.943162 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:38.943054 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-p44zj" podStartSLOduration=13.068668286 podStartE2EDuration="38.943035341s" totalImagesPullingTime="25.874367055s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:21:00 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:21:38.941499567 +0000 UTC m=+39.688940182" watchObservedRunningTime="2026-07-01 22:21:38.943035341 +0000 UTC m=+39.690475931" Jul 01 22:21:39.065274 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:39.065214 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vz9dp_8694157d-ebfe-44aa-ae98-4ef443d09ec2/konnectivity-agent/0.log" limit=1024 Jul 01 22:21:39.162955 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:39.162918 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vz9dp_8694157d-ebfe-44aa-ae98-4ef443d09ec2/konnectivity-agent/0.log" limit=1024 Jul 01 22:21:39.312722 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:39.312645 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:39.312722 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:39.312714 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls\") pod \"dns-default-7zjsw\" (UID: \"0ed3959f-efbe-43a9-a6f3-d7516f19dcd0\") " pod="openshift-dns/dns-default-7zjsw" Jul 01 22:21:39.312920 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:39.312808 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 22:21:39.312920 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:39.312865 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls podName:0ed3959f-efbe-43a9-a6f3-d7516f19dcd0 nodeName:}" failed. No retries permitted until 2026-07-01 22:21:43.312850495 +0000 UTC m=+44.060291068 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls") pod "dns-default-7zjsw" (UID: "0ed3959f-efbe-43a9-a6f3-d7516f19dcd0") : secret "dns-default-metrics-tls" not found Jul 01 22:21:39.312920 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:39.312814 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 22:21:39.312920 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:39.312913 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77d8dfbf5c-c4b7k: secret "image-registry-tls" not found Jul 01 22:21:39.313076 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:39.312976 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls podName:3740f848-7757-4f34-aac1-5318ff61e81d nodeName:}" failed. No retries permitted until 2026-07-01 22:21:43.312955835 +0000 UTC m=+44.060396428 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls") pod "image-registry-77d8dfbf5c-c4b7k" (UID: "3740f848-7757-4f34-aac1-5318ff61e81d") : secret "image-registry-tls" not found Jul 01 22:21:39.922631 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:39.922598 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5sqqv" event={"ID":"e405fa2c-a5db-47a7-9de0-909629fafdd8","Type":"ContainerStarted","Data":"83ff5d1dad6e4a7cab6b1ebe54480a77d7627851d65143e236a25b94fc337bcd"} Jul 01 22:21:39.939531 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:39.939460 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5sqqv" podStartSLOduration=2.083788826 podStartE2EDuration="4.939446572s" totalImagesPullingTime="2.855657746s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:21:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:21:39.938947468 +0000 UTC m=+40.686388057" watchObservedRunningTime="2026-07-01 22:21:39.939446572 +0000 UTC m=+40.686887162" Jul 01 22:21:41.263887 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:41.263848 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vz9dp_8694157d-ebfe-44aa-ae98-4ef443d09ec2/konnectivity-agent/0.log" limit=1024 Jul 01 22:21:41.347034 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:41.347001 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vz9dp_8694157d-ebfe-44aa-ae98-4ef443d09ec2/konnectivity-agent/0.log" limit=1024 Jul 01 22:21:41.447500 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:41.447477 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vz9dp_8694157d-ebfe-44aa-ae98-4ef443d09ec2/konnectivity-agent/0.log" limit=1024 Jul 01 22:21:41.548818 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:41.548760 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vz9dp_8694157d-ebfe-44aa-ae98-4ef443d09ec2/konnectivity-agent/0.log" limit=1024 Jul 01 22:21:41.648912 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:41.648893 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-vz9dp_8694157d-ebfe-44aa-ae98-4ef443d09ec2/konnectivity-agent/0.log" limit=1024 Jul 01 22:21:43.338742 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:43.338707 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls\") pod \"dns-default-7zjsw\" (UID: \"0ed3959f-efbe-43a9-a6f3-d7516f19dcd0\") " pod="openshift-dns/dns-default-7zjsw" Jul 01 22:21:43.339158 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:43.338750 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:43.339158 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:43.338845 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 22:21:43.339158 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:43.338857 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77d8dfbf5c-c4b7k: secret "image-registry-tls" not found Jul 01 22:21:43.339158 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:43.338870 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 22:21:43.339158 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:43.338912 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls podName:3740f848-7757-4f34-aac1-5318ff61e81d nodeName:}" failed. No retries permitted until 2026-07-01 22:21:51.338899883 +0000 UTC m=+52.086340450 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls") pod "image-registry-77d8dfbf5c-c4b7k" (UID: "3740f848-7757-4f34-aac1-5318ff61e81d") : secret "image-registry-tls" not found Jul 01 22:21:43.339158 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:43.338942 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls podName:0ed3959f-efbe-43a9-a6f3-d7516f19dcd0 nodeName:}" failed. No retries permitted until 2026-07-01 22:21:51.33892433 +0000 UTC m=+52.086364899 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls") pod "dns-default-7zjsw" (UID: "0ed3959f-efbe-43a9-a6f3-d7516f19dcd0") : secret "dns-default-metrics-tls" not found Jul 01 22:21:51.395203 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:51.395165 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls\") pod \"dns-default-7zjsw\" (UID: \"0ed3959f-efbe-43a9-a6f3-d7516f19dcd0\") " pod="openshift-dns/dns-default-7zjsw" Jul 01 22:21:51.395779 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:21:51.395216 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:21:51.395779 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:51.395312 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 22:21:51.395779 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:51.395323 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77d8dfbf5c-c4b7k: secret "image-registry-tls" not found Jul 01 22:21:51.395779 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:51.395327 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 22:21:51.395779 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:51.395370 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls podName:3740f848-7757-4f34-aac1-5318ff61e81d nodeName:}" failed. No retries permitted until 2026-07-01 22:22:07.395356781 +0000 UTC m=+68.142797349 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls") pod "image-registry-77d8dfbf5c-c4b7k" (UID: "3740f848-7757-4f34-aac1-5318ff61e81d") : secret "image-registry-tls" not found Jul 01 22:21:51.395779 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:21:51.395381 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls podName:0ed3959f-efbe-43a9-a6f3-d7516f19dcd0 nodeName:}" failed. No retries permitted until 2026-07-01 22:22:07.39537618 +0000 UTC m=+68.142816748 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls") pod "dns-default-7zjsw" (UID: "0ed3959f-efbe-43a9-a6f3-d7516f19dcd0") : secret "dns-default-metrics-tls" not found Jul 01 22:22:00.906986 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:00.906952 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bwsfz" Jul 01 22:22:01.636486 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.636453 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-mtpjs"] Jul 01 22:22:01.666631 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.666608 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-fw5wx"] Jul 01 22:22:01.666785 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.666739 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-mtpjs" Jul 01 22:22:01.670728 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.670709 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 01 22:22:01.671005 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.670986 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 01 22:22:01.671101 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.671079 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 01 22:22:01.671420 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.671403 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-5zzls\"" Jul 01 22:22:01.680638 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.680619 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-mtpjs"] Jul 01 22:22:01.680713 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.680652 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-fw5wx"] Jul 01 22:22:01.680750 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.680725 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-fw5wx" Jul 01 22:22:01.686288 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.686061 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 01 22:22:01.686288 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.686070 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 01 22:22:01.686288 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.686060 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 01 22:22:01.686288 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.686165 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 01 22:22:01.686288 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.686223 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-x5v9w\"" Jul 01 22:22:01.766284 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.766251 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cdb5968b9-8mthg"] Jul 01 22:22:01.766479 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.766453 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a4590f39-429a-43f3-8c4b-5d2e7fc4b378-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-fw5wx\" (UID: \"a4590f39-429a-43f3-8c4b-5d2e7fc4b378\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-fw5wx" Jul 01 22:22:01.766632 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.766572 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zxkd\" (UniqueName: \"kubernetes.io/projected/a4590f39-429a-43f3-8c4b-5d2e7fc4b378-kube-api-access-5zxkd\") pod \"kube-storage-version-migrator-operator-79964ff6f9-fw5wx\" (UID: \"a4590f39-429a-43f3-8c4b-5d2e7fc4b378\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-fw5wx" Jul 01 22:22:01.766702 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.766643 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a4590f39-429a-43f3-8c4b-5d2e7fc4b378-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-fw5wx\" (UID: \"a4590f39-429a-43f3-8c4b-5d2e7fc4b378\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-fw5wx" Jul 01 22:22:01.766789 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.766768 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d0659aec-8895-4713-bd11-3969478c7989-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-mtpjs\" (UID: \"d0659aec-8895-4713-bd11-3969478c7989\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-mtpjs" Jul 01 22:22:01.766844 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.766813 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrdwk\" (UniqueName: \"kubernetes.io/projected/d0659aec-8895-4713-bd11-3969478c7989-kube-api-access-mrdwk\") pod \"cluster-samples-operator-9ddc5ff76-mtpjs\" (UID: \"d0659aec-8895-4713-bd11-3969478c7989\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-mtpjs" Jul 01 22:22:01.779626 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.779602 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-nxtzc"] Jul 01 22:22:01.779738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.779706 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-8mthg" Jul 01 22:22:01.782797 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.782773 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 01 22:22:01.783042 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.783027 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-t22hq\"" Jul 01 22:22:01.783448 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.783433 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 01 22:22:01.796278 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.796255 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-p95p7"] Jul 01 22:22:01.797284 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.797265 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-nxtzc" Jul 01 22:22:01.805132 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.805115 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 01 22:22:01.805240 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.805113 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 01 22:22:01.805311 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.805240 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 01 22:22:01.805744 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.805730 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 01 22:22:01.806104 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.806082 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-pdbcg\"" Jul 01 22:22:01.806747 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.806731 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-dd9c9df89-c2lhf"] Jul 01 22:22:01.806868 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.806853 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-p95p7" Jul 01 22:22:01.810735 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.810704 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-2gbjk\"" Jul 01 22:22:01.810817 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.810790 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 01 22:22:01.811901 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.811884 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 01 22:22:01.818443 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.818427 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-fkr66"] Jul 01 22:22:01.818567 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.818537 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:01.821394 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.821375 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 01 22:22:01.821599 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.821547 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-c2kvs\"" Jul 01 22:22:01.821690 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.821672 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 01 22:22:01.822610 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.822594 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 01 22:22:01.822720 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.822700 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 01 22:22:01.822868 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.822750 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 01 22:22:01.822868 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.822757 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 01 22:22:01.829214 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.829198 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-nxtzc"] Jul 01 22:22:01.829303 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.829239 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-p95p7"] Jul 01 22:22:01.829303 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.829254 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cdb5968b9-8mthg"] Jul 01 22:22:01.829303 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.829268 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-dd9c9df89-c2lhf"] Jul 01 22:22:01.829303 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.829204 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56499946bc-fkr66" Jul 01 22:22:01.829474 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.829279 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-fkr66"] Jul 01 22:22:01.832478 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.832462 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-bsjcb\"" Jul 01 22:22:01.846078 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.846058 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-z8xh6"] Jul 01 22:22:01.860360 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.860345 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-mxlf8"] Jul 01 22:22:01.860487 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.860471 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" Jul 01 22:22:01.863452 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.863434 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 01 22:22:01.863609 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.863480 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 01 22:22:01.863609 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.863514 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 01 22:22:01.863773 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.863760 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 01 22:22:01.863864 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.863841 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-bqdkc\"" Jul 01 22:22:01.866971 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.866951 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-mxlf8" Jul 01 22:22:01.867340 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.867316 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3e857821-7b73-4345-b75f-eef8b20ef546-config\") pod \"service-ca-operator-5d7b7dc95c-nxtzc\" (UID: \"3e857821-7b73-4345-b75f-eef8b20ef546\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-nxtzc" Jul 01 22:22:01.867438 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.867359 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrdwk\" (UniqueName: \"kubernetes.io/projected/d0659aec-8895-4713-bd11-3969478c7989-kube-api-access-mrdwk\") pod \"cluster-samples-operator-9ddc5ff76-mtpjs\" (UID: \"d0659aec-8895-4713-bd11-3969478c7989\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-mtpjs" Jul 01 22:22:01.867857 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.867711 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zxkd\" (UniqueName: \"kubernetes.io/projected/a4590f39-429a-43f3-8c4b-5d2e7fc4b378-kube-api-access-5zxkd\") pod \"kube-storage-version-migrator-operator-79964ff6f9-fw5wx\" (UID: \"a4590f39-429a-43f3-8c4b-5d2e7fc4b378\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-fw5wx" Jul 01 22:22:01.867857 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.867762 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-stats-auth\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:01.867857 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.867794 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3e857821-7b73-4345-b75f-eef8b20ef546-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-nxtzc\" (UID: \"3e857821-7b73-4345-b75f-eef8b20ef546\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-nxtzc" Jul 01 22:22:01.867857 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.867846 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a4590f39-429a-43f3-8c4b-5d2e7fc4b378-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-fw5wx\" (UID: \"a4590f39-429a-43f3-8c4b-5d2e7fc4b378\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-fw5wx" Jul 01 22:22:01.868086 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.867883 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-metrics-certs\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:01.868086 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.867914 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpcf4\" (UniqueName: \"kubernetes.io/projected/5f7fd383-088b-42e4-bf37-f6ae25e5e263-kube-api-access-rpcf4\") pod \"volume-data-source-validator-69d9f679db-p95p7\" (UID: \"5f7fd383-088b-42e4-bf37-f6ae25e5e263\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-p95p7" Jul 01 22:22:01.868086 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.867943 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/67818314-407c-407b-bd1c-5870354be6bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-8mthg\" (UID: \"67818314-407c-407b-bd1c-5870354be6bd\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-8mthg" Jul 01 22:22:01.868086 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.867967 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-default-certificate\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:01.868086 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.868027 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtl64\" (UniqueName: \"kubernetes.io/projected/540453e5-45c6-4cbd-97ac-8924113b03ed-kube-api-access-dtl64\") pod \"network-check-source-56499946bc-fkr66\" (UID: \"540453e5-45c6-4cbd-97ac-8924113b03ed\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-fkr66" Jul 01 22:22:01.868086 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.868063 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxq8w\" (UniqueName: \"kubernetes.io/projected/39a42bc6-115f-4ce9-ba14-a35288061f5f-kube-api-access-rxq8w\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:01.868363 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.868088 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/67818314-407c-407b-bd1c-5870354be6bd-nginx-conf\") pod \"networking-console-plugin-6cdb5968b9-8mthg\" (UID: \"67818314-407c-407b-bd1c-5870354be6bd\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-8mthg" Jul 01 22:22:01.868363 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.868162 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39a42bc6-115f-4ce9-ba14-a35288061f5f-service-ca-bundle\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:01.868363 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.868247 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d0659aec-8895-4713-bd11-3969478c7989-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-mtpjs\" (UID: \"d0659aec-8895-4713-bd11-3969478c7989\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-mtpjs" Jul 01 22:22:01.868363 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.868276 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5whg\" (UniqueName: \"kubernetes.io/projected/3e857821-7b73-4345-b75f-eef8b20ef546-kube-api-access-m5whg\") pod \"service-ca-operator-5d7b7dc95c-nxtzc\" (UID: \"3e857821-7b73-4345-b75f-eef8b20ef546\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-nxtzc" Jul 01 22:22:01.868363 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.868312 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a4590f39-429a-43f3-8c4b-5d2e7fc4b378-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-fw5wx\" (UID: \"a4590f39-429a-43f3-8c4b-5d2e7fc4b378\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-fw5wx" Jul 01 22:22:01.868363 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:01.868344 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 01 22:22:01.868753 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:01.868412 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0659aec-8895-4713-bd11-3969478c7989-samples-operator-tls podName:d0659aec-8895-4713-bd11-3969478c7989 nodeName:}" failed. No retries permitted until 2026-07-01 22:22:02.368394506 +0000 UTC m=+63.115835090 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d0659aec-8895-4713-bd11-3969478c7989-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-mtpjs" (UID: "d0659aec-8895-4713-bd11-3969478c7989") : secret "samples-operator-tls" not found Jul 01 22:22:01.869019 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.868962 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a4590f39-429a-43f3-8c4b-5d2e7fc4b378-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-fw5wx\" (UID: \"a4590f39-429a-43f3-8c4b-5d2e7fc4b378\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-fw5wx" Jul 01 22:22:01.869855 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.869833 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-z8xh6"] Jul 01 22:22:01.870756 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.870733 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 01 22:22:01.871657 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.871641 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-mxlf8"] Jul 01 22:22:01.871915 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.871898 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 01 22:22:01.872038 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.872013 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 01 22:22:01.872144 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.872096 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 01 22:22:01.872206 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.872015 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-r5k9j\"" Jul 01 22:22:01.872895 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.872869 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 01 22:22:01.873015 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.873001 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a4590f39-429a-43f3-8c4b-5d2e7fc4b378-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-fw5wx\" (UID: \"a4590f39-429a-43f3-8c4b-5d2e7fc4b378\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-fw5wx" Jul 01 22:22:01.882755 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.882735 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zxkd\" (UniqueName: \"kubernetes.io/projected/a4590f39-429a-43f3-8c4b-5d2e7fc4b378-kube-api-access-5zxkd\") pod \"kube-storage-version-migrator-operator-79964ff6f9-fw5wx\" (UID: \"a4590f39-429a-43f3-8c4b-5d2e7fc4b378\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-fw5wx" Jul 01 22:22:01.883641 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.883626 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrdwk\" (UniqueName: \"kubernetes.io/projected/d0659aec-8895-4713-bd11-3969478c7989-kube-api-access-mrdwk\") pod \"cluster-samples-operator-9ddc5ff76-mtpjs\" (UID: \"d0659aec-8895-4713-bd11-3969478c7989\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-mtpjs" Jul 01 22:22:01.942426 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.942375 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5f67b55555-lzdgs"] Jul 01 22:22:01.959426 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.959407 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5f67b55555-lzdgs" Jul 01 22:22:01.964070 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.964047 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5f67b55555-lzdgs"] Jul 01 22:22:01.966465 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.966435 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-ws2z5\"" Jul 01 22:22:01.967050 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.967035 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 01 22:22:01.967455 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.967435 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 01 22:22:01.967681 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.967666 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 01 22:22:01.967841 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.967827 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 01 22:22:01.969778 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.969758 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39a42bc6-115f-4ce9-ba14-a35288061f5f-service-ca-bundle\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:01.969925 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.969909 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgvpd\" (UniqueName: \"kubernetes.io/projected/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-kube-api-access-mgvpd\") pod \"cluster-monitoring-operator-54cb84584-mxlf8\" (UID: \"2af46b0f-9553-4f48-abe5-76bdb3ae6a01\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-mxlf8" Jul 01 22:22:01.970080 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.970058 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw5cp\" (UniqueName: \"kubernetes.io/projected/18708697-4b5c-4f52-9878-e369f8aa9eb2-kube-api-access-cw5cp\") pod \"console-operator-7fdbdf9c99-z8xh6\" (UID: \"18708697-4b5c-4f52-9878-e369f8aa9eb2\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" Jul 01 22:22:01.970201 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.970181 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5whg\" (UniqueName: \"kubernetes.io/projected/3e857821-7b73-4345-b75f-eef8b20ef546-kube-api-access-m5whg\") pod \"service-ca-operator-5d7b7dc95c-nxtzc\" (UID: \"3e857821-7b73-4345-b75f-eef8b20ef546\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-nxtzc" Jul 01 22:22:01.970306 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.970239 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3e857821-7b73-4345-b75f-eef8b20ef546-config\") pod \"service-ca-operator-5d7b7dc95c-nxtzc\" (UID: \"3e857821-7b73-4345-b75f-eef8b20ef546\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-nxtzc" Jul 01 22:22:01.970306 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.970278 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-stats-auth\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:01.970420 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.970303 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3e857821-7b73-4345-b75f-eef8b20ef546-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-nxtzc\" (UID: \"3e857821-7b73-4345-b75f-eef8b20ef546\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-nxtzc" Jul 01 22:22:01.970420 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.970330 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-metrics-certs\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:01.970420 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.970357 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpcf4\" (UniqueName: \"kubernetes.io/projected/5f7fd383-088b-42e4-bf37-f6ae25e5e263-kube-api-access-rpcf4\") pod \"volume-data-source-validator-69d9f679db-p95p7\" (UID: \"5f7fd383-088b-42e4-bf37-f6ae25e5e263\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-p95p7" Jul 01 22:22:01.970420 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.970382 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/67818314-407c-407b-bd1c-5870354be6bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-8mthg\" (UID: \"67818314-407c-407b-bd1c-5870354be6bd\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-8mthg" Jul 01 22:22:01.970420 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.970407 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-mxlf8\" (UID: \"2af46b0f-9553-4f48-abe5-76bdb3ae6a01\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-mxlf8" Jul 01 22:22:01.970784 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.970431 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-default-certificate\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:01.970784 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.970484 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtl64\" (UniqueName: \"kubernetes.io/projected/540453e5-45c6-4cbd-97ac-8924113b03ed-kube-api-access-dtl64\") pod \"network-check-source-56499946bc-fkr66\" (UID: \"540453e5-45c6-4cbd-97ac-8924113b03ed\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-fkr66" Jul 01 22:22:01.970784 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.970506 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-mxlf8\" (UID: \"2af46b0f-9553-4f48-abe5-76bdb3ae6a01\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-mxlf8" Jul 01 22:22:01.970784 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.970534 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/18708697-4b5c-4f52-9878-e369f8aa9eb2-trusted-ca\") pod \"console-operator-7fdbdf9c99-z8xh6\" (UID: \"18708697-4b5c-4f52-9878-e369f8aa9eb2\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" Jul 01 22:22:01.970784 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.970572 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/18708697-4b5c-4f52-9878-e369f8aa9eb2-serving-cert\") pod \"console-operator-7fdbdf9c99-z8xh6\" (UID: \"18708697-4b5c-4f52-9878-e369f8aa9eb2\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" Jul 01 22:22:01.970784 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.970601 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxq8w\" (UniqueName: \"kubernetes.io/projected/39a42bc6-115f-4ce9-ba14-a35288061f5f-kube-api-access-rxq8w\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:01.970784 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.970624 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/67818314-407c-407b-bd1c-5870354be6bd-nginx-conf\") pod \"networking-console-plugin-6cdb5968b9-8mthg\" (UID: \"67818314-407c-407b-bd1c-5870354be6bd\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-8mthg" Jul 01 22:22:01.970784 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.970646 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/18708697-4b5c-4f52-9878-e369f8aa9eb2-config\") pod \"console-operator-7fdbdf9c99-z8xh6\" (UID: \"18708697-4b5c-4f52-9878-e369f8aa9eb2\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" Jul 01 22:22:01.970784 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:01.970780 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39a42bc6-115f-4ce9-ba14-a35288061f5f-service-ca-bundle podName:39a42bc6-115f-4ce9-ba14-a35288061f5f nodeName:}" failed. No retries permitted until 2026-07-01 22:22:02.470765268 +0000 UTC m=+63.218205836 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/39a42bc6-115f-4ce9-ba14-a35288061f5f-service-ca-bundle") pod "router-default-dd9c9df89-c2lhf" (UID: "39a42bc6-115f-4ce9-ba14-a35288061f5f") : configmap references non-existent config key: service-ca.crt Jul 01 22:22:01.971971 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.971584 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/3e857821-7b73-4345-b75f-eef8b20ef546-config\") pod \"service-ca-operator-5d7b7dc95c-nxtzc\" (UID: \"3e857821-7b73-4345-b75f-eef8b20ef546\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-nxtzc" Jul 01 22:22:01.994307 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.975495 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 01 22:22:01.994307 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.976720 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/67818314-407c-407b-bd1c-5870354be6bd-nginx-conf\") pod \"networking-console-plugin-6cdb5968b9-8mthg\" (UID: \"67818314-407c-407b-bd1c-5870354be6bd\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-8mthg" Jul 01 22:22:01.994307 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:01.976972 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 01 22:22:01.994307 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:01.977028 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67818314-407c-407b-bd1c-5870354be6bd-networking-console-plugin-cert podName:67818314-407c-407b-bd1c-5870354be6bd nodeName:}" failed. No retries permitted until 2026-07-01 22:22:02.477012372 +0000 UTC m=+63.224452961 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/67818314-407c-407b-bd1c-5870354be6bd-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-8mthg" (UID: "67818314-407c-407b-bd1c-5870354be6bd") : secret "networking-console-plugin-cert" not found Jul 01 22:22:01.994307 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:01.977082 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 01 22:22:01.994307 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:01.977111 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-metrics-certs podName:39a42bc6-115f-4ce9-ba14-a35288061f5f nodeName:}" failed. No retries permitted until 2026-07-01 22:22:02.477101038 +0000 UTC m=+63.224541623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-metrics-certs") pod "router-default-dd9c9df89-c2lhf" (UID: "39a42bc6-115f-4ce9-ba14-a35288061f5f") : secret "router-metrics-certs-default" not found Jul 01 22:22:01.994307 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.978468 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/3e857821-7b73-4345-b75f-eef8b20ef546-serving-cert\") pod \"service-ca-operator-5d7b7dc95c-nxtzc\" (UID: \"3e857821-7b73-4345-b75f-eef8b20ef546\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-nxtzc" Jul 01 22:22:01.994307 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.981348 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-stats-auth\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:01.994307 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.982223 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-default-certificate\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:01.994307 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.991242 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5whg\" (UniqueName: \"kubernetes.io/projected/3e857821-7b73-4345-b75f-eef8b20ef546-kube-api-access-m5whg\") pod \"service-ca-operator-5d7b7dc95c-nxtzc\" (UID: \"3e857821-7b73-4345-b75f-eef8b20ef546\") " pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-nxtzc" Jul 01 22:22:01.994307 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.993293 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-fw5wx" Jul 01 22:22:01.995305 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.995263 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 01 22:22:01.996747 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.996528 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxq8w\" (UniqueName: \"kubernetes.io/projected/39a42bc6-115f-4ce9-ba14-a35288061f5f-kube-api-access-rxq8w\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:01.997726 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:01.997706 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpcf4\" (UniqueName: \"kubernetes.io/projected/5f7fd383-088b-42e4-bf37-f6ae25e5e263-kube-api-access-rpcf4\") pod \"volume-data-source-validator-69d9f679db-p95p7\" (UID: \"5f7fd383-088b-42e4-bf37-f6ae25e5e263\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-p95p7" Jul 01 22:22:02.002727 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.002709 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 01 22:22:02.011800 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.011780 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtl64\" (UniqueName: \"kubernetes.io/projected/540453e5-45c6-4cbd-97ac-8924113b03ed-kube-api-access-dtl64\") pod \"network-check-source-56499946bc-fkr66\" (UID: \"540453e5-45c6-4cbd-97ac-8924113b03ed\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-fkr66" Jul 01 22:22:02.072023 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.071992 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c496dde0-19fa-46ee-82e2-41871f3fa2b7-service-ca-bundle\") pod \"insights-operator-5f67b55555-lzdgs\" (UID: \"c496dde0-19fa-46ee-82e2-41871f3fa2b7\") " pod="openshift-insights/insights-operator-5f67b55555-lzdgs" Jul 01 22:22:02.072156 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.072058 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-mxlf8\" (UID: \"2af46b0f-9553-4f48-abe5-76bdb3ae6a01\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-mxlf8" Jul 01 22:22:02.072156 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.072119 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-mxlf8\" (UID: \"2af46b0f-9553-4f48-abe5-76bdb3ae6a01\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-mxlf8" Jul 01 22:22:02.072156 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.072151 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/18708697-4b5c-4f52-9878-e369f8aa9eb2-trusted-ca\") pod \"console-operator-7fdbdf9c99-z8xh6\" (UID: \"18708697-4b5c-4f52-9878-e369f8aa9eb2\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" Jul 01 22:22:02.072311 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.072175 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/18708697-4b5c-4f52-9878-e369f8aa9eb2-serving-cert\") pod \"console-operator-7fdbdf9c99-z8xh6\" (UID: \"18708697-4b5c-4f52-9878-e369f8aa9eb2\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" Jul 01 22:22:02.072311 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.072207 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/18708697-4b5c-4f52-9878-e369f8aa9eb2-config\") pod \"console-operator-7fdbdf9c99-z8xh6\" (UID: \"18708697-4b5c-4f52-9878-e369f8aa9eb2\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" Jul 01 22:22:02.072311 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.072244 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgvpd\" (UniqueName: \"kubernetes.io/projected/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-kube-api-access-mgvpd\") pod \"cluster-monitoring-operator-54cb84584-mxlf8\" (UID: \"2af46b0f-9553-4f48-abe5-76bdb3ae6a01\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-mxlf8" Jul 01 22:22:02.072311 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:02.072269 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 22:22:02.072502 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:02.072334 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-cluster-monitoring-operator-tls podName:2af46b0f-9553-4f48-abe5-76bdb3ae6a01 nodeName:}" failed. No retries permitted until 2026-07-01 22:22:02.572320386 +0000 UTC m=+63.319760954 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-mxlf8" (UID: "2af46b0f-9553-4f48-abe5-76bdb3ae6a01") : secret "cluster-monitoring-operator-tls" not found Jul 01 22:22:02.072594 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.072269 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nr84\" (UniqueName: \"kubernetes.io/projected/c496dde0-19fa-46ee-82e2-41871f3fa2b7-kube-api-access-8nr84\") pod \"insights-operator-5f67b55555-lzdgs\" (UID: \"c496dde0-19fa-46ee-82e2-41871f3fa2b7\") " pod="openshift-insights/insights-operator-5f67b55555-lzdgs" Jul 01 22:22:02.072661 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.072617 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c496dde0-19fa-46ee-82e2-41871f3fa2b7-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-lzdgs\" (UID: \"c496dde0-19fa-46ee-82e2-41871f3fa2b7\") " pod="openshift-insights/insights-operator-5f67b55555-lzdgs" Jul 01 22:22:02.072717 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.072660 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c496dde0-19fa-46ee-82e2-41871f3fa2b7-snapshots\") pod \"insights-operator-5f67b55555-lzdgs\" (UID: \"c496dde0-19fa-46ee-82e2-41871f3fa2b7\") " pod="openshift-insights/insights-operator-5f67b55555-lzdgs" Jul 01 22:22:02.072717 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.072685 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c496dde0-19fa-46ee-82e2-41871f3fa2b7-serving-cert\") pod \"insights-operator-5f67b55555-lzdgs\" (UID: \"c496dde0-19fa-46ee-82e2-41871f3fa2b7\") " pod="openshift-insights/insights-operator-5f67b55555-lzdgs" Jul 01 22:22:02.072824 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.072726 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw5cp\" (UniqueName: \"kubernetes.io/projected/18708697-4b5c-4f52-9878-e369f8aa9eb2-kube-api-access-cw5cp\") pod \"console-operator-7fdbdf9c99-z8xh6\" (UID: \"18708697-4b5c-4f52-9878-e369f8aa9eb2\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" Jul 01 22:22:02.072824 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.072781 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c496dde0-19fa-46ee-82e2-41871f3fa2b7-tmp\") pod \"insights-operator-5f67b55555-lzdgs\" (UID: \"c496dde0-19fa-46ee-82e2-41871f3fa2b7\") " pod="openshift-insights/insights-operator-5f67b55555-lzdgs" Jul 01 22:22:02.073030 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.073005 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-telemetry-config\") pod \"cluster-monitoring-operator-54cb84584-mxlf8\" (UID: \"2af46b0f-9553-4f48-abe5-76bdb3ae6a01\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-mxlf8" Jul 01 22:22:02.073446 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.073401 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/18708697-4b5c-4f52-9878-e369f8aa9eb2-config\") pod \"console-operator-7fdbdf9c99-z8xh6\" (UID: \"18708697-4b5c-4f52-9878-e369f8aa9eb2\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" Jul 01 22:22:02.073700 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.073667 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/18708697-4b5c-4f52-9878-e369f8aa9eb2-trusted-ca\") pod \"console-operator-7fdbdf9c99-z8xh6\" (UID: \"18708697-4b5c-4f52-9878-e369f8aa9eb2\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" Jul 01 22:22:02.075513 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.075496 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/18708697-4b5c-4f52-9878-e369f8aa9eb2-serving-cert\") pod \"console-operator-7fdbdf9c99-z8xh6\" (UID: \"18708697-4b5c-4f52-9878-e369f8aa9eb2\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" Jul 01 22:22:02.095385 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.095361 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw5cp\" (UniqueName: \"kubernetes.io/projected/18708697-4b5c-4f52-9878-e369f8aa9eb2-kube-api-access-cw5cp\") pod \"console-operator-7fdbdf9c99-z8xh6\" (UID: \"18708697-4b5c-4f52-9878-e369f8aa9eb2\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" Jul 01 22:22:02.095483 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.095432 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgvpd\" (UniqueName: \"kubernetes.io/projected/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-kube-api-access-mgvpd\") pod \"cluster-monitoring-operator-54cb84584-mxlf8\" (UID: \"2af46b0f-9553-4f48-abe5-76bdb3ae6a01\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-mxlf8" Jul 01 22:22:02.107495 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.107477 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-nxtzc" Jul 01 22:22:02.114086 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.114068 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-p95p7" Jul 01 22:22:02.129768 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.129748 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-fw5wx"] Jul 01 22:22:02.130844 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:22:02.130816 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4590f39_429a_43f3_8c4b_5d2e7fc4b378.slice/crio-4154f55425f5fd94079e01014fe3d3c3f0e1e114984c55b98391daa361b08932 WatchSource:0}: Error finding container 4154f55425f5fd94079e01014fe3d3c3f0e1e114984c55b98391daa361b08932: Status 404 returned error can't find the container with id 4154f55425f5fd94079e01014fe3d3c3f0e1e114984c55b98391daa361b08932 Jul 01 22:22:02.136869 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.136850 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56499946bc-fkr66" Jul 01 22:22:02.170149 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.170089 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" Jul 01 22:22:02.173425 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.173254 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c496dde0-19fa-46ee-82e2-41871f3fa2b7-tmp\") pod \"insights-operator-5f67b55555-lzdgs\" (UID: \"c496dde0-19fa-46ee-82e2-41871f3fa2b7\") " pod="openshift-insights/insights-operator-5f67b55555-lzdgs" Jul 01 22:22:02.173425 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.173294 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c496dde0-19fa-46ee-82e2-41871f3fa2b7-service-ca-bundle\") pod \"insights-operator-5f67b55555-lzdgs\" (UID: \"c496dde0-19fa-46ee-82e2-41871f3fa2b7\") " pod="openshift-insights/insights-operator-5f67b55555-lzdgs" Jul 01 22:22:02.173425 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.173390 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nr84\" (UniqueName: \"kubernetes.io/projected/c496dde0-19fa-46ee-82e2-41871f3fa2b7-kube-api-access-8nr84\") pod \"insights-operator-5f67b55555-lzdgs\" (UID: \"c496dde0-19fa-46ee-82e2-41871f3fa2b7\") " pod="openshift-insights/insights-operator-5f67b55555-lzdgs" Jul 01 22:22:02.173425 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.173416 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c496dde0-19fa-46ee-82e2-41871f3fa2b7-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-lzdgs\" (UID: \"c496dde0-19fa-46ee-82e2-41871f3fa2b7\") " pod="openshift-insights/insights-operator-5f67b55555-lzdgs" Jul 01 22:22:02.173750 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.173457 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c496dde0-19fa-46ee-82e2-41871f3fa2b7-snapshots\") pod \"insights-operator-5f67b55555-lzdgs\" (UID: \"c496dde0-19fa-46ee-82e2-41871f3fa2b7\") " pod="openshift-insights/insights-operator-5f67b55555-lzdgs" Jul 01 22:22:02.173750 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.173484 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c496dde0-19fa-46ee-82e2-41871f3fa2b7-serving-cert\") pod \"insights-operator-5f67b55555-lzdgs\" (UID: \"c496dde0-19fa-46ee-82e2-41871f3fa2b7\") " pod="openshift-insights/insights-operator-5f67b55555-lzdgs" Jul 01 22:22:02.175528 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.175485 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c496dde0-19fa-46ee-82e2-41871f3fa2b7-tmp\") pod \"insights-operator-5f67b55555-lzdgs\" (UID: \"c496dde0-19fa-46ee-82e2-41871f3fa2b7\") " pod="openshift-insights/insights-operator-5f67b55555-lzdgs" Jul 01 22:22:02.177118 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.176697 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c496dde0-19fa-46ee-82e2-41871f3fa2b7-snapshots\") pod \"insights-operator-5f67b55555-lzdgs\" (UID: \"c496dde0-19fa-46ee-82e2-41871f3fa2b7\") " pod="openshift-insights/insights-operator-5f67b55555-lzdgs" Jul 01 22:22:02.177446 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.177320 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c496dde0-19fa-46ee-82e2-41871f3fa2b7-service-ca-bundle\") pod \"insights-operator-5f67b55555-lzdgs\" (UID: \"c496dde0-19fa-46ee-82e2-41871f3fa2b7\") " pod="openshift-insights/insights-operator-5f67b55555-lzdgs" Jul 01 22:22:02.177545 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.177461 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c496dde0-19fa-46ee-82e2-41871f3fa2b7-trusted-ca-bundle\") pod \"insights-operator-5f67b55555-lzdgs\" (UID: \"c496dde0-19fa-46ee-82e2-41871f3fa2b7\") " pod="openshift-insights/insights-operator-5f67b55555-lzdgs" Jul 01 22:22:02.180947 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.180904 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c496dde0-19fa-46ee-82e2-41871f3fa2b7-serving-cert\") pod \"insights-operator-5f67b55555-lzdgs\" (UID: \"c496dde0-19fa-46ee-82e2-41871f3fa2b7\") " pod="openshift-insights/insights-operator-5f67b55555-lzdgs" Jul 01 22:22:02.185673 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.185626 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nr84\" (UniqueName: \"kubernetes.io/projected/c496dde0-19fa-46ee-82e2-41871f3fa2b7-kube-api-access-8nr84\") pod \"insights-operator-5f67b55555-lzdgs\" (UID: \"c496dde0-19fa-46ee-82e2-41871f3fa2b7\") " pod="openshift-insights/insights-operator-5f67b55555-lzdgs" Jul 01 22:22:02.254450 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.254421 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-nxtzc"] Jul 01 22:22:02.258348 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:22:02.258319 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e857821_7b73_4345_b75f_eef8b20ef546.slice/crio-1935bad723a28139a40aa0a00d0d9121b182e17fa6381b574b85464aedf33523 WatchSource:0}: Error finding container 1935bad723a28139a40aa0a00d0d9121b182e17fa6381b574b85464aedf33523: Status 404 returned error can't find the container with id 1935bad723a28139a40aa0a00d0d9121b182e17fa6381b574b85464aedf33523 Jul 01 22:22:02.269065 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.269009 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5f67b55555-lzdgs" Jul 01 22:22:02.275708 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.275685 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-p95p7"] Jul 01 22:22:02.280847 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:22:02.280820 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f7fd383_088b_42e4_bf37_f6ae25e5e263.slice/crio-69661521496cf0a68e8a1074becb7c14f52075c43a5ea463e666aeb59d956909 WatchSource:0}: Error finding container 69661521496cf0a68e8a1074becb7c14f52075c43a5ea463e666aeb59d956909: Status 404 returned error can't find the container with id 69661521496cf0a68e8a1074becb7c14f52075c43a5ea463e666aeb59d956909 Jul 01 22:22:02.291704 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.291675 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-fkr66"] Jul 01 22:22:02.294617 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:22:02.294594 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod540453e5_45c6_4cbd_97ac_8924113b03ed.slice/crio-10372c14395891b0b520459165a5f2fd6f69af17951b22e5e07b04b2279e4623 WatchSource:0}: Error finding container 10372c14395891b0b520459165a5f2fd6f69af17951b22e5e07b04b2279e4623: Status 404 returned error can't find the container with id 10372c14395891b0b520459165a5f2fd6f69af17951b22e5e07b04b2279e4623 Jul 01 22:22:02.323050 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.322013 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-z8xh6"] Jul 01 22:22:02.327740 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:22:02.327715 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18708697_4b5c_4f52_9878_e369f8aa9eb2.slice/crio-ca9f399bf66696cac9caaf660ea234104160f05b8a3093c8d86cb86d61a18fcd WatchSource:0}: Error finding container ca9f399bf66696cac9caaf660ea234104160f05b8a3093c8d86cb86d61a18fcd: Status 404 returned error can't find the container with id ca9f399bf66696cac9caaf660ea234104160f05b8a3093c8d86cb86d61a18fcd Jul 01 22:22:02.375065 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.375037 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d0659aec-8895-4713-bd11-3969478c7989-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-mtpjs\" (UID: \"d0659aec-8895-4713-bd11-3969478c7989\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-mtpjs" Jul 01 22:22:02.375195 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:02.375178 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 01 22:22:02.375249 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:02.375238 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0659aec-8895-4713-bd11-3969478c7989-samples-operator-tls podName:d0659aec-8895-4713-bd11-3969478c7989 nodeName:}" failed. No retries permitted until 2026-07-01 22:22:03.375222928 +0000 UTC m=+64.122663500 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d0659aec-8895-4713-bd11-3969478c7989-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-mtpjs" (UID: "d0659aec-8895-4713-bd11-3969478c7989") : secret "samples-operator-tls" not found Jul 01 22:22:02.398578 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.398538 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5f67b55555-lzdgs"] Jul 01 22:22:02.400691 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:22:02.400671 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc496dde0_19fa_46ee_82e2_41871f3fa2b7.slice/crio-9b1842332016a9e28b8667b4354f2b9a8d3e6c4b61db7505d6353e7cee0f9db1 WatchSource:0}: Error finding container 9b1842332016a9e28b8667b4354f2b9a8d3e6c4b61db7505d6353e7cee0f9db1: Status 404 returned error can't find the container with id 9b1842332016a9e28b8667b4354f2b9a8d3e6c4b61db7505d6353e7cee0f9db1 Jul 01 22:22:02.475676 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.475629 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39a42bc6-115f-4ce9-ba14-a35288061f5f-service-ca-bundle\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:02.475795 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:02.475779 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39a42bc6-115f-4ce9-ba14-a35288061f5f-service-ca-bundle podName:39a42bc6-115f-4ce9-ba14-a35288061f5f nodeName:}" failed. No retries permitted until 2026-07-01 22:22:03.475763966 +0000 UTC m=+64.223204534 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/39a42bc6-115f-4ce9-ba14-a35288061f5f-service-ca-bundle") pod "router-default-dd9c9df89-c2lhf" (UID: "39a42bc6-115f-4ce9-ba14-a35288061f5f") : configmap references non-existent config key: service-ca.crt Jul 01 22:22:02.576564 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.576537 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/67818314-407c-407b-bd1c-5870354be6bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-8mthg\" (UID: \"67818314-407c-407b-bd1c-5870354be6bd\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-8mthg" Jul 01 22:22:02.576656 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.576606 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-mxlf8\" (UID: \"2af46b0f-9553-4f48-abe5-76bdb3ae6a01\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-mxlf8" Jul 01 22:22:02.576717 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:02.576694 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 01 22:22:02.576766 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:02.576741 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 22:22:02.576766 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:02.576747 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 01 22:22:02.576766 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:02.576753 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67818314-407c-407b-bd1c-5870354be6bd-networking-console-plugin-cert podName:67818314-407c-407b-bd1c-5870354be6bd nodeName:}" failed. No retries permitted until 2026-07-01 22:22:03.576736492 +0000 UTC m=+64.324177062 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/67818314-407c-407b-bd1c-5870354be6bd-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-8mthg" (UID: "67818314-407c-407b-bd1c-5870354be6bd") : secret "networking-console-plugin-cert" not found Jul 01 22:22:02.576766 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:02.576696 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-metrics-certs\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:02.576960 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:02.576781 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-cluster-monitoring-operator-tls podName:2af46b0f-9553-4f48-abe5-76bdb3ae6a01 nodeName:}" failed. No retries permitted until 2026-07-01 22:22:03.576770808 +0000 UTC m=+64.324211377 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-mxlf8" (UID: "2af46b0f-9553-4f48-abe5-76bdb3ae6a01") : secret "cluster-monitoring-operator-tls" not found Jul 01 22:22:02.576960 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:02.576804 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-metrics-certs podName:39a42bc6-115f-4ce9-ba14-a35288061f5f nodeName:}" failed. No retries permitted until 2026-07-01 22:22:03.576797847 +0000 UTC m=+64.324238415 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-metrics-certs") pod "router-default-dd9c9df89-c2lhf" (UID: "39a42bc6-115f-4ce9-ba14-a35288061f5f") : secret "router-metrics-certs-default" not found Jul 01 22:22:03.000844 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:03.000807 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-nxtzc" event={"ID":"3e857821-7b73-4345-b75f-eef8b20ef546","Type":"ContainerStarted","Data":"1935bad723a28139a40aa0a00d0d9121b182e17fa6381b574b85464aedf33523"} Jul 01 22:22:03.002759 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:03.002684 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-fw5wx" event={"ID":"a4590f39-429a-43f3-8c4b-5d2e7fc4b378","Type":"ContainerStarted","Data":"4154f55425f5fd94079e01014fe3d3c3f0e1e114984c55b98391daa361b08932"} Jul 01 22:22:03.004723 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:03.004698 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" event={"ID":"18708697-4b5c-4f52-9878-e369f8aa9eb2","Type":"ContainerStarted","Data":"ca9f399bf66696cac9caaf660ea234104160f05b8a3093c8d86cb86d61a18fcd"} Jul 01 22:22:03.006001 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:03.005979 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56499946bc-fkr66" event={"ID":"540453e5-45c6-4cbd-97ac-8924113b03ed","Type":"ContainerStarted","Data":"10372c14395891b0b520459165a5f2fd6f69af17951b22e5e07b04b2279e4623"} Jul 01 22:22:03.007304 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:03.007280 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-lzdgs" event={"ID":"c496dde0-19fa-46ee-82e2-41871f3fa2b7","Type":"ContainerStarted","Data":"9b1842332016a9e28b8667b4354f2b9a8d3e6c4b61db7505d6353e7cee0f9db1"} Jul 01 22:22:03.009236 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:03.009190 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-p95p7" event={"ID":"5f7fd383-088b-42e4-bf37-f6ae25e5e263","Type":"ContainerStarted","Data":"69661521496cf0a68e8a1074becb7c14f52075c43a5ea463e666aeb59d956909"} Jul 01 22:22:03.384153 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:03.384114 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d0659aec-8895-4713-bd11-3969478c7989-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-mtpjs\" (UID: \"d0659aec-8895-4713-bd11-3969478c7989\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-mtpjs" Jul 01 22:22:03.384445 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:03.384427 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 01 22:22:03.384513 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:03.384494 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0659aec-8895-4713-bd11-3969478c7989-samples-operator-tls podName:d0659aec-8895-4713-bd11-3969478c7989 nodeName:}" failed. No retries permitted until 2026-07-01 22:22:05.38447622 +0000 UTC m=+66.131916792 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d0659aec-8895-4713-bd11-3969478c7989-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-mtpjs" (UID: "d0659aec-8895-4713-bd11-3969478c7989") : secret "samples-operator-tls" not found Jul 01 22:22:03.486301 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:03.485716 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39a42bc6-115f-4ce9-ba14-a35288061f5f-service-ca-bundle\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:03.486301 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:03.485907 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39a42bc6-115f-4ce9-ba14-a35288061f5f-service-ca-bundle podName:39a42bc6-115f-4ce9-ba14-a35288061f5f nodeName:}" failed. No retries permitted until 2026-07-01 22:22:05.485887337 +0000 UTC m=+66.233327907 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/39a42bc6-115f-4ce9-ba14-a35288061f5f-service-ca-bundle") pod "router-default-dd9c9df89-c2lhf" (UID: "39a42bc6-115f-4ce9-ba14-a35288061f5f") : configmap references non-existent config key: service-ca.crt Jul 01 22:22:03.586710 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:03.586676 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-metrics-certs\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:03.586863 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:03.586725 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/67818314-407c-407b-bd1c-5870354be6bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-8mthg\" (UID: \"67818314-407c-407b-bd1c-5870354be6bd\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-8mthg" Jul 01 22:22:03.586863 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:03.586782 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-mxlf8\" (UID: \"2af46b0f-9553-4f48-abe5-76bdb3ae6a01\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-mxlf8" Jul 01 22:22:03.586982 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:03.586909 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 22:22:03.586982 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:03.586973 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-cluster-monitoring-operator-tls podName:2af46b0f-9553-4f48-abe5-76bdb3ae6a01 nodeName:}" failed. No retries permitted until 2026-07-01 22:22:05.586951788 +0000 UTC m=+66.334392375 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-mxlf8" (UID: "2af46b0f-9553-4f48-abe5-76bdb3ae6a01") : secret "cluster-monitoring-operator-tls" not found Jul 01 22:22:03.587340 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:03.587237 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 01 22:22:03.587340 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:03.587248 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 01 22:22:03.587340 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:03.587292 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-metrics-certs podName:39a42bc6-115f-4ce9-ba14-a35288061f5f nodeName:}" failed. No retries permitted until 2026-07-01 22:22:05.587276677 +0000 UTC m=+66.334717246 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-metrics-certs") pod "router-default-dd9c9df89-c2lhf" (UID: "39a42bc6-115f-4ce9-ba14-a35288061f5f") : secret "router-metrics-certs-default" not found Jul 01 22:22:03.587340 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:03.587308 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67818314-407c-407b-bd1c-5870354be6bd-networking-console-plugin-cert podName:67818314-407c-407b-bd1c-5870354be6bd nodeName:}" failed. No retries permitted until 2026-07-01 22:22:05.587299697 +0000 UTC m=+66.334740268 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/67818314-407c-407b-bd1c-5870354be6bd-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-8mthg" (UID: "67818314-407c-407b-bd1c-5870354be6bd") : secret "networking-console-plugin-cert" not found Jul 01 22:22:05.404695 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:05.404655 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d0659aec-8895-4713-bd11-3969478c7989-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-mtpjs\" (UID: \"d0659aec-8895-4713-bd11-3969478c7989\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-mtpjs" Jul 01 22:22:05.405110 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:05.404849 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 01 22:22:05.405110 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:05.404931 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0659aec-8895-4713-bd11-3969478c7989-samples-operator-tls podName:d0659aec-8895-4713-bd11-3969478c7989 nodeName:}" failed. No retries permitted until 2026-07-01 22:22:09.404909764 +0000 UTC m=+70.152350355 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d0659aec-8895-4713-bd11-3969478c7989-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-mtpjs" (UID: "d0659aec-8895-4713-bd11-3969478c7989") : secret "samples-operator-tls" not found Jul 01 22:22:05.506240 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:05.506198 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39a42bc6-115f-4ce9-ba14-a35288061f5f-service-ca-bundle\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:05.506388 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:05.506375 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39a42bc6-115f-4ce9-ba14-a35288061f5f-service-ca-bundle podName:39a42bc6-115f-4ce9-ba14-a35288061f5f nodeName:}" failed. No retries permitted until 2026-07-01 22:22:09.506357789 +0000 UTC m=+70.253798376 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/39a42bc6-115f-4ce9-ba14-a35288061f5f-service-ca-bundle") pod "router-default-dd9c9df89-c2lhf" (UID: "39a42bc6-115f-4ce9-ba14-a35288061f5f") : configmap references non-existent config key: service-ca.crt Jul 01 22:22:05.607440 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:05.607403 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-metrics-certs\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:05.607627 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:05.607459 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/67818314-407c-407b-bd1c-5870354be6bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-8mthg\" (UID: \"67818314-407c-407b-bd1c-5870354be6bd\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-8mthg" Jul 01 22:22:05.607627 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:05.607492 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs\") pod \"network-metrics-daemon-kfd7x\" (UID: \"246c052d-11a5-4032-9505-a1628e462a19\") " pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:22:05.607627 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:05.607544 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-mxlf8\" (UID: \"2af46b0f-9553-4f48-abe5-76bdb3ae6a01\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-mxlf8" Jul 01 22:22:05.607627 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:05.607598 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 01 22:22:05.607845 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:05.607654 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 01 22:22:05.607845 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:05.607658 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-metrics-certs podName:39a42bc6-115f-4ce9-ba14-a35288061f5f nodeName:}" failed. No retries permitted until 2026-07-01 22:22:09.607639143 +0000 UTC m=+70.355079715 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-metrics-certs") pod "router-default-dd9c9df89-c2lhf" (UID: "39a42bc6-115f-4ce9-ba14-a35288061f5f") : secret "router-metrics-certs-default" not found Jul 01 22:22:05.607845 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:05.607662 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 22:22:05.607845 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:05.607743 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret\") pod \"global-pull-secret-syncer-d5xct\" (UID: \"f700798a-ec9b-4ee0-8a62-d5ff885135cd\") " pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:22:05.607845 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:05.607776 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-cluster-monitoring-operator-tls podName:2af46b0f-9553-4f48-abe5-76bdb3ae6a01 nodeName:}" failed. No retries permitted until 2026-07-01 22:22:09.607757287 +0000 UTC m=+70.355197868 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-mxlf8" (UID: "2af46b0f-9553-4f48-abe5-76bdb3ae6a01") : secret "cluster-monitoring-operator-tls" not found Jul 01 22:22:05.607845 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:05.607803 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67818314-407c-407b-bd1c-5870354be6bd-networking-console-plugin-cert podName:67818314-407c-407b-bd1c-5870354be6bd nodeName:}" failed. No retries permitted until 2026-07-01 22:22:09.607793122 +0000 UTC m=+70.355233695 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/67818314-407c-407b-bd1c-5870354be6bd-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-8mthg" (UID: "67818314-407c-407b-bd1c-5870354be6bd") : secret "networking-console-plugin-cert" not found Jul 01 22:22:05.610410 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:05.610387 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 01 22:22:05.610410 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:05.610407 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 01 22:22:05.618385 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:05.618366 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 01 22:22:05.618471 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:05.618422 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs podName:246c052d-11a5-4032-9505-a1628e462a19 nodeName:}" failed. No retries permitted until 2026-07-01 22:23:09.618405789 +0000 UTC m=+130.365846357 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs") pod "network-metrics-daemon-kfd7x" (UID: "246c052d-11a5-4032-9505-a1628e462a19") : secret "metrics-daemon-secret" not found Jul 01 22:22:05.620668 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:05.620650 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f700798a-ec9b-4ee0-8a62-d5ff885135cd-original-pull-secret\") pod \"global-pull-secret-syncer-d5xct\" (UID: \"f700798a-ec9b-4ee0-8a62-d5ff885135cd\") " pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:22:05.709018 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:05.708944 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slktq\" (UniqueName: \"kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq\") pod \"network-check-target-8tjzj\" (UID: \"bebea64f-bddc-4c5a-8c28-837fb09e93c5\") " pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:22:05.711997 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:05.711975 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-slktq\" (UniqueName: \"kubernetes.io/projected/bebea64f-bddc-4c5a-8c28-837fb09e93c5-kube-api-access-slktq\") pod \"network-check-target-8tjzj\" (UID: \"bebea64f-bddc-4c5a-8c28-837fb09e93c5\") " pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:22:05.752146 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:05.752118 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-d5xct" Jul 01 22:22:05.858088 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:05.858067 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f7p2h\"" Jul 01 22:22:05.862825 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:05.862806 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:22:07.425261 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:07.424759 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:22:07.425261 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:07.424855 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls\") pod \"dns-default-7zjsw\" (UID: \"0ed3959f-efbe-43a9-a6f3-d7516f19dcd0\") " pod="openshift-dns/dns-default-7zjsw" Jul 01 22:22:07.425261 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:07.425013 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 22:22:07.425261 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:07.425078 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls podName:0ed3959f-efbe-43a9-a6f3-d7516f19dcd0 nodeName:}" failed. No retries permitted until 2026-07-01 22:22:39.425059568 +0000 UTC m=+100.172500145 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls") pod "dns-default-7zjsw" (UID: "0ed3959f-efbe-43a9-a6f3-d7516f19dcd0") : secret "dns-default-metrics-tls" not found Jul 01 22:22:07.425261 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:07.425157 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 22:22:07.425261 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:07.425168 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77d8dfbf5c-c4b7k: secret "image-registry-tls" not found Jul 01 22:22:07.425261 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:07.425202 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls podName:3740f848-7757-4f34-aac1-5318ff61e81d nodeName:}" failed. No retries permitted until 2026-07-01 22:22:39.425190202 +0000 UTC m=+100.172630774 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls") pod "image-registry-77d8dfbf5c-c4b7k" (UID: "3740f848-7757-4f34-aac1-5318ff61e81d") : secret "image-registry-tls" not found Jul 01 22:22:07.499095 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:07.498988 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8tjzj"] Jul 01 22:22:07.531397 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:07.531236 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-d5xct"] Jul 01 22:22:07.533754 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:22:07.533719 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf700798a_ec9b_4ee0_8a62_d5ff885135cd.slice/crio-a9f8e5f311caf2542bbf28bde371f3e1affe6c05e47b94f10f5c57bc7e469671 WatchSource:0}: Error finding container a9f8e5f311caf2542bbf28bde371f3e1affe6c05e47b94f10f5c57bc7e469671: Status 404 returned error can't find the container with id a9f8e5f311caf2542bbf28bde371f3e1affe6c05e47b94f10f5c57bc7e469671 Jul 01 22:22:08.030593 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.030543 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-fw5wx" event={"ID":"a4590f39-429a-43f3-8c4b-5d2e7fc4b378","Type":"ContainerStarted","Data":"3f2e68fd4f2f78eb0e048b73cac15f42e87dee07cf16ef086b6597bede53f51e"} Jul 01 22:22:08.032779 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.032757 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8xh6_18708697-4b5c-4f52-9878-e369f8aa9eb2/console-operator/0.log" Jul 01 22:22:08.032912 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.032799 2580 generic.go:356] "Generic (PLEG): container finished" podID="18708697-4b5c-4f52-9878-e369f8aa9eb2" containerID="cd018136e750a03e4f2c20c97f9921bf05dadaf65723abe5d5ac16b151a1ddc3" exitCode=255 Jul 01 22:22:08.032912 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.032873 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" event={"ID":"18708697-4b5c-4f52-9878-e369f8aa9eb2","Type":"ContainerDied","Data":"cd018136e750a03e4f2c20c97f9921bf05dadaf65723abe5d5ac16b151a1ddc3"} Jul 01 22:22:08.033072 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.033057 2580 scope.go:117] "RemoveContainer" containerID="cd018136e750a03e4f2c20c97f9921bf05dadaf65723abe5d5ac16b151a1ddc3" Jul 01 22:22:08.034755 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.034728 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8tjzj" event={"ID":"bebea64f-bddc-4c5a-8c28-837fb09e93c5","Type":"ContainerStarted","Data":"4df60ffec13eef2cf71150bf0b0af4be01e6c656db9a7e5d8e5d7e6da667fbd1"} Jul 01 22:22:08.034873 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.034774 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8tjzj" event={"ID":"bebea64f-bddc-4c5a-8c28-837fb09e93c5","Type":"ContainerStarted","Data":"e4356016fa2a451303b0a64816b2503c17b5e6fb33393872e1f00a6e6c3cbac0"} Jul 01 22:22:08.035014 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.034999 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:22:08.036919 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.036869 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56499946bc-fkr66" event={"ID":"540453e5-45c6-4cbd-97ac-8924113b03ed","Type":"ContainerStarted","Data":"d74857b0e3c5cc729da20d00d0e8a174f5786a18f103486963d263bd5d2c1775"} Jul 01 22:22:08.038499 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.038479 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-lzdgs" event={"ID":"c496dde0-19fa-46ee-82e2-41871f3fa2b7","Type":"ContainerStarted","Data":"bff3c1e937778307520d5fcd2d4d4c8d9ace6d988e75a321d0de0934ef40d3b3"} Jul 01 22:22:08.039660 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.039639 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d5xct" event={"ID":"f700798a-ec9b-4ee0-8a62-d5ff885135cd","Type":"ContainerStarted","Data":"a9f8e5f311caf2542bbf28bde371f3e1affe6c05e47b94f10f5c57bc7e469671"} Jul 01 22:22:08.042051 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.042027 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-p95p7" event={"ID":"5f7fd383-088b-42e4-bf37-f6ae25e5e263","Type":"ContainerStarted","Data":"a32b5057af4753e6427a2e458bc81b77a56f139df0f00218fb1f1acdf4f33428"} Jul 01 22:22:08.049585 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.049543 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-nxtzc" event={"ID":"3e857821-7b73-4345-b75f-eef8b20ef546","Type":"ContainerStarted","Data":"cc782a0e95af04dbe37ae25aa4aff0cfd8e611f57e940ce9745fc2fc6175e610"} Jul 01 22:22:08.050421 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.050313 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-fw5wx" podStartSLOduration=1.8677197909999999 podStartE2EDuration="7.050298785s" totalImagesPullingTime="5.182578994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:22:08.050117873 +0000 UTC m=+68.797558465" watchObservedRunningTime="2026-07-01 22:22:08.050298785 +0000 UTC m=+68.797739366" Jul 01 22:22:08.070853 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.070815 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-69d9f679db-p95p7" podStartSLOduration=2.051199078 podStartE2EDuration="7.070803151s" totalImagesPullingTime="5.019604073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:22:08.069972361 +0000 UTC m=+68.817412951" watchObservedRunningTime="2026-07-01 22:22:08.070803151 +0000 UTC m=+68.818243783" Jul 01 22:22:08.129151 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.129105 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8tjzj" podStartSLOduration=68.129087195 podStartE2EDuration="1m8.129087195s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:21:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:22:08.127950619 +0000 UTC m=+68.875391207" watchObservedRunningTime="2026-07-01 22:22:08.129087195 +0000 UTC m=+68.876527785" Jul 01 22:22:08.129628 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.129591 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5f67b55555-lzdgs" podStartSLOduration=2.217459675 podStartE2EDuration="7.129582405s" totalImagesPullingTime="4.91212273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:22:08.10213156 +0000 UTC m=+68.849572151" watchObservedRunningTime="2026-07-01 22:22:08.129582405 +0000 UTC m=+68.877022993" Jul 01 22:22:08.148143 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.148095 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-nxtzc" podStartSLOduration=2.093381201 podStartE2EDuration="7.148083812s" totalImagesPullingTime="5.054702611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:22:08.146958386 +0000 UTC m=+68.894398975" watchObservedRunningTime="2026-07-01 22:22:08.148083812 +0000 UTC m=+68.895524450" Jul 01 22:22:08.171589 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.171531 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-56499946bc-fkr66" podStartSLOduration=2.135904567 podStartE2EDuration="7.171513693s" totalImagesPullingTime="5.035609126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:22:08.169748528 +0000 UTC m=+68.917189119" watchObservedRunningTime="2026-07-01 22:22:08.171513693 +0000 UTC m=+68.918954284" Jul 01 22:22:08.559889 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.559853 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-2758l"] Jul 01 22:22:08.581572 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.580912 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-2758l" Jul 01 22:22:08.581996 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.581959 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-2758l"] Jul 01 22:22:08.585487 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.585211 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-tt8l8\"" Jul 01 22:22:08.585487 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.585480 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 01 22:22:08.585698 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.585676 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 01 22:22:08.635995 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.635968 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjdtt\" (UniqueName: \"kubernetes.io/projected/6570158d-3c50-41c9-93ca-0b912e915894-kube-api-access-pjdtt\") pod \"migrator-786f9fbc5-2758l\" (UID: \"6570158d-3c50-41c9-93ca-0b912e915894\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-2758l" Jul 01 22:22:08.736792 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.736756 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjdtt\" (UniqueName: \"kubernetes.io/projected/6570158d-3c50-41c9-93ca-0b912e915894-kube-api-access-pjdtt\") pod \"migrator-786f9fbc5-2758l\" (UID: \"6570158d-3c50-41c9-93ca-0b912e915894\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-2758l" Jul 01 22:22:08.746323 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.746296 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjdtt\" (UniqueName: \"kubernetes.io/projected/6570158d-3c50-41c9-93ca-0b912e915894-kube-api-access-pjdtt\") pod \"migrator-786f9fbc5-2758l\" (UID: \"6570158d-3c50-41c9-93ca-0b912e915894\") " pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-2758l" Jul 01 22:22:08.895878 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:08.895679 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-2758l" Jul 01 22:22:09.057117 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.057031 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8xh6_18708697-4b5c-4f52-9878-e369f8aa9eb2/console-operator/1.log" Jul 01 22:22:09.058544 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.057874 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8xh6_18708697-4b5c-4f52-9878-e369f8aa9eb2/console-operator/0.log" Jul 01 22:22:09.058544 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.057914 2580 generic.go:356] "Generic (PLEG): container finished" podID="18708697-4b5c-4f52-9878-e369f8aa9eb2" containerID="d355f925b75a069f6a1daef75b4eb373c7f062b70c9667149d99b9669c61c90b" exitCode=255 Jul 01 22:22:09.058544 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.058070 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" event={"ID":"18708697-4b5c-4f52-9878-e369f8aa9eb2","Type":"ContainerDied","Data":"d355f925b75a069f6a1daef75b4eb373c7f062b70c9667149d99b9669c61c90b"} Jul 01 22:22:09.058544 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.058122 2580 scope.go:117] "RemoveContainer" containerID="cd018136e750a03e4f2c20c97f9921bf05dadaf65723abe5d5ac16b151a1ddc3" Jul 01 22:22:09.059260 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.059239 2580 scope.go:117] "RemoveContainer" containerID="d355f925b75a069f6a1daef75b4eb373c7f062b70c9667149d99b9669c61c90b" Jul 01 22:22:09.059413 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:09.059386 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-z8xh6_openshift-console-operator(18708697-4b5c-4f52-9878-e369f8aa9eb2)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" podUID="18708697-4b5c-4f52-9878-e369f8aa9eb2" Jul 01 22:22:09.069524 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.069398 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-786f9fbc5-2758l"] Jul 01 22:22:09.074341 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:22:09.074300 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6570158d_3c50_41c9_93ca_0b912e915894.slice/crio-5595d3e26aa088bf8abff4ba66e6b24b2f5a997af9e0568248e2a80bef5f5e31 WatchSource:0}: Error finding container 5595d3e26aa088bf8abff4ba66e6b24b2f5a997af9e0568248e2a80bef5f5e31: Status 404 returned error can't find the container with id 5595d3e26aa088bf8abff4ba66e6b24b2f5a997af9e0568248e2a80bef5f5e31 Jul 01 22:22:09.207881 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.207755 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2st4x"] Jul 01 22:22:09.229653 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.229622 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2st4x"] Jul 01 22:22:09.229814 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.229728 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:09.234488 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.234464 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 01 22:22:09.234623 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.234605 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 01 22:22:09.235477 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.235353 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8gbff\"" Jul 01 22:22:09.342931 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.342904 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/629aff78-cbe4-45e4-8277-3cb809af23ca-data-volume\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:09.343063 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.343035 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkrcn\" (UniqueName: \"kubernetes.io/projected/629aff78-cbe4-45e4-8277-3cb809af23ca-kube-api-access-pkrcn\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:09.343118 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.343077 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/629aff78-cbe4-45e4-8277-3cb809af23ca-crio-socket\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:09.343170 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.343121 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/629aff78-cbe4-45e4-8277-3cb809af23ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:09.343259 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.343240 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/629aff78-cbe4-45e4-8277-3cb809af23ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:09.444315 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.444289 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkrcn\" (UniqueName: \"kubernetes.io/projected/629aff78-cbe4-45e4-8277-3cb809af23ca-kube-api-access-pkrcn\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:09.444441 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.444321 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/629aff78-cbe4-45e4-8277-3cb809af23ca-crio-socket\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:09.444441 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.444341 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/629aff78-cbe4-45e4-8277-3cb809af23ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:09.444441 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.444395 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/629aff78-cbe4-45e4-8277-3cb809af23ca-crio-socket\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:09.444441 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.444437 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/629aff78-cbe4-45e4-8277-3cb809af23ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:09.444668 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.444457 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d0659aec-8895-4713-bd11-3969478c7989-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-mtpjs\" (UID: \"d0659aec-8895-4713-bd11-3969478c7989\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-mtpjs" Jul 01 22:22:09.444668 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:09.444521 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 22:22:09.444668 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:09.444548 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 22:22:09.444668 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:09.444590 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/629aff78-cbe4-45e4-8277-3cb809af23ca-kube-rbac-proxy-cm podName:629aff78-cbe4-45e4-8277-3cb809af23ca nodeName:}" failed. No retries permitted until 2026-07-01 22:22:09.944571869 +0000 UTC m=+70.692012436 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/629aff78-cbe4-45e4-8277-3cb809af23ca-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2st4x" (UID: "629aff78-cbe4-45e4-8277-3cb809af23ca") : configmap "kube-rbac-proxy" not found Jul 01 22:22:09.444668 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:09.444609 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/629aff78-cbe4-45e4-8277-3cb809af23ca-insights-runtime-extractor-tls podName:629aff78-cbe4-45e4-8277-3cb809af23ca nodeName:}" failed. No retries permitted until 2026-07-01 22:22:09.944596798 +0000 UTC m=+70.692037366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/629aff78-cbe4-45e4-8277-3cb809af23ca-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2st4x" (UID: "629aff78-cbe4-45e4-8277-3cb809af23ca") : secret "insights-runtime-extractor-tls" not found Jul 01 22:22:09.444668 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:09.444630 2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 01 22:22:09.444668 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.444644 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/629aff78-cbe4-45e4-8277-3cb809af23ca-data-volume\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:09.444668 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:09.444669 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0659aec-8895-4713-bd11-3969478c7989-samples-operator-tls podName:d0659aec-8895-4713-bd11-3969478c7989 nodeName:}" failed. No retries permitted until 2026-07-01 22:22:17.444657207 +0000 UTC m=+78.192097788 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d0659aec-8895-4713-bd11-3969478c7989-samples-operator-tls") pod "cluster-samples-operator-9ddc5ff76-mtpjs" (UID: "d0659aec-8895-4713-bd11-3969478c7989") : secret "samples-operator-tls" not found Jul 01 22:22:09.444931 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.444859 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/629aff78-cbe4-45e4-8277-3cb809af23ca-data-volume\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:09.453345 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.453320 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkrcn\" (UniqueName: \"kubernetes.io/projected/629aff78-cbe4-45e4-8277-3cb809af23ca-kube-api-access-pkrcn\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:09.545848 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.545776 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39a42bc6-115f-4ce9-ba14-a35288061f5f-service-ca-bundle\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:09.545982 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:09.545952 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39a42bc6-115f-4ce9-ba14-a35288061f5f-service-ca-bundle podName:39a42bc6-115f-4ce9-ba14-a35288061f5f nodeName:}" failed. No retries permitted until 2026-07-01 22:22:17.54592899 +0000 UTC m=+78.293369562 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/39a42bc6-115f-4ce9-ba14-a35288061f5f-service-ca-bundle") pod "router-default-dd9c9df89-c2lhf" (UID: "39a42bc6-115f-4ce9-ba14-a35288061f5f") : configmap references non-existent config key: service-ca.crt Jul 01 22:22:09.647716 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.647678 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-mxlf8\" (UID: \"2af46b0f-9553-4f48-abe5-76bdb3ae6a01\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-mxlf8" Jul 01 22:22:09.648210 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.647899 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-metrics-certs\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:09.648210 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.647945 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/67818314-407c-407b-bd1c-5870354be6bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-8mthg\" (UID: \"67818314-407c-407b-bd1c-5870354be6bd\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-8mthg" Jul 01 22:22:09.648210 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:09.648102 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 01 22:22:09.648210 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:09.648180 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67818314-407c-407b-bd1c-5870354be6bd-networking-console-plugin-cert podName:67818314-407c-407b-bd1c-5870354be6bd nodeName:}" failed. No retries permitted until 2026-07-01 22:22:17.648161682 +0000 UTC m=+78.395602253 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/67818314-407c-407b-bd1c-5870354be6bd-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-8mthg" (UID: "67818314-407c-407b-bd1c-5870354be6bd") : secret "networking-console-plugin-cert" not found Jul 01 22:22:09.648790 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:09.648647 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 01 22:22:09.648790 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:09.648710 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 22:22:09.648992 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:09.648840 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-cluster-monitoring-operator-tls podName:2af46b0f-9553-4f48-abe5-76bdb3ae6a01 nodeName:}" failed. No retries permitted until 2026-07-01 22:22:17.648769103 +0000 UTC m=+78.396209671 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-mxlf8" (UID: "2af46b0f-9553-4f48-abe5-76bdb3ae6a01") : secret "cluster-monitoring-operator-tls" not found Jul 01 22:22:09.648992 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:09.648863 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-metrics-certs podName:39a42bc6-115f-4ce9-ba14-a35288061f5f nodeName:}" failed. No retries permitted until 2026-07-01 22:22:17.648849199 +0000 UTC m=+78.396289767 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-metrics-certs") pod "router-default-dd9c9df89-c2lhf" (UID: "39a42bc6-115f-4ce9-ba14-a35288061f5f") : secret "router-metrics-certs-default" not found Jul 01 22:22:09.951765 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.951723 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/629aff78-cbe4-45e4-8277-3cb809af23ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:09.951933 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:09.951843 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/629aff78-cbe4-45e4-8277-3cb809af23ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:09.951933 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:09.951889 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 22:22:09.952027 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:09.951958 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 22:22:09.952027 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:09.951970 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/629aff78-cbe4-45e4-8277-3cb809af23ca-insights-runtime-extractor-tls podName:629aff78-cbe4-45e4-8277-3cb809af23ca nodeName:}" failed. No retries permitted until 2026-07-01 22:22:10.951953834 +0000 UTC m=+71.699394402 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/629aff78-cbe4-45e4-8277-3cb809af23ca-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2st4x" (UID: "629aff78-cbe4-45e4-8277-3cb809af23ca") : secret "insights-runtime-extractor-tls" not found Jul 01 22:22:09.952027 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:09.952009 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/629aff78-cbe4-45e4-8277-3cb809af23ca-kube-rbac-proxy-cm podName:629aff78-cbe4-45e4-8277-3cb809af23ca nodeName:}" failed. No retries permitted until 2026-07-01 22:22:10.951996395 +0000 UTC m=+71.699436963 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/629aff78-cbe4-45e4-8277-3cb809af23ca-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2st4x" (UID: "629aff78-cbe4-45e4-8277-3cb809af23ca") : configmap "kube-rbac-proxy" not found Jul 01 22:22:10.061584 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:10.061538 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-2758l" event={"ID":"6570158d-3c50-41c9-93ca-0b912e915894","Type":"ContainerStarted","Data":"5595d3e26aa088bf8abff4ba66e6b24b2f5a997af9e0568248e2a80bef5f5e31"} Jul 01 22:22:10.063014 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:10.062979 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8xh6_18708697-4b5c-4f52-9878-e369f8aa9eb2/console-operator/1.log" Jul 01 22:22:10.063312 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:10.063298 2580 scope.go:117] "RemoveContainer" containerID="d355f925b75a069f6a1daef75b4eb373c7f062b70c9667149d99b9669c61c90b" Jul 01 22:22:10.063478 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:10.063459 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-z8xh6_openshift-console-operator(18708697-4b5c-4f52-9878-e369f8aa9eb2)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" podUID="18708697-4b5c-4f52-9878-e369f8aa9eb2" Jul 01 22:22:10.960817 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:10.960738 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/629aff78-cbe4-45e4-8277-3cb809af23ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:10.960817 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:10.960800 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/629aff78-cbe4-45e4-8277-3cb809af23ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:10.961222 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:10.960895 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 22:22:10.961222 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:10.960910 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 22:22:10.961222 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:10.960959 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/629aff78-cbe4-45e4-8277-3cb809af23ca-insights-runtime-extractor-tls podName:629aff78-cbe4-45e4-8277-3cb809af23ca nodeName:}" failed. No retries permitted until 2026-07-01 22:22:12.96094281 +0000 UTC m=+73.708383380 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/629aff78-cbe4-45e4-8277-3cb809af23ca-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2st4x" (UID: "629aff78-cbe4-45e4-8277-3cb809af23ca") : secret "insights-runtime-extractor-tls" not found Jul 01 22:22:10.961222 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:10.960975 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/629aff78-cbe4-45e4-8277-3cb809af23ca-kube-rbac-proxy-cm podName:629aff78-cbe4-45e4-8277-3cb809af23ca nodeName:}" failed. No retries permitted until 2026-07-01 22:22:12.960968596 +0000 UTC m=+73.708409163 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/629aff78-cbe4-45e4-8277-3cb809af23ca-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2st4x" (UID: "629aff78-cbe4-45e4-8277-3cb809af23ca") : configmap "kube-rbac-proxy" not found Jul 01 22:22:11.069055 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:11.068998 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-2758l" event={"ID":"6570158d-3c50-41c9-93ca-0b912e915894","Type":"ContainerStarted","Data":"79d50a6fb1653e6f9d5dcb8a0d8b004bc6d622a2bf01f8983e357f11df4afb7b"} Jul 01 22:22:11.069221 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:11.069080 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-2758l" event={"ID":"6570158d-3c50-41c9-93ca-0b912e915894","Type":"ContainerStarted","Data":"f16d7a5eba7f259ef8985d6554752a8ba6f8bb949b5e20686afac65a6e631818"} Jul 01 22:22:11.088646 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:11.088604 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-786f9fbc5-2758l" podStartSLOduration=1.514304657 podStartE2EDuration="3.088589491s" totalImagesPullingTime="1.574284834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:22:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:22:11.086406596 +0000 UTC m=+71.833847189" watchObservedRunningTime="2026-07-01 22:22:11.088589491 +0000 UTC m=+71.836030081" Jul 01 22:22:11.382830 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:11.382807 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ws2ns_33732b5f-89f5-4e70-a763-cfb5cda678e0/dns-node-resolver/0.log" Jul 01 22:22:12.170397 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:12.170362 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" Jul 01 22:22:12.170868 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:12.170424 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" Jul 01 22:22:12.170868 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:12.170766 2580 scope.go:117] "RemoveContainer" containerID="d355f925b75a069f6a1daef75b4eb373c7f062b70c9667149d99b9669c61c90b" Jul 01 22:22:12.171031 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:12.170915 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-z8xh6_openshift-console-operator(18708697-4b5c-4f52-9878-e369f8aa9eb2)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" podUID="18708697-4b5c-4f52-9878-e369f8aa9eb2" Jul 01 22:22:12.383436 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:12.383407 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wmmbz_fbbd6b24-4540-40b3-b0e8-a9d0b67559f5/node-ca/0.log" Jul 01 22:22:12.784117 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:12.784093 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5sqqv_e405fa2c-a5db-47a7-9de0-909629fafdd8/serve-healthcheck-canary/0.log" Jul 01 22:22:12.977874 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:12.977790 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/629aff78-cbe4-45e4-8277-3cb809af23ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:12.977874 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:12.977844 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/629aff78-cbe4-45e4-8277-3cb809af23ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:12.978103 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:12.977959 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 22:22:12.978103 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:12.978050 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/629aff78-cbe4-45e4-8277-3cb809af23ca-insights-runtime-extractor-tls podName:629aff78-cbe4-45e4-8277-3cb809af23ca nodeName:}" failed. No retries permitted until 2026-07-01 22:22:16.978027203 +0000 UTC m=+77.725467783 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/629aff78-cbe4-45e4-8277-3cb809af23ca-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2st4x" (UID: "629aff78-cbe4-45e4-8277-3cb809af23ca") : secret "insights-runtime-extractor-tls" not found Jul 01 22:22:12.978103 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:12.977967 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 22:22:12.978103 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:12.978096 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/629aff78-cbe4-45e4-8277-3cb809af23ca-kube-rbac-proxy-cm podName:629aff78-cbe4-45e4-8277-3cb809af23ca nodeName:}" failed. No retries permitted until 2026-07-01 22:22:16.97808317 +0000 UTC m=+77.725523738 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/629aff78-cbe4-45e4-8277-3cb809af23ca-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2st4x" (UID: "629aff78-cbe4-45e4-8277-3cb809af23ca") : configmap "kube-rbac-proxy" not found Jul 01 22:22:12.983803 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:12.983784 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-2758l_6570158d-3c50-41c9-93ca-0b912e915894/migrator/0.log" Jul 01 22:22:13.075995 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:13.075962 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-d5xct" event={"ID":"f700798a-ec9b-4ee0-8a62-d5ff885135cd","Type":"ContainerStarted","Data":"567e9da221b239133bedfb042aa288b1c99a3615fa462bdbf7e2315ddc3400a7"} Jul 01 22:22:13.109693 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:13.109650 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-d5xct" podStartSLOduration=68.039075499 podStartE2EDuration="1m13.109634931s" totalImagesPullingTime="5.070559432s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:22:13.108749429 +0000 UTC m=+73.856190021" watchObservedRunningTime="2026-07-01 22:22:13.109634931 +0000 UTC m=+73.857075520" Jul 01 22:22:13.183882 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:13.183861 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-2758l_6570158d-3c50-41c9-93ca-0b912e915894/graceful-termination/0.log" Jul 01 22:22:13.387591 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:13.387547 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-79964ff6f9-fw5wx_a4590f39-429a-43f3-8c4b-5d2e7fc4b378/kube-storage-version-migrator-operator/0.log" Jul 01 22:22:17.009445 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:17.009408 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/629aff78-cbe4-45e4-8277-3cb809af23ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:17.009879 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:17.009460 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/629aff78-cbe4-45e4-8277-3cb809af23ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:17.009879 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:17.009548 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 22:22:17.009879 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:17.009578 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 22:22:17.009879 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:17.009632 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/629aff78-cbe4-45e4-8277-3cb809af23ca-kube-rbac-proxy-cm podName:629aff78-cbe4-45e4-8277-3cb809af23ca nodeName:}" failed. No retries permitted until 2026-07-01 22:22:25.009617598 +0000 UTC m=+85.757058167 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/629aff78-cbe4-45e4-8277-3cb809af23ca-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2st4x" (UID: "629aff78-cbe4-45e4-8277-3cb809af23ca") : configmap "kube-rbac-proxy" not found Jul 01 22:22:17.009879 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:17.009647 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/629aff78-cbe4-45e4-8277-3cb809af23ca-insights-runtime-extractor-tls podName:629aff78-cbe4-45e4-8277-3cb809af23ca nodeName:}" failed. No retries permitted until 2026-07-01 22:22:25.009640954 +0000 UTC m=+85.757081522 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/629aff78-cbe4-45e4-8277-3cb809af23ca-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2st4x" (UID: "629aff78-cbe4-45e4-8277-3cb809af23ca") : secret "insights-runtime-extractor-tls" not found Jul 01 22:22:17.515066 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:17.515034 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d0659aec-8895-4713-bd11-3969478c7989-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-mtpjs\" (UID: \"d0659aec-8895-4713-bd11-3969478c7989\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-mtpjs" Jul 01 22:22:17.517382 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:17.517354 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d0659aec-8895-4713-bd11-3969478c7989-samples-operator-tls\") pod \"cluster-samples-operator-9ddc5ff76-mtpjs\" (UID: \"d0659aec-8895-4713-bd11-3969478c7989\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-mtpjs" Jul 01 22:22:17.575521 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:17.575498 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-mtpjs" Jul 01 22:22:17.615679 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:17.615650 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39a42bc6-115f-4ce9-ba14-a35288061f5f-service-ca-bundle\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:17.615851 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:17.615832 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/39a42bc6-115f-4ce9-ba14-a35288061f5f-service-ca-bundle podName:39a42bc6-115f-4ce9-ba14-a35288061f5f nodeName:}" failed. No retries permitted until 2026-07-01 22:22:33.615812726 +0000 UTC m=+94.363253315 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/39a42bc6-115f-4ce9-ba14-a35288061f5f-service-ca-bundle") pod "router-default-dd9c9df89-c2lhf" (UID: "39a42bc6-115f-4ce9-ba14-a35288061f5f") : configmap references non-existent config key: service-ca.crt Jul 01 22:22:17.702504 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:17.702481 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-mtpjs"] Jul 01 22:22:17.716641 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:17.716616 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-metrics-certs\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:17.716753 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:17.716651 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/67818314-407c-407b-bd1c-5870354be6bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-8mthg\" (UID: \"67818314-407c-407b-bd1c-5870354be6bd\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-8mthg" Jul 01 22:22:17.716789 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:17.716758 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-mxlf8\" (UID: \"2af46b0f-9553-4f48-abe5-76bdb3ae6a01\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-mxlf8" Jul 01 22:22:17.716789 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:17.716761 2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 01 22:22:17.716844 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:17.716819 2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 01 22:22:17.716844 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:17.716827 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-metrics-certs podName:39a42bc6-115f-4ce9-ba14-a35288061f5f nodeName:}" failed. No retries permitted until 2026-07-01 22:22:33.716808391 +0000 UTC m=+94.464248975 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-metrics-certs") pod "router-default-dd9c9df89-c2lhf" (UID: "39a42bc6-115f-4ce9-ba14-a35288061f5f") : secret "router-metrics-certs-default" not found Jul 01 22:22:17.716911 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:17.716848 2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 01 22:22:17.716911 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:17.716856 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67818314-407c-407b-bd1c-5870354be6bd-networking-console-plugin-cert podName:67818314-407c-407b-bd1c-5870354be6bd nodeName:}" failed. No retries permitted until 2026-07-01 22:22:33.716845607 +0000 UTC m=+94.464286175 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/67818314-407c-407b-bd1c-5870354be6bd-networking-console-plugin-cert") pod "networking-console-plugin-6cdb5968b9-8mthg" (UID: "67818314-407c-407b-bd1c-5870354be6bd") : secret "networking-console-plugin-cert" not found Jul 01 22:22:17.716911 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:17.716893 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-cluster-monitoring-operator-tls podName:2af46b0f-9553-4f48-abe5-76bdb3ae6a01 nodeName:}" failed. No retries permitted until 2026-07-01 22:22:33.716882168 +0000 UTC m=+94.464322735 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-54cb84584-mxlf8" (UID: "2af46b0f-9553-4f48-abe5-76bdb3ae6a01") : secret "cluster-monitoring-operator-tls" not found Jul 01 22:22:18.093196 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:18.093165 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-mtpjs" event={"ID":"d0659aec-8895-4713-bd11-3969478c7989","Type":"ContainerStarted","Data":"b30d8fadb05955a3b4b4329993ac2bfedb6a91fc3f8e6abdf069be14b04ca7fc"} Jul 01 22:22:21.106198 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:21.106155 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-mtpjs" event={"ID":"d0659aec-8895-4713-bd11-3969478c7989","Type":"ContainerStarted","Data":"992d7d31e1588338658b0b2f20d45d7a550953c91a5eee740bfe9f9b93ec7ac7"} Jul 01 22:22:21.106596 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:21.106215 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-mtpjs" event={"ID":"d0659aec-8895-4713-bd11-3969478c7989","Type":"ContainerStarted","Data":"4492a0c1ef0eaac1054b43dc4a6bc04fa90e4884a0c24e841da7dbecc38e1dfb"} Jul 01 22:22:21.125161 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:21.125117 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-9ddc5ff76-mtpjs" podStartSLOduration=17.578549528 podStartE2EDuration="20.125102848s" totalImagesPullingTime="2.54655332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:22:21.125092645 +0000 UTC m=+81.872533235" watchObservedRunningTime="2026-07-01 22:22:21.125102848 +0000 UTC m=+81.872543493" Jul 01 22:22:25.078295 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:25.078258 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/629aff78-cbe4-45e4-8277-3cb809af23ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:25.078825 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:25.078425 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/629aff78-cbe4-45e4-8277-3cb809af23ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:25.078825 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:25.078583 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 22:22:25.078825 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:25.078649 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/629aff78-cbe4-45e4-8277-3cb809af23ca-insights-runtime-extractor-tls podName:629aff78-cbe4-45e4-8277-3cb809af23ca nodeName:}" failed. No retries permitted until 2026-07-01 22:22:41.078634899 +0000 UTC m=+101.826075466 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/629aff78-cbe4-45e4-8277-3cb809af23ca-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2st4x" (UID: "629aff78-cbe4-45e4-8277-3cb809af23ca") : secret "insights-runtime-extractor-tls" not found Jul 01 22:22:25.078825 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:25.078784 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/629aff78-cbe4-45e4-8277-3cb809af23ca-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:26.741875 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:26.741847 2580 scope.go:117] "RemoveContainer" containerID="d355f925b75a069f6a1daef75b4eb373c7f062b70c9667149d99b9669c61c90b" Jul 01 22:22:27.125169 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:27.125144 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8xh6_18708697-4b5c-4f52-9878-e369f8aa9eb2/console-operator/2.log" Jul 01 22:22:27.125586 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:27.125548 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8xh6_18708697-4b5c-4f52-9878-e369f8aa9eb2/console-operator/1.log" Jul 01 22:22:27.125636 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:27.125606 2580 generic.go:356] "Generic (PLEG): container finished" podID="18708697-4b5c-4f52-9878-e369f8aa9eb2" containerID="58e4c7dfbfdb29c99b65529f3dd42db62e52c87e31863bf7769229cd245d3666" exitCode=255 Jul 01 22:22:27.125675 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:27.125658 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" event={"ID":"18708697-4b5c-4f52-9878-e369f8aa9eb2","Type":"ContainerDied","Data":"58e4c7dfbfdb29c99b65529f3dd42db62e52c87e31863bf7769229cd245d3666"} Jul 01 22:22:27.125709 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:27.125701 2580 scope.go:117] "RemoveContainer" containerID="d355f925b75a069f6a1daef75b4eb373c7f062b70c9667149d99b9669c61c90b" Jul 01 22:22:27.126064 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:27.126049 2580 scope.go:117] "RemoveContainer" containerID="58e4c7dfbfdb29c99b65529f3dd42db62e52c87e31863bf7769229cd245d3666" Jul 01 22:22:27.126261 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:27.126236 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-z8xh6_openshift-console-operator(18708697-4b5c-4f52-9878-e369f8aa9eb2)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" podUID="18708697-4b5c-4f52-9878-e369f8aa9eb2" Jul 01 22:22:28.130565 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:28.130524 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8xh6_18708697-4b5c-4f52-9878-e369f8aa9eb2/console-operator/2.log" Jul 01 22:22:32.170945 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:32.170912 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" Jul 01 22:22:32.171317 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:32.170992 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" Jul 01 22:22:32.171317 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:32.171302 2580 scope.go:117] "RemoveContainer" containerID="58e4c7dfbfdb29c99b65529f3dd42db62e52c87e31863bf7769229cd245d3666" Jul 01 22:22:32.171474 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:32.171453 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-z8xh6_openshift-console-operator(18708697-4b5c-4f52-9878-e369f8aa9eb2)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" podUID="18708697-4b5c-4f52-9878-e369f8aa9eb2" Jul 01 22:22:33.646112 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:33.646065 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39a42bc6-115f-4ce9-ba14-a35288061f5f-service-ca-bundle\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:33.646705 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:33.646685 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39a42bc6-115f-4ce9-ba14-a35288061f5f-service-ca-bundle\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:33.747365 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:33.747336 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-metrics-certs\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:33.747365 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:33.747369 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/67818314-407c-407b-bd1c-5870354be6bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-8mthg\" (UID: \"67818314-407c-407b-bd1c-5870354be6bd\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-8mthg" Jul 01 22:22:33.747527 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:33.747403 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-mxlf8\" (UID: \"2af46b0f-9553-4f48-abe5-76bdb3ae6a01\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-mxlf8" Jul 01 22:22:33.749881 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:33.749841 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2af46b0f-9553-4f48-abe5-76bdb3ae6a01-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-54cb84584-mxlf8\" (UID: \"2af46b0f-9553-4f48-abe5-76bdb3ae6a01\") " pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-mxlf8" Jul 01 22:22:33.750019 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:33.749890 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/67818314-407c-407b-bd1c-5870354be6bd-networking-console-plugin-cert\") pod \"networking-console-plugin-6cdb5968b9-8mthg\" (UID: \"67818314-407c-407b-bd1c-5870354be6bd\") " pod="openshift-network-console/networking-console-plugin-6cdb5968b9-8mthg" Jul 01 22:22:33.750019 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:33.749989 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39a42bc6-115f-4ce9-ba14-a35288061f5f-metrics-certs\") pod \"router-default-dd9c9df89-c2lhf\" (UID: \"39a42bc6-115f-4ce9-ba14-a35288061f5f\") " pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:33.888590 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:33.888546 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-8mthg" Jul 01 22:22:33.926691 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:33.926667 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:33.982868 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:33.982839 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-mxlf8" Jul 01 22:22:34.043201 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:34.043129 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cdb5968b9-8mthg"] Jul 01 22:22:34.047184 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:22:34.047151 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67818314_407c_407b_bd1c_5870354be6bd.slice/crio-ea636cc3de5de72d60f3f6722d5d1259ddcea8737334412a1e7addab31222c32 WatchSource:0}: Error finding container ea636cc3de5de72d60f3f6722d5d1259ddcea8737334412a1e7addab31222c32: Status 404 returned error can't find the container with id ea636cc3de5de72d60f3f6722d5d1259ddcea8737334412a1e7addab31222c32 Jul 01 22:22:34.087223 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:34.087191 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-dd9c9df89-c2lhf"] Jul 01 22:22:34.091389 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:22:34.091359 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39a42bc6_115f_4ce9_ba14_a35288061f5f.slice/crio-9eb5b6d0355c4e70b050a64bafc52ff72d210b7779ee54ec321bb27d5c8d06ff WatchSource:0}: Error finding container 9eb5b6d0355c4e70b050a64bafc52ff72d210b7779ee54ec321bb27d5c8d06ff: Status 404 returned error can't find the container with id 9eb5b6d0355c4e70b050a64bafc52ff72d210b7779ee54ec321bb27d5c8d06ff Jul 01 22:22:34.117234 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:34.117213 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-54cb84584-mxlf8"] Jul 01 22:22:34.122446 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:22:34.122425 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2af46b0f_9553_4f48_abe5_76bdb3ae6a01.slice/crio-f538dc22de48effafc9f36fb8a009bff18ed5fb67bfc2a8d101187653728c2b0 WatchSource:0}: Error finding container f538dc22de48effafc9f36fb8a009bff18ed5fb67bfc2a8d101187653728c2b0: Status 404 returned error can't find the container with id f538dc22de48effafc9f36fb8a009bff18ed5fb67bfc2a8d101187653728c2b0 Jul 01 22:22:34.148792 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:34.148766 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-mxlf8" event={"ID":"2af46b0f-9553-4f48-abe5-76bdb3ae6a01","Type":"ContainerStarted","Data":"f538dc22de48effafc9f36fb8a009bff18ed5fb67bfc2a8d101187653728c2b0"} Jul 01 22:22:34.149888 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:34.149844 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-8mthg" event={"ID":"67818314-407c-407b-bd1c-5870354be6bd","Type":"ContainerStarted","Data":"ea636cc3de5de72d60f3f6722d5d1259ddcea8737334412a1e7addab31222c32"} Jul 01 22:22:34.150739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:34.150717 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-dd9c9df89-c2lhf" event={"ID":"39a42bc6-115f-4ce9-ba14-a35288061f5f","Type":"ContainerStarted","Data":"9eb5b6d0355c4e70b050a64bafc52ff72d210b7779ee54ec321bb27d5c8d06ff"} Jul 01 22:22:35.156572 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:35.156528 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-dd9c9df89-c2lhf" event={"ID":"39a42bc6-115f-4ce9-ba14-a35288061f5f","Type":"ContainerStarted","Data":"2e6a3e031ad3a15a8611efa8ff2c04065b08714b090e23b7f0de359dc7d78353"} Jul 01 22:22:35.190630 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:35.190523 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-dd9c9df89-c2lhf" podStartSLOduration=34.190507856 podStartE2EDuration="34.190507856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:22:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:22:35.188705159 +0000 UTC m=+95.936145750" watchObservedRunningTime="2026-07-01 22:22:35.190507856 +0000 UTC m=+95.937948446" Jul 01 22:22:35.927145 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:35.927107 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:35.931159 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:35.931134 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:36.161376 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:36.161339 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-8mthg" event={"ID":"67818314-407c-407b-bd1c-5870354be6bd","Type":"ContainerStarted","Data":"f642c1e3a65fdb3a968649fe4259ca735996858f7cf33c7af25df2568a07496a"} Jul 01 22:22:36.162650 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:36.162579 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:36.166500 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:36.166474 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-dd9c9df89-c2lhf" Jul 01 22:22:36.187007 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:36.186932 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cdb5968b9-8mthg" podStartSLOduration=33.990645765 podStartE2EDuration="35.186915687s" totalImagesPullingTime="1.196269922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:22:36.184736118 +0000 UTC m=+96.932176707" watchObservedRunningTime="2026-07-01 22:22:36.186915687 +0000 UTC m=+96.934356278" Jul 01 22:22:37.166226 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:37.166190 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-mxlf8" event={"ID":"2af46b0f-9553-4f48-abe5-76bdb3ae6a01","Type":"ContainerStarted","Data":"3dbc3300fbf3efd3575ba85cb2ba84a103e87ccf3f0f926ae2f48235a27fcade"} Jul 01 22:22:37.186951 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:37.186911 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-54cb84584-mxlf8" podStartSLOduration=34.120006235 podStartE2EDuration="36.186895899s" totalImagesPullingTime="2.066889664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:22:37.185305364 +0000 UTC m=+97.932745958" watchObservedRunningTime="2026-07-01 22:22:37.186895899 +0000 UTC m=+97.934336488" Jul 01 22:22:39.061109 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:39.061030 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8tjzj" Jul 01 22:22:39.495766 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:39.495719 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:22:39.495930 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:39.495800 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls\") pod \"dns-default-7zjsw\" (UID: \"0ed3959f-efbe-43a9-a6f3-d7516f19dcd0\") " pod="openshift-dns/dns-default-7zjsw" Jul 01 22:22:39.498021 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:39.497987 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0ed3959f-efbe-43a9-a6f3-d7516f19dcd0-metrics-tls\") pod \"dns-default-7zjsw\" (UID: \"0ed3959f-efbe-43a9-a6f3-d7516f19dcd0\") " pod="openshift-dns/dns-default-7zjsw" Jul 01 22:22:39.498174 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:39.498151 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls\") pod \"image-registry-77d8dfbf5c-c4b7k\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:22:39.512933 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:39.512915 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5b2tg\"" Jul 01 22:22:39.520206 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:39.520193 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:22:39.528229 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:39.528212 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tjcnx\"" Jul 01 22:22:39.535529 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:39.535506 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7zjsw" Jul 01 22:22:39.676525 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:39.676496 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77d8dfbf5c-c4b7k"] Jul 01 22:22:39.680971 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:22:39.680947 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3740f848_7757_4f34_aac1_5318ff61e81d.slice/crio-24b5a43d72446e60f6800d0bb3b3f2df1300b6c814bbeb989393dac0b303cb48 WatchSource:0}: Error finding container 24b5a43d72446e60f6800d0bb3b3f2df1300b6c814bbeb989393dac0b303cb48: Status 404 returned error can't find the container with id 24b5a43d72446e60f6800d0bb3b3f2df1300b6c814bbeb989393dac0b303cb48 Jul 01 22:22:39.684606 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:39.684581 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7zjsw"] Jul 01 22:22:39.688104 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:22:39.688080 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ed3959f_efbe_43a9_a6f3_d7516f19dcd0.slice/crio-e5e8224aa392c4e84d2ede86f8041353aac0797f44d30195c7e322f95c737d26 WatchSource:0}: Error finding container e5e8224aa392c4e84d2ede86f8041353aac0797f44d30195c7e322f95c737d26: Status 404 returned error can't find the container with id e5e8224aa392c4e84d2ede86f8041353aac0797f44d30195c7e322f95c737d26 Jul 01 22:22:40.177797 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:40.177762 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7zjsw" event={"ID":"0ed3959f-efbe-43a9-a6f3-d7516f19dcd0","Type":"ContainerStarted","Data":"e5e8224aa392c4e84d2ede86f8041353aac0797f44d30195c7e322f95c737d26"} Jul 01 22:22:40.179110 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:40.179079 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" event={"ID":"3740f848-7757-4f34-aac1-5318ff61e81d","Type":"ContainerStarted","Data":"a3005383abccf6d72dc87cd3329e804ab3b7319c5b13c218e108a92ecd990be0"} Jul 01 22:22:40.179240 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:40.179128 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" event={"ID":"3740f848-7757-4f34-aac1-5318ff61e81d","Type":"ContainerStarted","Data":"24b5a43d72446e60f6800d0bb3b3f2df1300b6c814bbeb989393dac0b303cb48"} Jul 01 22:22:40.179297 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:40.179249 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:22:40.202945 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:40.202899 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" podStartSLOduration=100.202886947 podStartE2EDuration="1m40.202886947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:21:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:22:40.201240671 +0000 UTC m=+100.948681262" watchObservedRunningTime="2026-07-01 22:22:40.202886947 +0000 UTC m=+100.950327537" Jul 01 22:22:41.108711 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:41.108673 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/629aff78-cbe4-45e4-8277-3cb809af23ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:41.111391 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:41.111363 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/629aff78-cbe4-45e4-8277-3cb809af23ca-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2st4x\" (UID: \"629aff78-cbe4-45e4-8277-3cb809af23ca\") " pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:41.340719 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:41.340689 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2st4x" Jul 01 22:22:41.560511 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:41.560480 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2st4x"] Jul 01 22:22:41.564140 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:22:41.564112 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod629aff78_cbe4_45e4_8277_3cb809af23ca.slice/crio-89e23717ea12fe97788ce8f4a2c5f18504cb284aba6881a108781f8cd2a95e63 WatchSource:0}: Error finding container 89e23717ea12fe97788ce8f4a2c5f18504cb284aba6881a108781f8cd2a95e63: Status 404 returned error can't find the container with id 89e23717ea12fe97788ce8f4a2c5f18504cb284aba6881a108781f8cd2a95e63 Jul 01 22:22:42.192109 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:42.192021 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7zjsw" event={"ID":"0ed3959f-efbe-43a9-a6f3-d7516f19dcd0","Type":"ContainerStarted","Data":"5857138ac2d208804e508c376c86e1255ed4a1f96ab9d19f7952e1ab4520cac0"} Jul 01 22:22:42.192109 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:42.192094 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7zjsw" event={"ID":"0ed3959f-efbe-43a9-a6f3-d7516f19dcd0","Type":"ContainerStarted","Data":"81912defb176f2c2f237e1764872f54765710081a9184a881b4749dac530c947"} Jul 01 22:22:42.192350 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:42.192172 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7zjsw" Jul 01 22:22:42.193829 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:42.193790 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2st4x" event={"ID":"629aff78-cbe4-45e4-8277-3cb809af23ca","Type":"ContainerStarted","Data":"89e23717ea12fe97788ce8f4a2c5f18504cb284aba6881a108781f8cd2a95e63"} Jul 01 22:22:42.273374 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:42.273312 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7zjsw" podStartSLOduration=65.170241435 podStartE2EDuration="1m7.273291685s" totalImagesPullingTime="2.10305025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:21:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:22:42.271582682 +0000 UTC m=+103.019023271" watchObservedRunningTime="2026-07-01 22:22:42.273291685 +0000 UTC m=+103.020732277" Jul 01 22:22:43.741992 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:43.741962 2580 scope.go:117] "RemoveContainer" containerID="58e4c7dfbfdb29c99b65529f3dd42db62e52c87e31863bf7769229cd245d3666" Jul 01 22:22:43.742355 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:43.742166 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-z8xh6_openshift-console-operator(18708697-4b5c-4f52-9878-e369f8aa9eb2)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" podUID="18708697-4b5c-4f52-9878-e369f8aa9eb2" Jul 01 22:22:46.210656 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:46.210619 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2st4x" event={"ID":"629aff78-cbe4-45e4-8277-3cb809af23ca","Type":"ContainerStarted","Data":"4cd85ae6ce33485919ae938123d8bfa1af0a07860c7bed23577caf44c248f534"} Jul 01 22:22:47.217325 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:47.217280 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2st4x" event={"ID":"629aff78-cbe4-45e4-8277-3cb809af23ca","Type":"ContainerStarted","Data":"cceaef8e50127a02a6e926e2eab2b4a87e2c97e1e64df120d6d94f3bf7fbfcbf"} Jul 01 22:22:48.499567 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.499509 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mzhlc"] Jul 01 22:22:48.505950 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.505924 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.509162 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.509135 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 01 22:22:48.509867 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.509830 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 01 22:22:48.510395 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.510375 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 01 22:22:48.510682 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.510665 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-h64q6\"" Jul 01 22:22:48.515696 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.515674 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 01 22:22:48.565164 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.565139 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2eb01e25-2de7-4fd6-aac5-8682ce27a267-root\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.565283 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.565172 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2eb01e25-2de7-4fd6-aac5-8682ce27a267-node-exporter-wtmp\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.565283 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.565205 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw9cf\" (UniqueName: \"kubernetes.io/projected/2eb01e25-2de7-4fd6-aac5-8682ce27a267-kube-api-access-pw9cf\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.565393 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.565279 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2eb01e25-2de7-4fd6-aac5-8682ce27a267-node-exporter-accelerators-collector-config\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.565393 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.565327 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2eb01e25-2de7-4fd6-aac5-8682ce27a267-metrics-client-ca\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.565393 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.565359 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2eb01e25-2de7-4fd6-aac5-8682ce27a267-node-exporter-textfile\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.565530 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.565393 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2eb01e25-2de7-4fd6-aac5-8682ce27a267-node-exporter-tls\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.565530 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.565450 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2eb01e25-2de7-4fd6-aac5-8682ce27a267-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.565530 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.565501 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2eb01e25-2de7-4fd6-aac5-8682ce27a267-sys\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.666761 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.666726 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2eb01e25-2de7-4fd6-aac5-8682ce27a267-root\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.666921 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.666773 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2eb01e25-2de7-4fd6-aac5-8682ce27a267-node-exporter-wtmp\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.666921 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.666807 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/2eb01e25-2de7-4fd6-aac5-8682ce27a267-root\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.666921 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.666813 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw9cf\" (UniqueName: \"kubernetes.io/projected/2eb01e25-2de7-4fd6-aac5-8682ce27a267-kube-api-access-pw9cf\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.666921 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.666893 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/2eb01e25-2de7-4fd6-aac5-8682ce27a267-node-exporter-wtmp\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.666921 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.666895 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2eb01e25-2de7-4fd6-aac5-8682ce27a267-node-exporter-accelerators-collector-config\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.667196 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.666949 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2eb01e25-2de7-4fd6-aac5-8682ce27a267-metrics-client-ca\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.667196 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.666995 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2eb01e25-2de7-4fd6-aac5-8682ce27a267-node-exporter-textfile\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.667196 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.667026 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2eb01e25-2de7-4fd6-aac5-8682ce27a267-node-exporter-tls\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.667196 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.667050 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2eb01e25-2de7-4fd6-aac5-8682ce27a267-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.667196 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.667084 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2eb01e25-2de7-4fd6-aac5-8682ce27a267-sys\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.667393 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.667237 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/2eb01e25-2de7-4fd6-aac5-8682ce27a267-sys\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.667393 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:48.667323 2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 01 22:22:48.667393 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:22:48.667379 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2eb01e25-2de7-4fd6-aac5-8682ce27a267-node-exporter-tls podName:2eb01e25-2de7-4fd6-aac5-8682ce27a267 nodeName:}" failed. No retries permitted until 2026-07-01 22:22:49.167360051 +0000 UTC m=+109.914800629 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/2eb01e25-2de7-4fd6-aac5-8682ce27a267-node-exporter-tls") pod "node-exporter-mzhlc" (UID: "2eb01e25-2de7-4fd6-aac5-8682ce27a267") : secret "node-exporter-tls" not found Jul 01 22:22:48.667491 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.667389 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/2eb01e25-2de7-4fd6-aac5-8682ce27a267-node-exporter-textfile\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.667522 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.667493 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/2eb01e25-2de7-4fd6-aac5-8682ce27a267-node-exporter-accelerators-collector-config\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.667887 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.667868 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2eb01e25-2de7-4fd6-aac5-8682ce27a267-metrics-client-ca\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.670006 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.669983 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2eb01e25-2de7-4fd6-aac5-8682ce27a267-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:48.684460 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:48.684440 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw9cf\" (UniqueName: \"kubernetes.io/projected/2eb01e25-2de7-4fd6-aac5-8682ce27a267-kube-api-access-pw9cf\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:49.171138 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:49.171107 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2eb01e25-2de7-4fd6-aac5-8682ce27a267-node-exporter-tls\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:49.173308 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:49.173281 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/2eb01e25-2de7-4fd6-aac5-8682ce27a267-node-exporter-tls\") pod \"node-exporter-mzhlc\" (UID: \"2eb01e25-2de7-4fd6-aac5-8682ce27a267\") " pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:49.226962 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:49.226894 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2st4x" event={"ID":"629aff78-cbe4-45e4-8277-3cb809af23ca","Type":"ContainerStarted","Data":"2f1fba07071dcb3ee2353897458ab92d8ec50ed84a76ab3bbda09bea71264c48"} Jul 01 22:22:49.255311 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:49.255269 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2st4x" podStartSLOduration=32.968951682 podStartE2EDuration="40.255255396s" totalImagesPullingTime="7.286303714s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:22:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:22:49.254728468 +0000 UTC m=+110.002169057" watchObservedRunningTime="2026-07-01 22:22:49.255255396 +0000 UTC m=+110.002695986" Jul 01 22:22:49.416522 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:49.416491 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mzhlc" Jul 01 22:22:49.425951 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:22:49.425923 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2eb01e25_2de7_4fd6_aac5_8682ce27a267.slice/crio-672a24c6d3acbebc25b69b17cdf82da14cf966ce98dff3044e6b28cdd26c28d3 WatchSource:0}: Error finding container 672a24c6d3acbebc25b69b17cdf82da14cf966ce98dff3044e6b28cdd26c28d3: Status 404 returned error can't find the container with id 672a24c6d3acbebc25b69b17cdf82da14cf966ce98dff3044e6b28cdd26c28d3 Jul 01 22:22:50.231923 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:50.231884 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mzhlc" event={"ID":"2eb01e25-2de7-4fd6-aac5-8682ce27a267","Type":"ContainerStarted","Data":"672a24c6d3acbebc25b69b17cdf82da14cf966ce98dff3044e6b28cdd26c28d3"} Jul 01 22:22:51.236441 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:51.236411 2580 generic.go:356] "Generic (PLEG): container finished" podID="2eb01e25-2de7-4fd6-aac5-8682ce27a267" containerID="887def0e031ee90a1e690e6ece78e6c831e7f1421c57c1f97439e59ae77967c5" exitCode=0 Jul 01 22:22:51.236829 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:51.236461 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mzhlc" event={"ID":"2eb01e25-2de7-4fd6-aac5-8682ce27a267","Type":"ContainerDied","Data":"887def0e031ee90a1e690e6ece78e6c831e7f1421c57c1f97439e59ae77967c5"} Jul 01 22:22:52.200096 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:52.200065 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7zjsw" Jul 01 22:22:52.243365 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:52.243330 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mzhlc" event={"ID":"2eb01e25-2de7-4fd6-aac5-8682ce27a267","Type":"ContainerStarted","Data":"6787304349c8b84bf0f592c0c8b88da0e7d608c6bf24f96e7612af0f0ab1079b"} Jul 01 22:22:52.243750 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:52.243389 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mzhlc" event={"ID":"2eb01e25-2de7-4fd6-aac5-8682ce27a267","Type":"ContainerStarted","Data":"7edd364bfe41691072bd03e1c7891682145487b0148288657e335a32546d2708"} Jul 01 22:22:52.273421 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:52.273373 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mzhlc" podStartSLOduration=3.29271716 podStartE2EDuration="4.273357622s" totalImagesPullingTime="912.86842ms" totalInitContainerRuntime="67.772042ms" isStatefulPod=true podCreationTimestamp="2026-07-01 22:22:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:22:52.272408915 +0000 UTC m=+113.019849504" watchObservedRunningTime="2026-07-01 22:22:52.273357622 +0000 UTC m=+113.020798267" Jul 01 22:22:57.422726 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:57.422695 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77d8dfbf5c-c4b7k"] Jul 01 22:22:57.426818 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:57.426796 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:22:57.742330 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:57.742245 2580 scope.go:117] "RemoveContainer" containerID="58e4c7dfbfdb29c99b65529f3dd42db62e52c87e31863bf7769229cd245d3666" Jul 01 22:22:58.266771 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:58.266747 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8xh6_18708697-4b5c-4f52-9878-e369f8aa9eb2/console-operator/2.log" Jul 01 22:22:58.266921 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:58.266816 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" event={"ID":"18708697-4b5c-4f52-9878-e369f8aa9eb2","Type":"ContainerStarted","Data":"289910d0614c09e2acd66f3650c8e3966e90cd1bd878863b352aa8e9042c02c4"} Jul 01 22:22:58.267094 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:58.267071 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" Jul 01 22:22:58.284958 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:58.284923 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" Jul 01 22:22:58.285994 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:22:58.285954 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7fdbdf9c99-z8xh6" podStartSLOduration=52.299771524 podStartE2EDuration="57.285939977s" totalImagesPullingTime="4.986168453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:22:58.284214471 +0000 UTC m=+119.031655062" watchObservedRunningTime="2026-07-01 22:22:58.285939977 +0000 UTC m=+119.033380567" Jul 01 22:23:09.621427 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:09.621388 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs\") pod \"network-metrics-daemon-kfd7x\" (UID: \"246c052d-11a5-4032-9505-a1628e462a19\") " pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:23:09.623784 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:09.623760 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/246c052d-11a5-4032-9505-a1628e462a19-metrics-certs\") pod \"network-metrics-daemon-kfd7x\" (UID: \"246c052d-11a5-4032-9505-a1628e462a19\") " pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:23:09.761602 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:09.761571 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-smgl4\"" Jul 01 22:23:09.768637 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:09.768602 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kfd7x" Jul 01 22:23:09.896469 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:09.896407 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kfd7x"] Jul 01 22:23:09.899617 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:23:09.899592 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod246c052d_11a5_4032_9505_a1628e462a19.slice/crio-7044ddb81e07aa0fc6078d2d1ec01c994f5d8ab87f13c95e6a66f35ba01a1ae0 WatchSource:0}: Error finding container 7044ddb81e07aa0fc6078d2d1ec01c994f5d8ab87f13c95e6a66f35ba01a1ae0: Status 404 returned error can't find the container with id 7044ddb81e07aa0fc6078d2d1ec01c994f5d8ab87f13c95e6a66f35ba01a1ae0 Jul 01 22:23:10.311194 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:10.311117 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kfd7x" event={"ID":"246c052d-11a5-4032-9505-a1628e462a19","Type":"ContainerStarted","Data":"7044ddb81e07aa0fc6078d2d1ec01c994f5d8ab87f13c95e6a66f35ba01a1ae0"} Jul 01 22:23:11.317251 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:11.317219 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kfd7x" event={"ID":"246c052d-11a5-4032-9505-a1628e462a19","Type":"ContainerStarted","Data":"2403a63fb509f612491312067b64834d944064c8bf453fe4342f5994ee70c336"} Jul 01 22:23:11.317627 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:11.317276 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kfd7x" event={"ID":"246c052d-11a5-4032-9505-a1628e462a19","Type":"ContainerStarted","Data":"37faf389ffab0128ca7a68cddfcd9c721cf4012a2868e2840b64303145214713"} Jul 01 22:23:11.336236 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:11.336187 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kfd7x" podStartSLOduration=130.51241503 podStartE2EDuration="2m11.336170603s" totalImagesPullingTime="823.755573ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:23:11.334261115 +0000 UTC m=+132.081701706" watchObservedRunningTime="2026-07-01 22:23:11.336170603 +0000 UTC m=+132.083611191" Jul 01 22:23:22.442381 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.442331 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" podUID="3740f848-7757-4f34-aac1-5318ff61e81d" containerName="registry" containerID="cri-o://a3005383abccf6d72dc87cd3329e804ab3b7319c5b13c218e108a92ecd990be0" gracePeriod=30 Jul 01 22:23:22.677746 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.677721 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:23:22.812759 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.812702 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3740f848-7757-4f34-aac1-5318ff61e81d-image-registry-private-configuration\") pod \"3740f848-7757-4f34-aac1-5318ff61e81d\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " Jul 01 22:23:22.812759 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.812736 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3740f848-7757-4f34-aac1-5318ff61e81d-installation-pull-secrets\") pod \"3740f848-7757-4f34-aac1-5318ff61e81d\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " Jul 01 22:23:22.812913 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.812763 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3740f848-7757-4f34-aac1-5318ff61e81d-ca-trust-extracted\") pod \"3740f848-7757-4f34-aac1-5318ff61e81d\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " Jul 01 22:23:22.812913 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.812781 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls\") pod \"3740f848-7757-4f34-aac1-5318ff61e81d\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " Jul 01 22:23:22.812913 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.812813 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcvfg\" (UniqueName: \"kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-kube-api-access-bcvfg\") pod \"3740f848-7757-4f34-aac1-5318ff61e81d\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " Jul 01 22:23:22.812913 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.812849 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-bound-sa-token\") pod \"3740f848-7757-4f34-aac1-5318ff61e81d\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " Jul 01 22:23:22.812913 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.812893 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3740f848-7757-4f34-aac1-5318ff61e81d-trusted-ca\") pod \"3740f848-7757-4f34-aac1-5318ff61e81d\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " Jul 01 22:23:22.813164 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.812930 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3740f848-7757-4f34-aac1-5318ff61e81d-registry-certificates\") pod \"3740f848-7757-4f34-aac1-5318ff61e81d\" (UID: \"3740f848-7757-4f34-aac1-5318ff61e81d\") " Jul 01 22:23:22.814349 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.814124 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3740f848-7757-4f34-aac1-5318ff61e81d-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "3740f848-7757-4f34-aac1-5318ff61e81d" (UID: "3740f848-7757-4f34-aac1-5318ff61e81d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 22:23:22.814617 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.814589 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3740f848-7757-4f34-aac1-5318ff61e81d-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "3740f848-7757-4f34-aac1-5318ff61e81d" (UID: "3740f848-7757-4f34-aac1-5318ff61e81d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 22:23:22.815437 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.815408 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "3740f848-7757-4f34-aac1-5318ff61e81d" (UID: "3740f848-7757-4f34-aac1-5318ff61e81d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 22:23:22.815620 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.815576 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "3740f848-7757-4f34-aac1-5318ff61e81d" (UID: "3740f848-7757-4f34-aac1-5318ff61e81d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 22:23:22.815620 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.815593 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3740f848-7757-4f34-aac1-5318ff61e81d-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "3740f848-7757-4f34-aac1-5318ff61e81d" (UID: "3740f848-7757-4f34-aac1-5318ff61e81d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 22:23:22.815620 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.815585 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-kube-api-access-bcvfg" (OuterVolumeSpecName: "kube-api-access-bcvfg") pod "3740f848-7757-4f34-aac1-5318ff61e81d" (UID: "3740f848-7757-4f34-aac1-5318ff61e81d"). InnerVolumeSpecName "kube-api-access-bcvfg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 22:23:22.815842 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.815705 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3740f848-7757-4f34-aac1-5318ff61e81d-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "3740f848-7757-4f34-aac1-5318ff61e81d" (UID: "3740f848-7757-4f34-aac1-5318ff61e81d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 22:23:22.821337 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.821309 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3740f848-7757-4f34-aac1-5318ff61e81d-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "3740f848-7757-4f34-aac1-5318ff61e81d" (UID: "3740f848-7757-4f34-aac1-5318ff61e81d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 22:23:22.914033 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.914012 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcvfg\" (UniqueName: \"kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-kube-api-access-bcvfg\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:23:22.914033 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.914033 2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-bound-sa-token\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:23:22.914154 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.914042 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3740f848-7757-4f34-aac1-5318ff61e81d-trusted-ca\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:23:22.914154 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.914050 2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3740f848-7757-4f34-aac1-5318ff61e81d-registry-certificates\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:23:22.914154 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.914060 2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3740f848-7757-4f34-aac1-5318ff61e81d-image-registry-private-configuration\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:23:22.914154 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.914069 2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3740f848-7757-4f34-aac1-5318ff61e81d-installation-pull-secrets\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:23:22.914154 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.914077 2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3740f848-7757-4f34-aac1-5318ff61e81d-ca-trust-extracted\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:23:22.914154 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:22.914086 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3740f848-7757-4f34-aac1-5318ff61e81d-registry-tls\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:23:23.365018 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:23.364989 2580 generic.go:356] "Generic (PLEG): container finished" podID="c496dde0-19fa-46ee-82e2-41871f3fa2b7" containerID="bff3c1e937778307520d5fcd2d4d4c8d9ace6d988e75a321d0de0934ef40d3b3" exitCode=0 Jul 01 22:23:23.365153 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:23.365062 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-lzdgs" event={"ID":"c496dde0-19fa-46ee-82e2-41871f3fa2b7","Type":"ContainerDied","Data":"bff3c1e937778307520d5fcd2d4d4c8d9ace6d988e75a321d0de0934ef40d3b3"} Jul 01 22:23:23.365453 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:23.365431 2580 scope.go:117] "RemoveContainer" containerID="bff3c1e937778307520d5fcd2d4d4c8d9ace6d988e75a321d0de0934ef40d3b3" Jul 01 22:23:23.366358 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:23.366335 2580 generic.go:356] "Generic (PLEG): container finished" podID="3740f848-7757-4f34-aac1-5318ff61e81d" containerID="a3005383abccf6d72dc87cd3329e804ab3b7319c5b13c218e108a92ecd990be0" exitCode=0 Jul 01 22:23:23.366453 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:23.366386 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" Jul 01 22:23:23.366523 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:23.366447 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" event={"ID":"3740f848-7757-4f34-aac1-5318ff61e81d","Type":"ContainerDied","Data":"a3005383abccf6d72dc87cd3329e804ab3b7319c5b13c218e108a92ecd990be0"} Jul 01 22:23:23.366523 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:23.366498 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77d8dfbf5c-c4b7k" event={"ID":"3740f848-7757-4f34-aac1-5318ff61e81d","Type":"ContainerDied","Data":"24b5a43d72446e60f6800d0bb3b3f2df1300b6c814bbeb989393dac0b303cb48"} Jul 01 22:23:23.366666 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:23.366528 2580 scope.go:117] "RemoveContainer" containerID="a3005383abccf6d72dc87cd3329e804ab3b7319c5b13c218e108a92ecd990be0" Jul 01 22:23:23.376450 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:23.376431 2580 scope.go:117] "RemoveContainer" containerID="a3005383abccf6d72dc87cd3329e804ab3b7319c5b13c218e108a92ecd990be0" Jul 01 22:23:23.376786 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:23:23.376748 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3005383abccf6d72dc87cd3329e804ab3b7319c5b13c218e108a92ecd990be0\": container with ID starting with a3005383abccf6d72dc87cd3329e804ab3b7319c5b13c218e108a92ecd990be0 not found: ID does not exist" containerID="a3005383abccf6d72dc87cd3329e804ab3b7319c5b13c218e108a92ecd990be0" Jul 01 22:23:23.376886 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:23.376793 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3005383abccf6d72dc87cd3329e804ab3b7319c5b13c218e108a92ecd990be0"} err="failed to get container status \"a3005383abccf6d72dc87cd3329e804ab3b7319c5b13c218e108a92ecd990be0\": rpc error: code = NotFound desc = could not find container \"a3005383abccf6d72dc87cd3329e804ab3b7319c5b13c218e108a92ecd990be0\": container with ID starting with a3005383abccf6d72dc87cd3329e804ab3b7319c5b13c218e108a92ecd990be0 not found: ID does not exist" Jul 01 22:23:23.403975 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:23.403932 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77d8dfbf5c-c4b7k"] Jul 01 22:23:23.410879 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:23.410858 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-77d8dfbf5c-c4b7k"] Jul 01 22:23:23.746412 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:23.746341 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3740f848-7757-4f34-aac1-5318ff61e81d" path="/var/lib/kubelet/pods/3740f848-7757-4f34-aac1-5318ff61e81d/volumes" Jul 01 22:23:24.372376 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:24.372339 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5f67b55555-lzdgs" event={"ID":"c496dde0-19fa-46ee-82e2-41871f3fa2b7","Type":"ContainerStarted","Data":"267d4eb2bff7acf3f8109e73b2fc832e88f8971e2884476f08d4cc39e78c24e1"} Jul 01 22:23:25.437280 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:25.437251 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-mxlf8_2af46b0f-9553-4f48-abe5-76bdb3ae6a01/cluster-monitoring-operator/0.log" Jul 01 22:23:27.235448 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:27.235421 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mzhlc_2eb01e25-2de7-4fd6-aac5-8682ce27a267/init-textfile/0.log" Jul 01 22:23:27.436420 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:27.436395 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mzhlc_2eb01e25-2de7-4fd6-aac5-8682ce27a267/node-exporter/0.log" Jul 01 22:23:27.634505 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:27.634482 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mzhlc_2eb01e25-2de7-4fd6-aac5-8682ce27a267/kube-rbac-proxy/0.log" Jul 01 22:23:32.834771 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:32.834735 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cdb5968b9-8mthg_67818314-407c-407b-bd1c-5870354be6bd/networking-console-plugin/0.log" Jul 01 22:23:33.034394 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:33.034371 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8xh6_18708697-4b5c-4f52-9878-e369f8aa9eb2/console-operator/2.log" Jul 01 22:23:33.237910 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:33.237879 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8xh6_18708697-4b5c-4f52-9878-e369f8aa9eb2/console-operator/3.log" Jul 01 22:23:33.408218 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:33.408191 2580 generic.go:356] "Generic (PLEG): container finished" podID="a4590f39-429a-43f3-8c4b-5d2e7fc4b378" containerID="3f2e68fd4f2f78eb0e048b73cac15f42e87dee07cf16ef086b6597bede53f51e" exitCode=0 Jul 01 22:23:33.408352 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:33.408277 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-fw5wx" event={"ID":"a4590f39-429a-43f3-8c4b-5d2e7fc4b378","Type":"ContainerDied","Data":"3f2e68fd4f2f78eb0e048b73cac15f42e87dee07cf16ef086b6597bede53f51e"} Jul 01 22:23:33.408620 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:33.408603 2580 scope.go:117] "RemoveContainer" containerID="3f2e68fd4f2f78eb0e048b73cac15f42e87dee07cf16ef086b6597bede53f51e" Jul 01 22:23:34.415306 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:34.415266 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-fw5wx" event={"ID":"a4590f39-429a-43f3-8c4b-5d2e7fc4b378","Type":"ContainerStarted","Data":"7e968c01ce1839da6567589c2bdbcb243abe8535af32ddd470057eefe33ba764"} Jul 01 22:23:38.434604 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:38.434542 2580 generic.go:356] "Generic (PLEG): container finished" podID="3e857821-7b73-4345-b75f-eef8b20ef546" containerID="cc782a0e95af04dbe37ae25aa4aff0cfd8e611f57e940ce9745fc2fc6175e610" exitCode=0 Jul 01 22:23:38.434966 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:38.434623 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-nxtzc" event={"ID":"3e857821-7b73-4345-b75f-eef8b20ef546","Type":"ContainerDied","Data":"cc782a0e95af04dbe37ae25aa4aff0cfd8e611f57e940ce9745fc2fc6175e610"} Jul 01 22:23:38.434966 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:38.434935 2580 scope.go:117] "RemoveContainer" containerID="cc782a0e95af04dbe37ae25aa4aff0cfd8e611f57e940ce9745fc2fc6175e610" Jul 01 22:23:39.441067 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:23:39.441033 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d7b7dc95c-nxtzc" event={"ID":"3e857821-7b73-4345-b75f-eef8b20ef546","Type":"ContainerStarted","Data":"1ba40c38bfda55276f5cf0c058a1ad9481531f320a1a72c5f0d519a3174cc3cd"} Jul 01 22:24:38.782354 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:38.782321 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c75b9ccb-qrtfm"] Jul 01 22:24:38.782752 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:38.782600 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3740f848-7757-4f34-aac1-5318ff61e81d" containerName="registry" Jul 01 22:24:38.782752 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:38.782612 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3740f848-7757-4f34-aac1-5318ff61e81d" containerName="registry" Jul 01 22:24:38.782752 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:38.782656 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3740f848-7757-4f34-aac1-5318ff61e81d" containerName="registry" Jul 01 22:24:38.785509 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:38.785493 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c75b9ccb-qrtfm" Jul 01 22:24:38.789263 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:38.789238 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 01 22:24:38.789263 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:38.789253 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 01 22:24:38.789443 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:38.789259 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 01 22:24:38.789443 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:38.789257 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 01 22:24:38.799296 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:38.799258 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c75b9ccb-qrtfm"] Jul 01 22:24:38.842607 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:38.842579 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d976\" (UniqueName: \"kubernetes.io/projected/d464f449-a49a-4f69-87a0-7646afac6c67-kube-api-access-6d976\") pod \"klusterlet-addon-workmgr-84c75b9ccb-qrtfm\" (UID: \"d464f449-a49a-4f69-87a0-7646afac6c67\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c75b9ccb-qrtfm" Jul 01 22:24:38.842717 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:38.842627 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d464f449-a49a-4f69-87a0-7646afac6c67-klusterlet-config\") pod \"klusterlet-addon-workmgr-84c75b9ccb-qrtfm\" (UID: \"d464f449-a49a-4f69-87a0-7646afac6c67\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c75b9ccb-qrtfm" Jul 01 22:24:38.842717 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:38.842656 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d464f449-a49a-4f69-87a0-7646afac6c67-tmp\") pod \"klusterlet-addon-workmgr-84c75b9ccb-qrtfm\" (UID: \"d464f449-a49a-4f69-87a0-7646afac6c67\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c75b9ccb-qrtfm" Jul 01 22:24:38.944061 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:38.943970 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d976\" (UniqueName: \"kubernetes.io/projected/d464f449-a49a-4f69-87a0-7646afac6c67-kube-api-access-6d976\") pod \"klusterlet-addon-workmgr-84c75b9ccb-qrtfm\" (UID: \"d464f449-a49a-4f69-87a0-7646afac6c67\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c75b9ccb-qrtfm" Jul 01 22:24:38.944061 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:38.944042 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d464f449-a49a-4f69-87a0-7646afac6c67-klusterlet-config\") pod \"klusterlet-addon-workmgr-84c75b9ccb-qrtfm\" (UID: \"d464f449-a49a-4f69-87a0-7646afac6c67\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c75b9ccb-qrtfm" Jul 01 22:24:38.944407 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:38.944072 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d464f449-a49a-4f69-87a0-7646afac6c67-tmp\") pod \"klusterlet-addon-workmgr-84c75b9ccb-qrtfm\" (UID: \"d464f449-a49a-4f69-87a0-7646afac6c67\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c75b9ccb-qrtfm" Jul 01 22:24:38.944484 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:38.944446 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d464f449-a49a-4f69-87a0-7646afac6c67-tmp\") pod \"klusterlet-addon-workmgr-84c75b9ccb-qrtfm\" (UID: \"d464f449-a49a-4f69-87a0-7646afac6c67\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c75b9ccb-qrtfm" Jul 01 22:24:38.946631 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:38.946605 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d464f449-a49a-4f69-87a0-7646afac6c67-klusterlet-config\") pod \"klusterlet-addon-workmgr-84c75b9ccb-qrtfm\" (UID: \"d464f449-a49a-4f69-87a0-7646afac6c67\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c75b9ccb-qrtfm" Jul 01 22:24:38.954754 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:38.954732 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d976\" (UniqueName: \"kubernetes.io/projected/d464f449-a49a-4f69-87a0-7646afac6c67-kube-api-access-6d976\") pod \"klusterlet-addon-workmgr-84c75b9ccb-qrtfm\" (UID: \"d464f449-a49a-4f69-87a0-7646afac6c67\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c75b9ccb-qrtfm" Jul 01 22:24:39.094263 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:39.094238 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c75b9ccb-qrtfm" Jul 01 22:24:39.223959 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:39.223889 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c75b9ccb-qrtfm"] Jul 01 22:24:39.228137 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:24:39.228109 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd464f449_a49a_4f69_87a0_7646afac6c67.slice/crio-f1b4365b861ebc710cb457ed916f7cec80a10fa99c2493017504fa5147b2f25c WatchSource:0}: Error finding container f1b4365b861ebc710cb457ed916f7cec80a10fa99c2493017504fa5147b2f25c: Status 404 returned error can't find the container with id f1b4365b861ebc710cb457ed916f7cec80a10fa99c2493017504fa5147b2f25c Jul 01 22:24:39.679535 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:39.679502 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c75b9ccb-qrtfm" event={"ID":"d464f449-a49a-4f69-87a0-7646afac6c67","Type":"ContainerStarted","Data":"f1b4365b861ebc710cb457ed916f7cec80a10fa99c2493017504fa5147b2f25c"} Jul 01 22:24:42.693760 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:42.693705 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c75b9ccb-qrtfm" event={"ID":"d464f449-a49a-4f69-87a0-7646afac6c67","Type":"ContainerStarted","Data":"9ded2ab9f597f2d46cd33d5be5f1076b2d192c3a2ac27069dd2a3d22c5cc5554"} Jul 01 22:24:42.694160 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:42.693897 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c75b9ccb-qrtfm" Jul 01 22:24:42.695191 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:42.695171 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c75b9ccb-qrtfm" Jul 01 22:24:42.716540 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:24:42.716458 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-84c75b9ccb-qrtfm" podStartSLOduration=1.349673799 podStartE2EDuration="4.716441382s" totalImagesPullingTime="3.366767583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:24:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:24:42.714989742 +0000 UTC m=+223.462430333" watchObservedRunningTime="2026-07-01 22:24:42.716441382 +0000 UTC m=+223.463881976" Jul 01 22:25:09.889034 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:09.888997 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-6sd4w"] Jul 01 22:25:09.893418 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:09.893392 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-6sd4w" Jul 01 22:25:09.896348 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:09.896321 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-w7ch6\"" Jul 01 22:25:09.896348 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:09.896331 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 01 22:25:09.896535 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:09.896329 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 01 22:25:09.902568 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:09.902536 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-6sd4w"] Jul 01 22:25:09.959317 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:09.959293 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a305ee3-9389-46bc-a821-96e81a002b7c-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-6sd4w\" (UID: \"3a305ee3-9389-46bc-a821-96e81a002b7c\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-6sd4w" Jul 01 22:25:09.959420 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:09.959344 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58lwr\" (UniqueName: \"kubernetes.io/projected/3a305ee3-9389-46bc-a821-96e81a002b7c-kube-api-access-58lwr\") pod \"cert-manager-webhook-7dc76cbb94-6sd4w\" (UID: \"3a305ee3-9389-46bc-a821-96e81a002b7c\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-6sd4w" Jul 01 22:25:10.059836 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:10.059811 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a305ee3-9389-46bc-a821-96e81a002b7c-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-6sd4w\" (UID: \"3a305ee3-9389-46bc-a821-96e81a002b7c\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-6sd4w" Jul 01 22:25:10.059931 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:10.059841 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58lwr\" (UniqueName: \"kubernetes.io/projected/3a305ee3-9389-46bc-a821-96e81a002b7c-kube-api-access-58lwr\") pod \"cert-manager-webhook-7dc76cbb94-6sd4w\" (UID: \"3a305ee3-9389-46bc-a821-96e81a002b7c\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-6sd4w" Jul 01 22:25:10.068442 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:10.068415 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3a305ee3-9389-46bc-a821-96e81a002b7c-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-6sd4w\" (UID: \"3a305ee3-9389-46bc-a821-96e81a002b7c\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-6sd4w" Jul 01 22:25:10.068530 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:10.068459 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58lwr\" (UniqueName: \"kubernetes.io/projected/3a305ee3-9389-46bc-a821-96e81a002b7c-kube-api-access-58lwr\") pod \"cert-manager-webhook-7dc76cbb94-6sd4w\" (UID: \"3a305ee3-9389-46bc-a821-96e81a002b7c\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-6sd4w" Jul 01 22:25:10.216803 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:10.216747 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-6sd4w" Jul 01 22:25:10.347256 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:10.347233 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-6sd4w"] Jul 01 22:25:10.349789 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:25:10.349764 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3a305ee3_9389_46bc_a821_96e81a002b7c.slice/crio-51fef422380e94ec6b3a31030cfb950780328837a805131c560fe176b79cae22 WatchSource:0}: Error finding container 51fef422380e94ec6b3a31030cfb950780328837a805131c560fe176b79cae22: Status 404 returned error can't find the container with id 51fef422380e94ec6b3a31030cfb950780328837a805131c560fe176b79cae22 Jul 01 22:25:10.794580 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:10.794533 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-6sd4w" event={"ID":"3a305ee3-9389-46bc-a821-96e81a002b7c","Type":"ContainerStarted","Data":"51fef422380e94ec6b3a31030cfb950780328837a805131c560fe176b79cae22"} Jul 01 22:25:14.811967 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:14.811932 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-6sd4w" event={"ID":"3a305ee3-9389-46bc-a821-96e81a002b7c","Type":"ContainerStarted","Data":"1072ab6a80742a3c09fc7dcb311982bffa59a750bd4afdbe449016dd2723c3bb"} Jul 01 22:25:14.812366 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:14.811988 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-6sd4w" Jul 01 22:25:14.831610 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:14.831566 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-7dc76cbb94-6sd4w" podStartSLOduration=2.133461898 podStartE2EDuration="5.831539173s" totalImagesPullingTime="3.698077275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:25:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:25:14.829711193 +0000 UTC m=+255.577151777" watchObservedRunningTime="2026-07-01 22:25:14.831539173 +0000 UTC m=+255.578979763" Jul 01 22:25:20.816975 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:20.816940 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-6sd4w" Jul 01 22:25:43.404252 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.404217 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln"] Jul 01 22:25:43.412430 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.412406 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" Jul 01 22:25:43.415901 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.415877 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 01 22:25:43.415901 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.415888 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 01 22:25:43.416089 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.415960 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 01 22:25:43.416219 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.416202 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cc98k\"" Jul 01 22:25:43.416307 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.416273 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 01 22:25:43.429533 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.429511 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln"] Jul 01 22:25:43.492995 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.492962 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e5c96b43-1669-4a57-8a32-a8eb28e35d00-manifests\") pod \"opendatahub-operator-controller-manager-7b6ffc8659-cqbln\" (UID: \"e5c96b43-1669-4a57-8a32-a8eb28e35d00\") " pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" Jul 01 22:25:43.493139 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.493016 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e5c96b43-1669-4a57-8a32-a8eb28e35d00-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b6ffc8659-cqbln\" (UID: \"e5c96b43-1669-4a57-8a32-a8eb28e35d00\") " pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" Jul 01 22:25:43.493139 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.493037 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkkx2\" (UniqueName: \"kubernetes.io/projected/e5c96b43-1669-4a57-8a32-a8eb28e35d00-kube-api-access-xkkx2\") pod \"opendatahub-operator-controller-manager-7b6ffc8659-cqbln\" (UID: \"e5c96b43-1669-4a57-8a32-a8eb28e35d00\") " pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" Jul 01 22:25:43.493139 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.493075 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5c96b43-1669-4a57-8a32-a8eb28e35d00-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b6ffc8659-cqbln\" (UID: \"e5c96b43-1669-4a57-8a32-a8eb28e35d00\") " pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" Jul 01 22:25:43.493139 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.493094 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5c96b43-1669-4a57-8a32-a8eb28e35d00-tmp\") pod \"opendatahub-operator-controller-manager-7b6ffc8659-cqbln\" (UID: \"e5c96b43-1669-4a57-8a32-a8eb28e35d00\") " pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" Jul 01 22:25:43.594302 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.594258 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e5c96b43-1669-4a57-8a32-a8eb28e35d00-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b6ffc8659-cqbln\" (UID: \"e5c96b43-1669-4a57-8a32-a8eb28e35d00\") " pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" Jul 01 22:25:43.594302 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.594303 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkkx2\" (UniqueName: \"kubernetes.io/projected/e5c96b43-1669-4a57-8a32-a8eb28e35d00-kube-api-access-xkkx2\") pod \"opendatahub-operator-controller-manager-7b6ffc8659-cqbln\" (UID: \"e5c96b43-1669-4a57-8a32-a8eb28e35d00\") " pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" Jul 01 22:25:43.594495 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.594333 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5c96b43-1669-4a57-8a32-a8eb28e35d00-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b6ffc8659-cqbln\" (UID: \"e5c96b43-1669-4a57-8a32-a8eb28e35d00\") " pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" Jul 01 22:25:43.594495 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.594353 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5c96b43-1669-4a57-8a32-a8eb28e35d00-tmp\") pod \"opendatahub-operator-controller-manager-7b6ffc8659-cqbln\" (UID: \"e5c96b43-1669-4a57-8a32-a8eb28e35d00\") " pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" Jul 01 22:25:43.594495 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.594374 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e5c96b43-1669-4a57-8a32-a8eb28e35d00-manifests\") pod \"opendatahub-operator-controller-manager-7b6ffc8659-cqbln\" (UID: \"e5c96b43-1669-4a57-8a32-a8eb28e35d00\") " pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" Jul 01 22:25:43.594836 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.594815 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e5c96b43-1669-4a57-8a32-a8eb28e35d00-manifests\") pod \"opendatahub-operator-controller-manager-7b6ffc8659-cqbln\" (UID: \"e5c96b43-1669-4a57-8a32-a8eb28e35d00\") " pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" Jul 01 22:25:43.596638 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.596606 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5c96b43-1669-4a57-8a32-a8eb28e35d00-tmp\") pod \"opendatahub-operator-controller-manager-7b6ffc8659-cqbln\" (UID: \"e5c96b43-1669-4a57-8a32-a8eb28e35d00\") " pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" Jul 01 22:25:43.596824 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.596800 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e5c96b43-1669-4a57-8a32-a8eb28e35d00-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b6ffc8659-cqbln\" (UID: \"e5c96b43-1669-4a57-8a32-a8eb28e35d00\") " pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" Jul 01 22:25:43.596988 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.596973 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5c96b43-1669-4a57-8a32-a8eb28e35d00-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b6ffc8659-cqbln\" (UID: \"e5c96b43-1669-4a57-8a32-a8eb28e35d00\") " pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" Jul 01 22:25:43.641654 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.641627 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkkx2\" (UniqueName: \"kubernetes.io/projected/e5c96b43-1669-4a57-8a32-a8eb28e35d00-kube-api-access-xkkx2\") pod \"opendatahub-operator-controller-manager-7b6ffc8659-cqbln\" (UID: \"e5c96b43-1669-4a57-8a32-a8eb28e35d00\") " pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" Jul 01 22:25:43.722597 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.722535 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" Jul 01 22:25:43.862723 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.862628 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln"] Jul 01 22:25:43.865016 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:25:43.864992 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5c96b43_1669_4a57_8a32_a8eb28e35d00.slice/crio-4298f136d77fc737f031e1d5c9ad2faf63ae5d54043667494cb5f5d2bf76574c WatchSource:0}: Error finding container 4298f136d77fc737f031e1d5c9ad2faf63ae5d54043667494cb5f5d2bf76574c: Status 404 returned error can't find the container with id 4298f136d77fc737f031e1d5c9ad2faf63ae5d54043667494cb5f5d2bf76574c Jul 01 22:25:43.920196 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:43.920171 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" event={"ID":"e5c96b43-1669-4a57-8a32-a8eb28e35d00","Type":"ContainerStarted","Data":"4298f136d77fc737f031e1d5c9ad2faf63ae5d54043667494cb5f5d2bf76574c"} Jul 01 22:25:46.935215 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:46.935174 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" event={"ID":"e5c96b43-1669-4a57-8a32-a8eb28e35d00","Type":"ContainerStarted","Data":"93afe31ba26cdb14e5dd8f8214ab341e3f2777338b12081b4800c32d22ee7fb3"} Jul 01 22:25:47.940122 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:47.940091 2580 generic.go:356] "Generic (PLEG): container finished" podID="e5c96b43-1669-4a57-8a32-a8eb28e35d00" containerID="93afe31ba26cdb14e5dd8f8214ab341e3f2777338b12081b4800c32d22ee7fb3" exitCode=0 Jul 01 22:25:47.940440 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:47.940173 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" event={"ID":"e5c96b43-1669-4a57-8a32-a8eb28e35d00","Type":"ContainerDied","Data":"93afe31ba26cdb14e5dd8f8214ab341e3f2777338b12081b4800c32d22ee7fb3"} Jul 01 22:25:48.946032 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:48.945998 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" event={"ID":"e5c96b43-1669-4a57-8a32-a8eb28e35d00","Type":"ContainerStarted","Data":"083b1a47cdf6489630e4e9eb355aa1e33d27f0352a5b5b03877481e20d0274e8"} Jul 01 22:25:48.946434 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:48.946142 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" Jul 01 22:25:48.972859 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:48.972798 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" podStartSLOduration=1.665701267 podStartE2EDuration="5.972776107s" totalImagesPullingTime="3.229315218s" totalInitContainerRuntime="1.077759622s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:25:43 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:25:48.970464769 +0000 UTC m=+289.717905440" watchObservedRunningTime="2026-07-01 22:25:48.972776107 +0000 UTC m=+289.720216699" Jul 01 22:25:59.684826 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:59.684791 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8xh6_18708697-4b5c-4f52-9878-e369f8aa9eb2/console-operator/2.log" Jul 01 22:25:59.685599 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:59.685577 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8xh6_18708697-4b5c-4f52-9878-e369f8aa9eb2/console-operator/2.log" Jul 01 22:25:59.699781 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:59.699759 2580 kubelet.go:1648] "Image garbage collection succeeded" Jul 01 22:25:59.951877 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:25:59.951734 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7b6ffc8659-cqbln" Jul 01 22:26:02.465044 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:02.465013 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-658788d8fc-rzxrn"] Jul 01 22:26:02.468582 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:02.468543 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-658788d8fc-rzxrn" Jul 01 22:26:02.475094 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:02.475069 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-p4ppf\"" Jul 01 22:26:02.475237 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:02.475208 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 01 22:26:02.475385 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:02.475242 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 01 22:26:02.475385 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:02.475305 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 01 22:26:02.475385 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:02.475354 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 01 22:26:02.476154 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:02.476139 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 01 22:26:02.499101 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:02.499076 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-658788d8fc-rzxrn"] Jul 01 22:26:02.634463 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:02.634436 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7hnj\" (UniqueName: \"kubernetes.io/projected/8283436a-43f3-4e1b-bc39-814874332817-kube-api-access-z7hnj\") pod \"lws-controller-manager-658788d8fc-rzxrn\" (UID: \"8283436a-43f3-4e1b-bc39-814874332817\") " pod="openshift-lws-operator/lws-controller-manager-658788d8fc-rzxrn" Jul 01 22:26:02.634623 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:02.634482 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8283436a-43f3-4e1b-bc39-814874332817-cert\") pod \"lws-controller-manager-658788d8fc-rzxrn\" (UID: \"8283436a-43f3-4e1b-bc39-814874332817\") " pod="openshift-lws-operator/lws-controller-manager-658788d8fc-rzxrn" Jul 01 22:26:02.634623 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:02.634512 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8283436a-43f3-4e1b-bc39-814874332817-manager-config\") pod \"lws-controller-manager-658788d8fc-rzxrn\" (UID: \"8283436a-43f3-4e1b-bc39-814874332817\") " pod="openshift-lws-operator/lws-controller-manager-658788d8fc-rzxrn" Jul 01 22:26:02.634623 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:02.634599 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8283436a-43f3-4e1b-bc39-814874332817-metrics-cert\") pod \"lws-controller-manager-658788d8fc-rzxrn\" (UID: \"8283436a-43f3-4e1b-bc39-814874332817\") " pod="openshift-lws-operator/lws-controller-manager-658788d8fc-rzxrn" Jul 01 22:26:02.735658 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:02.735582 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7hnj\" (UniqueName: \"kubernetes.io/projected/8283436a-43f3-4e1b-bc39-814874332817-kube-api-access-z7hnj\") pod \"lws-controller-manager-658788d8fc-rzxrn\" (UID: \"8283436a-43f3-4e1b-bc39-814874332817\") " pod="openshift-lws-operator/lws-controller-manager-658788d8fc-rzxrn" Jul 01 22:26:02.735658 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:02.735628 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8283436a-43f3-4e1b-bc39-814874332817-cert\") pod \"lws-controller-manager-658788d8fc-rzxrn\" (UID: \"8283436a-43f3-4e1b-bc39-814874332817\") " pod="openshift-lws-operator/lws-controller-manager-658788d8fc-rzxrn" Jul 01 22:26:02.735658 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:02.735653 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8283436a-43f3-4e1b-bc39-814874332817-manager-config\") pod \"lws-controller-manager-658788d8fc-rzxrn\" (UID: \"8283436a-43f3-4e1b-bc39-814874332817\") " pod="openshift-lws-operator/lws-controller-manager-658788d8fc-rzxrn" Jul 01 22:26:02.735861 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:02.735675 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8283436a-43f3-4e1b-bc39-814874332817-metrics-cert\") pod \"lws-controller-manager-658788d8fc-rzxrn\" (UID: \"8283436a-43f3-4e1b-bc39-814874332817\") " pod="openshift-lws-operator/lws-controller-manager-658788d8fc-rzxrn" Jul 01 22:26:02.736371 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:02.736353 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8283436a-43f3-4e1b-bc39-814874332817-manager-config\") pod \"lws-controller-manager-658788d8fc-rzxrn\" (UID: \"8283436a-43f3-4e1b-bc39-814874332817\") " pod="openshift-lws-operator/lws-controller-manager-658788d8fc-rzxrn" Jul 01 22:26:02.737954 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:02.737935 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8283436a-43f3-4e1b-bc39-814874332817-cert\") pod \"lws-controller-manager-658788d8fc-rzxrn\" (UID: \"8283436a-43f3-4e1b-bc39-814874332817\") " pod="openshift-lws-operator/lws-controller-manager-658788d8fc-rzxrn" Jul 01 22:26:02.738087 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:02.738070 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8283436a-43f3-4e1b-bc39-814874332817-metrics-cert\") pod \"lws-controller-manager-658788d8fc-rzxrn\" (UID: \"8283436a-43f3-4e1b-bc39-814874332817\") " pod="openshift-lws-operator/lws-controller-manager-658788d8fc-rzxrn" Jul 01 22:26:02.780238 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:02.780210 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7hnj\" (UniqueName: \"kubernetes.io/projected/8283436a-43f3-4e1b-bc39-814874332817-kube-api-access-z7hnj\") pod \"lws-controller-manager-658788d8fc-rzxrn\" (UID: \"8283436a-43f3-4e1b-bc39-814874332817\") " pod="openshift-lws-operator/lws-controller-manager-658788d8fc-rzxrn" Jul 01 22:26:03.077808 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:03.077730 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-658788d8fc-rzxrn" Jul 01 22:26:03.227465 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:03.227437 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-658788d8fc-rzxrn"] Jul 01 22:26:03.228355 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:26:03.228325 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8283436a_43f3_4e1b_bc39_814874332817.slice/crio-4159c281b01b293725d6391fa21e4e6f9d181bc04764e3084a545f80322e6cae WatchSource:0}: Error finding container 4159c281b01b293725d6391fa21e4e6f9d181bc04764e3084a545f80322e6cae: Status 404 returned error can't find the container with id 4159c281b01b293725d6391fa21e4e6f9d181bc04764e3084a545f80322e6cae Jul 01 22:26:03.230447 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:03.230429 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 22:26:04.011394 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:04.011337 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-658788d8fc-rzxrn" event={"ID":"8283436a-43f3-4e1b-bc39-814874332817","Type":"ContainerStarted","Data":"4159c281b01b293725d6391fa21e4e6f9d181bc04764e3084a545f80322e6cae"} Jul 01 22:26:06.020391 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:06.020355 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-658788d8fc-rzxrn" event={"ID":"8283436a-43f3-4e1b-bc39-814874332817","Type":"ContainerStarted","Data":"d0fa0e6bca855c20d322896546c431c7d7914fad4cd2d467b2a09751fb784c7c"} Jul 01 22:26:06.020871 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:06.020425 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-658788d8fc-rzxrn" Jul 01 22:26:06.038094 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:06.038035 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-658788d8fc-rzxrn" podStartSLOduration=1.714468353 podStartE2EDuration="4.038016869s" totalImagesPullingTime="2.323548516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:26:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:26:06.037854472 +0000 UTC m=+306.785295075" watchObservedRunningTime="2026-07-01 22:26:06.038016869 +0000 UTC m=+306.785457461" Jul 01 22:26:10.827916 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.827887 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb"] Jul 01 22:26:10.832221 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.832201 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.835870 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.835851 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-glqv7\"" Jul 01 22:26:10.836632 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.836615 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 01 22:26:10.862262 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.862240 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb"] Jul 01 22:26:10.895422 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.895392 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.895594 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.895447 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/736f7188-2e2f-49cd-b669-badf6005e419-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.895594 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.895538 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.895724 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.895592 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.895724 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.895628 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.895724 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.895670 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/736f7188-2e2f-49cd-b669-badf6005e419-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.895724 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.895694 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.895916 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.895741 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/736f7188-2e2f-49cd-b669-badf6005e419-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.895916 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.895774 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rttks\" (UniqueName: \"kubernetes.io/projected/736f7188-2e2f-49cd-b669-badf6005e419-kube-api-access-rttks\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.977766 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.977735 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9"] Jul 01 22:26:10.980994 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.980978 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:10.996615 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.996594 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.996738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.996623 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.996738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.996645 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:10.996738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.996662 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:10.996738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.996682 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/736f7188-2e2f-49cd-b669-badf6005e419-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.996738 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.996698 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:10.996965 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.996731 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:10.996965 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.996791 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.996965 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.996818 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:10.996965 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.996842 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:10.996965 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.996871 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/736f7188-2e2f-49cd-b669-badf6005e419-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.996965 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.996907 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rttks\" (UniqueName: \"kubernetes.io/projected/736f7188-2e2f-49cd-b669-badf6005e419-kube-api-access-rttks\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.996965 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.996939 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:10.996965 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.996941 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.997311 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.996963 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:10.997311 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.997045 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.997311 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.997097 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/736f7188-2e2f-49cd-b669-badf6005e419-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.997311 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.997122 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.997311 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.997136 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrmck\" (UniqueName: \"kubernetes.io/projected/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-kube-api-access-xrmck\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:10.997311 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.997182 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.997492 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.997372 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.997492 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.997434 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/736f7188-2e2f-49cd-b669-badf6005e419-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.997572 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.997492 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.998807 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.998787 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:10.999448 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:10.999432 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/736f7188-2e2f-49cd-b669-badf6005e419-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:11.003662 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.003642 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9"] Jul 01 22:26:11.015318 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.015298 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rttks\" (UniqueName: \"kubernetes.io/projected/736f7188-2e2f-49cd-b669-badf6005e419-kube-api-access-rttks\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:11.016325 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.016299 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/736f7188-2e2f-49cd-b669-badf6005e419-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:11.097935 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.097904 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrmck\" (UniqueName: \"kubernetes.io/projected/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-kube-api-access-xrmck\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:11.098091 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.097948 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:11.098091 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.097967 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:11.098091 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.097988 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:11.098091 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.098010 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:11.098091 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.098044 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:11.098091 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.098069 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:11.098370 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.098104 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:11.098370 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.098128 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:11.098495 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.098473 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:11.098576 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.098511 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:11.098640 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.098617 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:11.098707 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.098682 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:11.098841 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.098823 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:11.100430 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.100401 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:11.100523 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.100475 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:11.109069 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.109043 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrmck\" (UniqueName: \"kubernetes.io/projected/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-kube-api-access-xrmck\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:11.109871 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.109853 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4b18eaaa-c9ea-429d-b752-ec0f9284a4c0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4qz8f9\" (UID: \"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:11.142005 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.141982 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:11.290439 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.290416 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb"] Jul 01 22:26:11.291801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.291774 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:11.292903 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:26:11.292876 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod736f7188_2e2f_49cd_b669_badf6005e419.slice/crio-533bfee3ca8dc831da54d73d641768375b4dbe7341c541d27a68d95732f5401d WatchSource:0}: Error finding container 533bfee3ca8dc831da54d73d641768375b4dbe7341c541d27a68d95732f5401d: Status 404 returned error can't find the container with id 533bfee3ca8dc831da54d73d641768375b4dbe7341c541d27a68d95732f5401d Jul 01 22:26:11.442429 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:11.442394 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9"] Jul 01 22:26:11.445039 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:26:11.445008 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b18eaaa_c9ea_429d_b752_ec0f9284a4c0.slice/crio-f4c98ce7f31e614f68c493df1d7adb1ddebff78ff23d70b68872ba9710fa056e WatchSource:0}: Error finding container f4c98ce7f31e614f68c493df1d7adb1ddebff78ff23d70b68872ba9710fa056e: Status 404 returned error can't find the container with id f4c98ce7f31e614f68c493df1d7adb1ddebff78ff23d70b68872ba9710fa056e Jul 01 22:26:12.045829 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:12.045791 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" event={"ID":"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0","Type":"ContainerStarted","Data":"f4c98ce7f31e614f68c493df1d7adb1ddebff78ff23d70b68872ba9710fa056e"} Jul 01 22:26:12.046902 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:12.046876 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" event={"ID":"736f7188-2e2f-49cd-b669-badf6005e419","Type":"ContainerStarted","Data":"533bfee3ca8dc831da54d73d641768375b4dbe7341c541d27a68d95732f5401d"} Jul 01 22:26:14.756791 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:14.756759 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-k7spv"] Jul 01 22:26:14.765352 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:14.765325 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k7spv" Jul 01 22:26:14.768146 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:14.768105 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 01 22:26:14.769205 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:14.769187 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-qbhbr\"" Jul 01 22:26:14.777820 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:14.777795 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-k7spv"] Jul 01 22:26:14.838152 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:14.838117 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcdgq\" (UniqueName: \"kubernetes.io/projected/2b92f5a0-0a01-4676-8adf-ad047c2f6ca4-kube-api-access-pcdgq\") pod \"llmisvc-controller-manager-777c6bd7d5-k7spv\" (UID: \"2b92f5a0-0a01-4676-8adf-ad047c2f6ca4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k7spv" Jul 01 22:26:14.838298 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:14.838217 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2b92f5a0-0a01-4676-8adf-ad047c2f6ca4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-k7spv\" (UID: \"2b92f5a0-0a01-4676-8adf-ad047c2f6ca4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k7spv" Jul 01 22:26:14.849265 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:14.849225 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 01 22:26:14.849355 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:14.849290 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 01 22:26:14.849355 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:14.849319 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 01 22:26:14.855242 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:14.855208 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 01 22:26:14.855307 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:14.855290 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 01 22:26:14.855353 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:14.855340 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 01 22:26:14.938960 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:14.938928 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2b92f5a0-0a01-4676-8adf-ad047c2f6ca4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-k7spv\" (UID: \"2b92f5a0-0a01-4676-8adf-ad047c2f6ca4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k7spv" Jul 01 22:26:14.939109 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:14.938989 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcdgq\" (UniqueName: \"kubernetes.io/projected/2b92f5a0-0a01-4676-8adf-ad047c2f6ca4-kube-api-access-pcdgq\") pod \"llmisvc-controller-manager-777c6bd7d5-k7spv\" (UID: \"2b92f5a0-0a01-4676-8adf-ad047c2f6ca4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k7spv" Jul 01 22:26:14.941708 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:14.941684 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2b92f5a0-0a01-4676-8adf-ad047c2f6ca4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-k7spv\" (UID: \"2b92f5a0-0a01-4676-8adf-ad047c2f6ca4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k7spv" Jul 01 22:26:14.957295 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:14.957234 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcdgq\" (UniqueName: \"kubernetes.io/projected/2b92f5a0-0a01-4676-8adf-ad047c2f6ca4-kube-api-access-pcdgq\") pod \"llmisvc-controller-manager-777c6bd7d5-k7spv\" (UID: \"2b92f5a0-0a01-4676-8adf-ad047c2f6ca4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k7spv" Jul 01 22:26:15.063407 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:15.063305 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" event={"ID":"736f7188-2e2f-49cd-b669-badf6005e419","Type":"ContainerStarted","Data":"767e35f775cb22e2c57d5d6d3ed1930790311daf2be229efdf508b7f0267943b"} Jul 01 22:26:15.065012 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:15.064984 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" event={"ID":"4b18eaaa-c9ea-429d-b752-ec0f9284a4c0","Type":"ContainerStarted","Data":"4834a71648227a99dc8feb6122ef4ad751af9929da0315b718b2a25b6351d11c"} Jul 01 22:26:15.076854 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:15.076835 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k7spv" Jul 01 22:26:15.111448 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:15.111395 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" podStartSLOduration=1.557162395 podStartE2EDuration="5.111378777s" totalImagesPullingTime="3.554216382s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:26:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:26:15.10925747 +0000 UTC m=+315.856698072" watchObservedRunningTime="2026-07-01 22:26:15.111378777 +0000 UTC m=+315.858819367" Jul 01 22:26:15.143319 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:15.142244 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:15.143612 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:15.143508 2580 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.25:15021/healthz/ready\": dial tcp 10.132.0.25:15021: connect: connection refused" start-of-body= Jul 01 22:26:15.144527 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:15.144357 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" podUID="736f7188-2e2f-49cd-b669-badf6005e419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.25:15021/healthz/ready\": dial tcp 10.132.0.25:15021: connect: connection refused" Jul 01 22:26:15.147213 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:15.145817 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" podStartSLOduration=1.737899925 podStartE2EDuration="5.145796182s" totalImagesPullingTime="3.407896257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:26:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:26:15.144006625 +0000 UTC m=+315.891447215" watchObservedRunningTime="2026-07-01 22:26:15.145796182 +0000 UTC m=+315.893236773" Jul 01 22:26:15.229744 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:15.229690 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-k7spv"] Jul 01 22:26:15.234165 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:26:15.234123 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2b92f5a0_0a01_4676_8adf_ad047c2f6ca4.slice/crio-4b29459d9460648aeb6da8a3fa22a8c864f7d60cd5a30509fde785a19ce5f93b WatchSource:0}: Error finding container 4b29459d9460648aeb6da8a3fa22a8c864f7d60cd5a30509fde785a19ce5f93b: Status 404 returned error can't find the container with id 4b29459d9460648aeb6da8a3fa22a8c864f7d60cd5a30509fde785a19ce5f93b Jul 01 22:26:15.292685 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:15.292653 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:15.294044 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:15.294018 2580 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" start-of-body= Jul 01 22:26:15.294162 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:15.294064 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" podUID="4b18eaaa-c9ea-429d-b752-ec0f9284a4c0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.26:15021/healthz/ready\": dial tcp 10.132.0.26:15021: connect: connection refused" Jul 01 22:26:16.072315 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:16.072286 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k7spv" event={"ID":"2b92f5a0-0a01-4676-8adf-ad047c2f6ca4","Type":"ContainerStarted","Data":"4b29459d9460648aeb6da8a3fa22a8c864f7d60cd5a30509fde785a19ce5f93b"} Jul 01 22:26:16.142612 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:16.142583 2580 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.25:15021/healthz/ready\": dial tcp 10.132.0.25:15021: connect: connection refused" start-of-body= Jul 01 22:26:16.142765 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:16.142632 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" podUID="736f7188-2e2f-49cd-b669-badf6005e419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.25:15021/healthz/ready\": dial tcp 10.132.0.25:15021: connect: connection refused" Jul 01 22:26:16.296523 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:16.296493 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:17.026858 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:17.026830 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-658788d8fc-rzxrn" Jul 01 22:26:17.076760 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:17.076730 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:17.077589 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:17.077545 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4qz8f9" Jul 01 22:26:17.142402 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:17.142373 2580 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.25:15021/healthz/ready\": dial tcp 10.132.0.25:15021: connect: connection refused" start-of-body= Jul 01 22:26:17.142532 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:17.142435 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" podUID="736f7188-2e2f-49cd-b669-badf6005e419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.25:15021/healthz/ready\": dial tcp 10.132.0.25:15021: connect: connection refused" Jul 01 22:26:17.201192 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:17.201158 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb"] Jul 01 22:26:18.082599 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:18.082420 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" podUID="736f7188-2e2f-49cd-b669-badf6005e419" containerName="istio-proxy" containerID="cri-o://767e35f775cb22e2c57d5d6d3ed1930790311daf2be229efdf508b7f0267943b" gracePeriod=30 Jul 01 22:26:19.087844 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:19.087805 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k7spv" event={"ID":"2b92f5a0-0a01-4676-8adf-ad047c2f6ca4","Type":"ContainerStarted","Data":"10b40810b9ea87445b2a7a7c52758fec3c3e6f4ad74507b14b330c3c325b403d"} Jul 01 22:26:19.088181 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:19.087880 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k7spv" Jul 01 22:26:23.350330 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.350308 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:23.373544 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.373489 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k7spv" podStartSLOduration=6.381789322 podStartE2EDuration="9.373472513s" totalImagesPullingTime="2.991683191s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:26:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:26:19.117748011 +0000 UTC m=+319.865188600" watchObservedRunningTime="2026-07-01 22:26:23.373472513 +0000 UTC m=+324.120913102" Jul 01 22:26:23.412351 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.412323 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-workload-certs\") pod \"736f7188-2e2f-49cd-b669-badf6005e419\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " Jul 01 22:26:23.412517 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.412358 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-credential-socket\") pod \"736f7188-2e2f-49cd-b669-badf6005e419\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " Jul 01 22:26:23.412517 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.412396 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/736f7188-2e2f-49cd-b669-badf6005e419-istio-podinfo\") pod \"736f7188-2e2f-49cd-b669-badf6005e419\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " Jul 01 22:26:23.412517 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.412422 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-workload-socket\") pod \"736f7188-2e2f-49cd-b669-badf6005e419\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " Jul 01 22:26:23.412517 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.412444 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/736f7188-2e2f-49cd-b669-badf6005e419-istiod-ca-cert\") pod \"736f7188-2e2f-49cd-b669-badf6005e419\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " Jul 01 22:26:23.412517 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.412467 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/736f7188-2e2f-49cd-b669-badf6005e419-istio-token\") pod \"736f7188-2e2f-49cd-b669-badf6005e419\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " Jul 01 22:26:23.412517 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.412499 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rttks\" (UniqueName: \"kubernetes.io/projected/736f7188-2e2f-49cd-b669-badf6005e419-kube-api-access-rttks\") pod \"736f7188-2e2f-49cd-b669-badf6005e419\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " Jul 01 22:26:23.412517 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.412517 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-istio-data\") pod \"736f7188-2e2f-49cd-b669-badf6005e419\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " Jul 01 22:26:23.412880 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.412573 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-istio-envoy\") pod \"736f7188-2e2f-49cd-b669-badf6005e419\" (UID: \"736f7188-2e2f-49cd-b669-badf6005e419\") " Jul 01 22:26:23.412880 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.412639 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "736f7188-2e2f-49cd-b669-badf6005e419" (UID: "736f7188-2e2f-49cd-b669-badf6005e419"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 22:26:23.412880 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.412693 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "736f7188-2e2f-49cd-b669-badf6005e419" (UID: "736f7188-2e2f-49cd-b669-badf6005e419"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 22:26:23.412880 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.412814 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "736f7188-2e2f-49cd-b669-badf6005e419" (UID: "736f7188-2e2f-49cd-b669-badf6005e419"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 22:26:23.413070 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.412880 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/736f7188-2e2f-49cd-b669-badf6005e419-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "736f7188-2e2f-49cd-b669-badf6005e419" (UID: "736f7188-2e2f-49cd-b669-badf6005e419"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 22:26:23.413070 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.412894 2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-workload-certs\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:26:23.413070 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.412912 2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-credential-socket\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:26:23.413070 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.412926 2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-workload-socket\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:26:23.413268 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.413136 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-istio-data" (OuterVolumeSpecName: "istio-data") pod "736f7188-2e2f-49cd-b669-badf6005e419" (UID: "736f7188-2e2f-49cd-b669-badf6005e419"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 22:26:23.414933 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.414903 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/736f7188-2e2f-49cd-b669-badf6005e419-istio-token" (OuterVolumeSpecName: "istio-token") pod "736f7188-2e2f-49cd-b669-badf6005e419" (UID: "736f7188-2e2f-49cd-b669-badf6005e419"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 22:26:23.414933 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.414916 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/736f7188-2e2f-49cd-b669-badf6005e419-kube-api-access-rttks" (OuterVolumeSpecName: "kube-api-access-rttks") pod "736f7188-2e2f-49cd-b669-badf6005e419" (UID: "736f7188-2e2f-49cd-b669-badf6005e419"). InnerVolumeSpecName "kube-api-access-rttks". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 22:26:23.415102 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.415075 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "736f7188-2e2f-49cd-b669-badf6005e419" (UID: "736f7188-2e2f-49cd-b669-badf6005e419"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 22:26:23.415102 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.415085 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/736f7188-2e2f-49cd-b669-badf6005e419-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "736f7188-2e2f-49cd-b669-badf6005e419" (UID: "736f7188-2e2f-49cd-b669-badf6005e419"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 01 22:26:23.514312 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.514269 2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-istio-envoy\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:26:23.514312 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.514305 2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/736f7188-2e2f-49cd-b669-badf6005e419-istio-podinfo\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:26:23.514312 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.514316 2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/736f7188-2e2f-49cd-b669-badf6005e419-istiod-ca-cert\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:26:23.514530 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.514328 2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/736f7188-2e2f-49cd-b669-badf6005e419-istio-token\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:26:23.514530 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.514339 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rttks\" (UniqueName: \"kubernetes.io/projected/736f7188-2e2f-49cd-b669-badf6005e419-kube-api-access-rttks\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:26:23.514530 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:23.514350 2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/736f7188-2e2f-49cd-b669-badf6005e419-istio-data\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:26:24.109817 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:24.109778 2580 generic.go:356] "Generic (PLEG): container finished" podID="736f7188-2e2f-49cd-b669-badf6005e419" containerID="767e35f775cb22e2c57d5d6d3ed1930790311daf2be229efdf508b7f0267943b" exitCode=0 Jul 01 22:26:24.109972 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:24.109832 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" Jul 01 22:26:24.109972 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:24.109864 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" event={"ID":"736f7188-2e2f-49cd-b669-badf6005e419","Type":"ContainerDied","Data":"767e35f775cb22e2c57d5d6d3ed1930790311daf2be229efdf508b7f0267943b"} Jul 01 22:26:24.109972 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:24.109917 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb" event={"ID":"736f7188-2e2f-49cd-b669-badf6005e419","Type":"ContainerDied","Data":"533bfee3ca8dc831da54d73d641768375b4dbe7341c541d27a68d95732f5401d"} Jul 01 22:26:24.109972 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:24.109939 2580 scope.go:117] "RemoveContainer" containerID="767e35f775cb22e2c57d5d6d3ed1930790311daf2be229efdf508b7f0267943b" Jul 01 22:26:24.118098 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:24.118082 2580 scope.go:117] "RemoveContainer" containerID="767e35f775cb22e2c57d5d6d3ed1930790311daf2be229efdf508b7f0267943b" Jul 01 22:26:24.118335 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:26:24.118318 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"767e35f775cb22e2c57d5d6d3ed1930790311daf2be229efdf508b7f0267943b\": container with ID starting with 767e35f775cb22e2c57d5d6d3ed1930790311daf2be229efdf508b7f0267943b not found: ID does not exist" containerID="767e35f775cb22e2c57d5d6d3ed1930790311daf2be229efdf508b7f0267943b" Jul 01 22:26:24.118384 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:24.118342 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"767e35f775cb22e2c57d5d6d3ed1930790311daf2be229efdf508b7f0267943b"} err="failed to get container status \"767e35f775cb22e2c57d5d6d3ed1930790311daf2be229efdf508b7f0267943b\": rpc error: code = NotFound desc = could not find container \"767e35f775cb22e2c57d5d6d3ed1930790311daf2be229efdf508b7f0267943b\": container with ID starting with 767e35f775cb22e2c57d5d6d3ed1930790311daf2be229efdf508b7f0267943b not found: ID does not exist" Jul 01 22:26:24.129328 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:24.129307 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb"] Jul 01 22:26:24.136118 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:24.136097 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-xqlcb"] Jul 01 22:26:25.747543 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:25.747508 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="736f7188-2e2f-49cd-b669-badf6005e419" path="/var/lib/kubelet/pods/736f7188-2e2f-49cd-b669-badf6005e419/volumes" Jul 01 22:26:50.094087 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:26:50.094055 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-k7spv" Jul 01 22:27:04.321884 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:04.321845 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4q7sx"] Jul 01 22:27:04.322352 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:04.322168 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="736f7188-2e2f-49cd-b669-badf6005e419" containerName="istio-proxy" Jul 01 22:27:04.322352 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:04.322180 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="736f7188-2e2f-49cd-b669-badf6005e419" containerName="istio-proxy" Jul 01 22:27:04.322352 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:04.322249 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="736f7188-2e2f-49cd-b669-badf6005e419" containerName="istio-proxy" Jul 01 22:27:04.326155 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:04.326132 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4q7sx" Jul 01 22:27:04.328854 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:04.328831 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 01 22:27:04.330015 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:04.329991 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-wgmx5\"" Jul 01 22:27:04.330135 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:04.330003 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 01 22:27:04.335217 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:04.335194 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4q7sx"] Jul 01 22:27:04.403798 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:04.403774 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cj47\" (UniqueName: \"kubernetes.io/projected/b8a67d49-cb90-4214-81ae-869402e3ba3c-kube-api-access-9cj47\") pod \"kuadrant-operator-catalog-4q7sx\" (UID: \"b8a67d49-cb90-4214-81ae-869402e3ba3c\") " pod="kuadrant-system/kuadrant-operator-catalog-4q7sx" Jul 01 22:27:04.504745 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:04.504720 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cj47\" (UniqueName: \"kubernetes.io/projected/b8a67d49-cb90-4214-81ae-869402e3ba3c-kube-api-access-9cj47\") pod \"kuadrant-operator-catalog-4q7sx\" (UID: \"b8a67d49-cb90-4214-81ae-869402e3ba3c\") " pod="kuadrant-system/kuadrant-operator-catalog-4q7sx" Jul 01 22:27:04.516079 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:04.516052 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cj47\" (UniqueName: \"kubernetes.io/projected/b8a67d49-cb90-4214-81ae-869402e3ba3c-kube-api-access-9cj47\") pod \"kuadrant-operator-catalog-4q7sx\" (UID: \"b8a67d49-cb90-4214-81ae-869402e3ba3c\") " pod="kuadrant-system/kuadrant-operator-catalog-4q7sx" Jul 01 22:27:04.637473 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:04.637447 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4q7sx" Jul 01 22:27:04.668612 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:04.668588 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4q7sx"] Jul 01 22:27:04.775633 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:04.775604 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4q7sx"] Jul 01 22:27:04.778528 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:27:04.778505 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8a67d49_cb90_4214_81ae_869402e3ba3c.slice/crio-b2ddd3f61d8fc191d49faf6884119a3f25b114753bd3d7e3a53b7db850eaae92 WatchSource:0}: Error finding container b2ddd3f61d8fc191d49faf6884119a3f25b114753bd3d7e3a53b7db850eaae92: Status 404 returned error can't find the container with id b2ddd3f61d8fc191d49faf6884119a3f25b114753bd3d7e3a53b7db850eaae92 Jul 01 22:27:04.880728 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:04.880701 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-86kxj"] Jul 01 22:27:04.885056 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:04.885041 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-86kxj" Jul 01 22:27:04.893334 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:04.893311 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-86kxj"] Jul 01 22:27:05.008156 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:05.008132 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mjg7\" (UniqueName: \"kubernetes.io/projected/c92f9cce-4542-45af-8c59-850e8df59df6-kube-api-access-6mjg7\") pod \"kuadrant-operator-catalog-86kxj\" (UID: \"c92f9cce-4542-45af-8c59-850e8df59df6\") " pod="kuadrant-system/kuadrant-operator-catalog-86kxj" Jul 01 22:27:05.109024 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:05.108997 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mjg7\" (UniqueName: \"kubernetes.io/projected/c92f9cce-4542-45af-8c59-850e8df59df6-kube-api-access-6mjg7\") pod \"kuadrant-operator-catalog-86kxj\" (UID: \"c92f9cce-4542-45af-8c59-850e8df59df6\") " pod="kuadrant-system/kuadrant-operator-catalog-86kxj" Jul 01 22:27:05.119926 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:05.119906 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mjg7\" (UniqueName: \"kubernetes.io/projected/c92f9cce-4542-45af-8c59-850e8df59df6-kube-api-access-6mjg7\") pod \"kuadrant-operator-catalog-86kxj\" (UID: \"c92f9cce-4542-45af-8c59-850e8df59df6\") " pod="kuadrant-system/kuadrant-operator-catalog-86kxj" Jul 01 22:27:05.194639 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:05.194541 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-86kxj" Jul 01 22:27:05.289783 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:05.289751 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4q7sx" event={"ID":"b8a67d49-cb90-4214-81ae-869402e3ba3c","Type":"ContainerStarted","Data":"b2ddd3f61d8fc191d49faf6884119a3f25b114753bd3d7e3a53b7db850eaae92"} Jul 01 22:27:05.339465 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:05.339440 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-86kxj"] Jul 01 22:27:05.344942 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:27:05.344913 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc92f9cce_4542_45af_8c59_850e8df59df6.slice/crio-3cae83c418f590a9f8cd9be93ae0ede6110424d759311497d96601ecca187673 WatchSource:0}: Error finding container 3cae83c418f590a9f8cd9be93ae0ede6110424d759311497d96601ecca187673: Status 404 returned error can't find the container with id 3cae83c418f590a9f8cd9be93ae0ede6110424d759311497d96601ecca187673 Jul 01 22:27:06.297354 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:06.297308 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-86kxj" event={"ID":"c92f9cce-4542-45af-8c59-850e8df59df6","Type":"ContainerStarted","Data":"3cae83c418f590a9f8cd9be93ae0ede6110424d759311497d96601ecca187673"} Jul 01 22:27:07.302765 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:07.302672 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4q7sx" event={"ID":"b8a67d49-cb90-4214-81ae-869402e3ba3c","Type":"ContainerStarted","Data":"1b778298c61ca3e4a59de4a2e7e1eabd759e5d25464098335de8bf2e92f80476"} Jul 01 22:27:07.303178 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:07.302739 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-4q7sx" podUID="b8a67d49-cb90-4214-81ae-869402e3ba3c" containerName="registry-server" containerID="cri-o://1b778298c61ca3e4a59de4a2e7e1eabd759e5d25464098335de8bf2e92f80476" gracePeriod=2 Jul 01 22:27:07.304268 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:07.304242 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-86kxj" event={"ID":"c92f9cce-4542-45af-8c59-850e8df59df6","Type":"ContainerStarted","Data":"2d4aac5495d95d6669c4f28f0146490d0b483a47dfc143209ed3e2e025b7227a"} Jul 01 22:27:07.321274 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:07.321235 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-4q7sx" podStartSLOduration=1.236588696 podStartE2EDuration="3.321222434s" totalImagesPullingTime="2.084633738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:27:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:27:07.319537578 +0000 UTC m=+368.066978169" watchObservedRunningTime="2026-07-01 22:27:07.321222434 +0000 UTC m=+368.068663023" Jul 01 22:27:07.546546 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:07.546521 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4q7sx" Jul 01 22:27:07.568224 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:07.568149 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-86kxj" podStartSLOduration=2.047421872 podStartE2EDuration="3.568129226s" totalImagesPullingTime="1.520707354s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:27:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:27:07.34100313 +0000 UTC m=+368.088443720" watchObservedRunningTime="2026-07-01 22:27:07.568129226 +0000 UTC m=+368.315569819" Jul 01 22:27:07.633840 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:07.633816 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9cj47\" (UniqueName: \"kubernetes.io/projected/b8a67d49-cb90-4214-81ae-869402e3ba3c-kube-api-access-9cj47\") pod \"b8a67d49-cb90-4214-81ae-869402e3ba3c\" (UID: \"b8a67d49-cb90-4214-81ae-869402e3ba3c\") " Jul 01 22:27:07.635878 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:07.635854 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8a67d49-cb90-4214-81ae-869402e3ba3c-kube-api-access-9cj47" (OuterVolumeSpecName: "kube-api-access-9cj47") pod "b8a67d49-cb90-4214-81ae-869402e3ba3c" (UID: "b8a67d49-cb90-4214-81ae-869402e3ba3c"). InnerVolumeSpecName "kube-api-access-9cj47". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 22:27:07.734527 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:07.734505 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9cj47\" (UniqueName: \"kubernetes.io/projected/b8a67d49-cb90-4214-81ae-869402e3ba3c-kube-api-access-9cj47\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:27:08.310388 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:08.310299 2580 generic.go:356] "Generic (PLEG): container finished" podID="b8a67d49-cb90-4214-81ae-869402e3ba3c" containerID="1b778298c61ca3e4a59de4a2e7e1eabd759e5d25464098335de8bf2e92f80476" exitCode=0 Jul 01 22:27:08.310388 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:08.310343 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4q7sx" Jul 01 22:27:08.310900 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:08.310396 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4q7sx" event={"ID":"b8a67d49-cb90-4214-81ae-869402e3ba3c","Type":"ContainerDied","Data":"1b778298c61ca3e4a59de4a2e7e1eabd759e5d25464098335de8bf2e92f80476"} Jul 01 22:27:08.310900 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:08.310446 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4q7sx" event={"ID":"b8a67d49-cb90-4214-81ae-869402e3ba3c","Type":"ContainerDied","Data":"b2ddd3f61d8fc191d49faf6884119a3f25b114753bd3d7e3a53b7db850eaae92"} Jul 01 22:27:08.310900 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:08.310475 2580 scope.go:117] "RemoveContainer" containerID="1b778298c61ca3e4a59de4a2e7e1eabd759e5d25464098335de8bf2e92f80476" Jul 01 22:27:08.320087 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:08.320066 2580 scope.go:117] "RemoveContainer" containerID="1b778298c61ca3e4a59de4a2e7e1eabd759e5d25464098335de8bf2e92f80476" Jul 01 22:27:08.320349 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:27:08.320328 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b778298c61ca3e4a59de4a2e7e1eabd759e5d25464098335de8bf2e92f80476\": container with ID starting with 1b778298c61ca3e4a59de4a2e7e1eabd759e5d25464098335de8bf2e92f80476 not found: ID does not exist" containerID="1b778298c61ca3e4a59de4a2e7e1eabd759e5d25464098335de8bf2e92f80476" Jul 01 22:27:08.320406 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:08.320369 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b778298c61ca3e4a59de4a2e7e1eabd759e5d25464098335de8bf2e92f80476"} err="failed to get container status \"1b778298c61ca3e4a59de4a2e7e1eabd759e5d25464098335de8bf2e92f80476\": rpc error: code = NotFound desc = could not find container \"1b778298c61ca3e4a59de4a2e7e1eabd759e5d25464098335de8bf2e92f80476\": container with ID starting with 1b778298c61ca3e4a59de4a2e7e1eabd759e5d25464098335de8bf2e92f80476 not found: ID does not exist" Jul 01 22:27:08.327592 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:08.327566 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4q7sx"] Jul 01 22:27:08.334484 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:08.334465 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4q7sx"] Jul 01 22:27:09.747434 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:09.747398 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8a67d49-cb90-4214-81ae-869402e3ba3c" path="/var/lib/kubelet/pods/b8a67d49-cb90-4214-81ae-869402e3ba3c/volumes" Jul 01 22:27:15.195627 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:15.195590 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-86kxj" Jul 01 22:27:15.195996 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:15.195897 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-86kxj" Jul 01 22:27:15.217690 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:15.217667 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-86kxj" Jul 01 22:27:15.362637 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:15.362615 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-86kxj" Jul 01 22:27:27.863712 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:27.863679 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rwcn2"] Jul 01 22:27:27.864075 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:27.864012 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8a67d49-cb90-4214-81ae-869402e3ba3c" containerName="registry-server" Jul 01 22:27:27.864075 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:27.864026 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8a67d49-cb90-4214-81ae-869402e3ba3c" containerName="registry-server" Jul 01 22:27:27.864156 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:27.864109 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8a67d49-cb90-4214-81ae-869402e3ba3c" containerName="registry-server" Jul 01 22:27:27.865797 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:27.865780 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rwcn2" Jul 01 22:27:27.870060 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:27.870038 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-m7xdf\"" Jul 01 22:27:27.877986 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:27.877967 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rwcn2"] Jul 01 22:27:27.878216 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:27.878191 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w865r\" (UniqueName: \"kubernetes.io/projected/be2fe675-dd8c-4f97-ad39-f1891ccb2cc8-kube-api-access-w865r\") pod \"limitador-operator-controller-manager-859d7b78bd-rwcn2\" (UID: \"be2fe675-dd8c-4f97-ad39-f1891ccb2cc8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rwcn2" Jul 01 22:27:27.978920 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:27.978890 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w865r\" (UniqueName: \"kubernetes.io/projected/be2fe675-dd8c-4f97-ad39-f1891ccb2cc8-kube-api-access-w865r\") pod \"limitador-operator-controller-manager-859d7b78bd-rwcn2\" (UID: \"be2fe675-dd8c-4f97-ad39-f1891ccb2cc8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rwcn2" Jul 01 22:27:27.989904 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:27.989876 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w865r\" (UniqueName: \"kubernetes.io/projected/be2fe675-dd8c-4f97-ad39-f1891ccb2cc8-kube-api-access-w865r\") pod \"limitador-operator-controller-manager-859d7b78bd-rwcn2\" (UID: \"be2fe675-dd8c-4f97-ad39-f1891ccb2cc8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rwcn2" Jul 01 22:27:28.175407 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:28.175328 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rwcn2" Jul 01 22:27:28.303504 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:28.303481 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rwcn2"] Jul 01 22:27:28.305283 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:27:28.305255 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe2fe675_dd8c_4f97_ad39_f1891ccb2cc8.slice/crio-ff5b81673f2f79c538fe52b8477039e0cc63d365da536c0fd9654bcf8fe1df4f WatchSource:0}: Error finding container ff5b81673f2f79c538fe52b8477039e0cc63d365da536c0fd9654bcf8fe1df4f: Status 404 returned error can't find the container with id ff5b81673f2f79c538fe52b8477039e0cc63d365da536c0fd9654bcf8fe1df4f Jul 01 22:27:28.398365 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:28.398332 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rwcn2" event={"ID":"be2fe675-dd8c-4f97-ad39-f1891ccb2cc8","Type":"ContainerStarted","Data":"ff5b81673f2f79c538fe52b8477039e0cc63d365da536c0fd9654bcf8fe1df4f"} Jul 01 22:27:30.408676 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:30.408642 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rwcn2" event={"ID":"be2fe675-dd8c-4f97-ad39-f1891ccb2cc8","Type":"ContainerStarted","Data":"952d0f75364360dcfa55f857aed80e10ded63dea01f95e0d32b710a8484838de"} Jul 01 22:27:30.409071 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:30.408805 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rwcn2" Jul 01 22:27:30.434035 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:30.433989 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rwcn2" podStartSLOduration=1.967750476 podStartE2EDuration="3.433974128s" totalImagesPullingTime="1.466223652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:27:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:27:30.433354361 +0000 UTC m=+391.180794964" watchObservedRunningTime="2026-07-01 22:27:30.433974128 +0000 UTC m=+391.181414718" Jul 01 22:27:35.258203 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:35.258167 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ft7mh"] Jul 01 22:27:35.261653 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:35.261635 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ft7mh" Jul 01 22:27:35.267301 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:35.267278 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 01 22:27:35.267408 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:35.267381 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-hs8l7\"" Jul 01 22:27:35.285132 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:35.285111 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ft7mh"] Jul 01 22:27:35.336708 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:35.336681 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkv29\" (UniqueName: \"kubernetes.io/projected/fbdf2bcc-979e-4fda-b2aa-ca6894cd0f51-kube-api-access-pkv29\") pod \"dns-operator-controller-manager-864dd78c4c-ft7mh\" (UID: \"fbdf2bcc-979e-4fda-b2aa-ca6894cd0f51\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ft7mh" Jul 01 22:27:35.437634 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:35.437608 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkv29\" (UniqueName: \"kubernetes.io/projected/fbdf2bcc-979e-4fda-b2aa-ca6894cd0f51-kube-api-access-pkv29\") pod \"dns-operator-controller-manager-864dd78c4c-ft7mh\" (UID: \"fbdf2bcc-979e-4fda-b2aa-ca6894cd0f51\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ft7mh" Jul 01 22:27:35.453499 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:35.453478 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkv29\" (UniqueName: \"kubernetes.io/projected/fbdf2bcc-979e-4fda-b2aa-ca6894cd0f51-kube-api-access-pkv29\") pod \"dns-operator-controller-manager-864dd78c4c-ft7mh\" (UID: \"fbdf2bcc-979e-4fda-b2aa-ca6894cd0f51\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ft7mh" Jul 01 22:27:35.571349 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:35.571283 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ft7mh" Jul 01 22:27:35.704416 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:35.704389 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-ft7mh"] Jul 01 22:27:35.708586 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:27:35.708542 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbdf2bcc_979e_4fda_b2aa_ca6894cd0f51.slice/crio-4a21d9927b531d1d5674109c307b853dd930220d95ca06b912a51eb85e8bcc25 WatchSource:0}: Error finding container 4a21d9927b531d1d5674109c307b853dd930220d95ca06b912a51eb85e8bcc25: Status 404 returned error can't find the container with id 4a21d9927b531d1d5674109c307b853dd930220d95ca06b912a51eb85e8bcc25 Jul 01 22:27:36.435489 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:36.435459 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ft7mh" event={"ID":"fbdf2bcc-979e-4fda-b2aa-ca6894cd0f51","Type":"ContainerStarted","Data":"4a21d9927b531d1d5674109c307b853dd930220d95ca06b912a51eb85e8bcc25"} Jul 01 22:27:38.446467 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:38.446433 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ft7mh" event={"ID":"fbdf2bcc-979e-4fda-b2aa-ca6894cd0f51","Type":"ContainerStarted","Data":"7dc4cbe154ddb9f20b015208d93684606bcf276450b1f8336ca5131189bff348"} Jul 01 22:27:38.446844 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:38.446588 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ft7mh" Jul 01 22:27:38.490030 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:38.489954 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ft7mh" podStartSLOduration=1.642501968 podStartE2EDuration="3.489942821s" totalImagesPullingTime="1.847440853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:27:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:27:38.488740242 +0000 UTC m=+399.236180833" watchObservedRunningTime="2026-07-01 22:27:38.489942821 +0000 UTC m=+399.237383410" Jul 01 22:27:40.007648 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:40.007619 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq"] Jul 01 22:27:40.010909 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:40.010894 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq" Jul 01 22:27:40.015116 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:40.015086 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-bbrvv\"" Jul 01 22:27:40.027922 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:40.027900 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq"] Jul 01 22:27:40.073770 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:40.073742 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddf4g\" (UniqueName: \"kubernetes.io/projected/4347a2a2-92d5-4115-ad2e-7571d8fb1e25-kube-api-access-ddf4g\") pod \"kuadrant-operator-controller-manager-8684c9ff49-77sjq\" (UID: \"4347a2a2-92d5-4115-ad2e-7571d8fb1e25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq" Jul 01 22:27:40.073918 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:40.073781 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4347a2a2-92d5-4115-ad2e-7571d8fb1e25-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-77sjq\" (UID: \"4347a2a2-92d5-4115-ad2e-7571d8fb1e25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq" Jul 01 22:27:40.174620 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:40.174587 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddf4g\" (UniqueName: \"kubernetes.io/projected/4347a2a2-92d5-4115-ad2e-7571d8fb1e25-kube-api-access-ddf4g\") pod \"kuadrant-operator-controller-manager-8684c9ff49-77sjq\" (UID: \"4347a2a2-92d5-4115-ad2e-7571d8fb1e25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq" Jul 01 22:27:40.174737 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:40.174627 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4347a2a2-92d5-4115-ad2e-7571d8fb1e25-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-77sjq\" (UID: \"4347a2a2-92d5-4115-ad2e-7571d8fb1e25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq" Jul 01 22:27:40.174926 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:40.174912 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4347a2a2-92d5-4115-ad2e-7571d8fb1e25-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-77sjq\" (UID: \"4347a2a2-92d5-4115-ad2e-7571d8fb1e25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq" Jul 01 22:27:40.184172 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:40.184146 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddf4g\" (UniqueName: \"kubernetes.io/projected/4347a2a2-92d5-4115-ad2e-7571d8fb1e25-kube-api-access-ddf4g\") pod \"kuadrant-operator-controller-manager-8684c9ff49-77sjq\" (UID: \"4347a2a2-92d5-4115-ad2e-7571d8fb1e25\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq" Jul 01 22:27:40.320802 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:40.320726 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq" Jul 01 22:27:40.456117 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:40.456087 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq"] Jul 01 22:27:40.458158 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:27:40.458132 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4347a2a2_92d5_4115_ad2e_7571d8fb1e25.slice/crio-bd26eb783b7e3c7e5fea613a77812e414756e8a1b9a9cb7e15728499de0ec815 WatchSource:0}: Error finding container bd26eb783b7e3c7e5fea613a77812e414756e8a1b9a9cb7e15728499de0ec815: Status 404 returned error can't find the container with id bd26eb783b7e3c7e5fea613a77812e414756e8a1b9a9cb7e15728499de0ec815 Jul 01 22:27:41.415190 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:41.415154 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rwcn2" Jul 01 22:27:41.467097 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:41.467063 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq" event={"ID":"4347a2a2-92d5-4115-ad2e-7571d8fb1e25","Type":"ContainerStarted","Data":"bd26eb783b7e3c7e5fea613a77812e414756e8a1b9a9cb7e15728499de0ec815"} Jul 01 22:27:46.493132 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:46.493042 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq" event={"ID":"4347a2a2-92d5-4115-ad2e-7571d8fb1e25","Type":"ContainerStarted","Data":"e0faefc3e9ab18b1dac37e3ed0a9b578ae6d76e780e8224d802bf35ad886a4e4"} Jul 01 22:27:46.493509 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:46.493170 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq" Jul 01 22:27:46.517643 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:46.517594 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq" podStartSLOduration=1.874895506 podStartE2EDuration="7.517581305s" totalImagesPullingTime="5.642685799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:27:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:27:46.515853915 +0000 UTC m=+407.263294505" watchObservedRunningTime="2026-07-01 22:27:46.517581305 +0000 UTC m=+407.265021892" Jul 01 22:27:49.454202 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:49.454154 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-ft7mh" Jul 01 22:27:57.500492 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:57.500461 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq" Jul 01 22:27:58.993243 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:58.993211 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v"] Jul 01 22:27:58.996655 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:58.996631 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v" Jul 01 22:27:59.010973 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.010949 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v"] Jul 01 22:27:59.125077 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.125054 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fd6d26a-054a-4333-a843-165742c78b8c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-hcw8v\" (UID: \"5fd6d26a-054a-4333-a843-165742c78b8c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v" Jul 01 22:27:59.125186 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.125092 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krh9t\" (UniqueName: \"kubernetes.io/projected/5fd6d26a-054a-4333-a843-165742c78b8c-kube-api-access-krh9t\") pod \"kuadrant-operator-controller-manager-55467d9f7-hcw8v\" (UID: \"5fd6d26a-054a-4333-a843-165742c78b8c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v" Jul 01 22:27:59.225582 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.225545 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fd6d26a-054a-4333-a843-165742c78b8c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-hcw8v\" (UID: \"5fd6d26a-054a-4333-a843-165742c78b8c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v" Jul 01 22:27:59.225691 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.225603 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krh9t\" (UniqueName: \"kubernetes.io/projected/5fd6d26a-054a-4333-a843-165742c78b8c-kube-api-access-krh9t\") pod \"kuadrant-operator-controller-manager-55467d9f7-hcw8v\" (UID: \"5fd6d26a-054a-4333-a843-165742c78b8c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v" Jul 01 22:27:59.225903 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.225884 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fd6d26a-054a-4333-a843-165742c78b8c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-hcw8v\" (UID: \"5fd6d26a-054a-4333-a843-165742c78b8c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v" Jul 01 22:27:59.236605 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.236578 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krh9t\" (UniqueName: \"kubernetes.io/projected/5fd6d26a-054a-4333-a843-165742c78b8c-kube-api-access-krh9t\") pod \"kuadrant-operator-controller-manager-55467d9f7-hcw8v\" (UID: \"5fd6d26a-054a-4333-a843-165742c78b8c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v" Jul 01 22:27:59.306693 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.306631 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v" Jul 01 22:27:59.379134 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.379102 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v"] Jul 01 22:27:59.383713 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.383686 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v"] Jul 01 22:27:59.410088 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.410061 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm"] Jul 01 22:27:59.414113 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.414093 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm" Jul 01 22:27:59.415902 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.415882 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq"] Jul 01 22:27:59.416053 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.416034 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq" podUID="4347a2a2-92d5-4115-ad2e-7571d8fb1e25" containerName="manager" containerID="cri-o://e0faefc3e9ab18b1dac37e3ed0a9b578ae6d76e780e8224d802bf35ad886a4e4" gracePeriod=2 Jul 01 22:27:59.419212 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.419186 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq"] Jul 01 22:27:59.430747 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.430724 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm"] Jul 01 22:27:59.439853 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.439829 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rwcn2"] Jul 01 22:27:59.440060 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.440035 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rwcn2" podUID="be2fe675-dd8c-4f97-ad39-f1891ccb2cc8" containerName="manager" containerID="cri-o://952d0f75364360dcfa55f857aed80e10ded63dea01f95e0d32b710a8484838de" gracePeriod=2 Jul 01 22:27:59.453342 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:27:59.453216 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-77sjq\" is forbidden: User \"system:node:ip-10-0-129-238.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-238.ec2.internal' and this object" podUID="4347a2a2-92d5-4115-ad2e-7571d8fb1e25" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq" Jul 01 22:27:59.454103 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.454079 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rwcn2"] Jul 01 22:27:59.455289 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:27:59.455262 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rwcn2\" is forbidden: User \"system:node:ip-10-0-129-238.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-238.ec2.internal' and this object" podUID="be2fe675-dd8c-4f97-ad39-f1891ccb2cc8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rwcn2" Jul 01 22:27:59.486853 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.486824 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rfpms"] Jul 01 22:27:59.487314 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.487295 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4347a2a2-92d5-4115-ad2e-7571d8fb1e25" containerName="manager" Jul 01 22:27:59.487314 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.487315 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4347a2a2-92d5-4115-ad2e-7571d8fb1e25" containerName="manager" Jul 01 22:27:59.487459 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.487354 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be2fe675-dd8c-4f97-ad39-f1891ccb2cc8" containerName="manager" Jul 01 22:27:59.487459 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.487363 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="be2fe675-dd8c-4f97-ad39-f1891ccb2cc8" containerName="manager" Jul 01 22:27:59.487459 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.487446 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="be2fe675-dd8c-4f97-ad39-f1891ccb2cc8" containerName="manager" Jul 01 22:27:59.487459 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.487459 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4347a2a2-92d5-4115-ad2e-7571d8fb1e25" containerName="manager" Jul 01 22:27:59.490547 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.490530 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rfpms" Jul 01 22:27:59.493293 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:27:59.493265 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rwcn2\" is forbidden: User \"system:node:ip-10-0-129-238.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-238.ec2.internal' and this object" podUID="be2fe675-dd8c-4f97-ad39-f1891ccb2cc8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rwcn2" Jul 01 22:27:59.504399 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.504343 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rfpms"] Jul 01 22:27:59.524886 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:27:59.524834 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-77sjq\" is forbidden: User \"system:node:ip-10-0-129-238.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-238.ec2.internal' and this object" podUID="4347a2a2-92d5-4115-ad2e-7571d8fb1e25" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq" Jul 01 22:27:59.528316 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.528288 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8743787b-5af3-4c74-946f-f07dc44de775-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-m5kcm\" (UID: \"8743787b-5af3-4c74-946f-f07dc44de775\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm" Jul 01 22:27:59.528438 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.528341 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztc6x\" (UniqueName: \"kubernetes.io/projected/8743787b-5af3-4c74-946f-f07dc44de775-kube-api-access-ztc6x\") pod \"kuadrant-operator-controller-manager-55467d9f7-m5kcm\" (UID: \"8743787b-5af3-4c74-946f-f07dc44de775\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm" Jul 01 22:27:59.553772 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.553752 2580 generic.go:356] "Generic (PLEG): container finished" podID="4347a2a2-92d5-4115-ad2e-7571d8fb1e25" containerID="e0faefc3e9ab18b1dac37e3ed0a9b578ae6d76e780e8224d802bf35ad886a4e4" exitCode=0 Jul 01 22:27:59.580727 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:27:59.580684 2580 log.go:32] "RunPodSandbox from runtime service failed" err=< Jul 01 22:27:59.580727 ip-10-0-129-238 kubenswrapper[2580]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-hcw8v_kuadrant-system_5fd6d26a-054a-4333-a843-165742c78b8c_0(da0aa4467db98f296337740f56b680e6cc8ec31f98185c5d501f3434d9012f12): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-hcw8v to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"da0aa4467db98f296337740f56b680e6cc8ec31f98185c5d501f3434d9012f12" Netns:"/var/run/netns/823cbd45-59ee-41a9-bc1c-e530c77f392b" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-hcw8v;K8S_POD_INFRA_CONTAINER_ID=da0aa4467db98f296337740f56b680e6cc8ec31f98185c5d501f3434d9012f12;K8S_POD_UID=5fd6d26a-054a-4333-a843-165742c78b8c" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v] networking: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v/5fd6d26a-054a-4333-a843-165742c78b8c:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v da0aa4467db98f296337740f56b680e6cc8ec31f98185c5d501f3434d9012f12 network default NAD default NAD key default] [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v da0aa4467db98f296337740f56b680e6cc8ec31f98185c5d501f3434d9012f12 network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:84:00:21 [10.132.0.33/23] Jul 01 22:27:59.580727 ip-10-0-129-238 kubenswrapper[2580]: ' Jul 01 22:27:59.580727 ip-10-0-129-238 kubenswrapper[2580]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Jul 01 22:27:59.580727 ip-10-0-129-238 kubenswrapper[2580]: > Jul 01 22:27:59.580969 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:27:59.580758 2580 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=< Jul 01 22:27:59.580969 ip-10-0-129-238 kubenswrapper[2580]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-hcw8v_kuadrant-system_5fd6d26a-054a-4333-a843-165742c78b8c_0(da0aa4467db98f296337740f56b680e6cc8ec31f98185c5d501f3434d9012f12): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-hcw8v to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"da0aa4467db98f296337740f56b680e6cc8ec31f98185c5d501f3434d9012f12" Netns:"/var/run/netns/823cbd45-59ee-41a9-bc1c-e530c77f392b" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-hcw8v;K8S_POD_INFRA_CONTAINER_ID=da0aa4467db98f296337740f56b680e6cc8ec31f98185c5d501f3434d9012f12;K8S_POD_UID=5fd6d26a-054a-4333-a843-165742c78b8c" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v] networking: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v/5fd6d26a-054a-4333-a843-165742c78b8c:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v da0aa4467db98f296337740f56b680e6cc8ec31f98185c5d501f3434d9012f12 network default NAD default NAD key default] [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v da0aa4467db98f296337740f56b680e6cc8ec31f98185c5d501f3434d9012f12 network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:84:00:21 [10.132.0.33/23] Jul 01 22:27:59.580969 ip-10-0-129-238 kubenswrapper[2580]: ' Jul 01 22:27:59.580969 ip-10-0-129-238 kubenswrapper[2580]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} Jul 01 22:27:59.580969 ip-10-0-129-238 kubenswrapper[2580]: > pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v" Jul 01 22:27:59.628987 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.628959 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6msp\" (UniqueName: \"kubernetes.io/projected/b3766e89-67d4-4fa3-a3a6-3fcac443522b-kube-api-access-p6msp\") pod \"limitador-operator-controller-manager-859d7b78bd-rfpms\" (UID: \"b3766e89-67d4-4fa3-a3a6-3fcac443522b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rfpms" Jul 01 22:27:59.629106 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.629034 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8743787b-5af3-4c74-946f-f07dc44de775-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-m5kcm\" (UID: \"8743787b-5af3-4c74-946f-f07dc44de775\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm" Jul 01 22:27:59.629106 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.629082 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztc6x\" (UniqueName: \"kubernetes.io/projected/8743787b-5af3-4c74-946f-f07dc44de775-kube-api-access-ztc6x\") pod \"kuadrant-operator-controller-manager-55467d9f7-m5kcm\" (UID: \"8743787b-5af3-4c74-946f-f07dc44de775\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm" Jul 01 22:27:59.629470 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.629435 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8743787b-5af3-4c74-946f-f07dc44de775-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-m5kcm\" (UID: \"8743787b-5af3-4c74-946f-f07dc44de775\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm" Jul 01 22:27:59.642680 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.642656 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztc6x\" (UniqueName: \"kubernetes.io/projected/8743787b-5af3-4c74-946f-f07dc44de775-kube-api-access-ztc6x\") pod \"kuadrant-operator-controller-manager-55467d9f7-m5kcm\" (UID: \"8743787b-5af3-4c74-946f-f07dc44de775\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm" Jul 01 22:27:59.685415 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.685390 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq" Jul 01 22:27:59.687304 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.687290 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rwcn2" Jul 01 22:27:59.690878 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:27:59.690848 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rwcn2\" is forbidden: User \"system:node:ip-10-0-129-238.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-238.ec2.internal' and this object" podUID="be2fe675-dd8c-4f97-ad39-f1891ccb2cc8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rwcn2" Jul 01 22:27:59.693903 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:27:59.693880 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-77sjq\" is forbidden: User \"system:node:ip-10-0-129-238.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-238.ec2.internal' and this object" podUID="4347a2a2-92d5-4115-ad2e-7571d8fb1e25" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq" Jul 01 22:27:59.700024 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:27:59.700000 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rwcn2\" is forbidden: User \"system:node:ip-10-0-129-238.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-238.ec2.internal' and this object" podUID="be2fe675-dd8c-4f97-ad39-f1891ccb2cc8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rwcn2" Jul 01 22:27:59.706086 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:27:59.706063 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-77sjq\" is forbidden: User \"system:node:ip-10-0-129-238.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-238.ec2.internal' and this object" podUID="4347a2a2-92d5-4115-ad2e-7571d8fb1e25" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq" Jul 01 22:27:59.727520 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.727498 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm" Jul 01 22:27:59.729848 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.729830 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6msp\" (UniqueName: \"kubernetes.io/projected/b3766e89-67d4-4fa3-a3a6-3fcac443522b-kube-api-access-p6msp\") pod \"limitador-operator-controller-manager-859d7b78bd-rfpms\" (UID: \"b3766e89-67d4-4fa3-a3a6-3fcac443522b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rfpms" Jul 01 22:27:59.748801 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:27:59.748770 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-rwcn2\" is forbidden: User \"system:node:ip-10-0-129-238.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-238.ec2.internal' and this object" podUID="be2fe675-dd8c-4f97-ad39-f1891ccb2cc8" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rwcn2" Jul 01 22:27:59.751238 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.751219 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6msp\" (UniqueName: \"kubernetes.io/projected/b3766e89-67d4-4fa3-a3a6-3fcac443522b-kube-api-access-p6msp\") pod \"limitador-operator-controller-manager-859d7b78bd-rfpms\" (UID: \"b3766e89-67d4-4fa3-a3a6-3fcac443522b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rfpms" Jul 01 22:27:59.751368 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:27:59.751345 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-77sjq\" is forbidden: User \"system:node:ip-10-0-129-238.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-238.ec2.internal' and this object" podUID="4347a2a2-92d5-4115-ad2e-7571d8fb1e25" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq" Jul 01 22:27:59.830658 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.830589 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w865r\" (UniqueName: \"kubernetes.io/projected/be2fe675-dd8c-4f97-ad39-f1891ccb2cc8-kube-api-access-w865r\") pod \"be2fe675-dd8c-4f97-ad39-f1891ccb2cc8\" (UID: \"be2fe675-dd8c-4f97-ad39-f1891ccb2cc8\") " Jul 01 22:27:59.830784 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.830671 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4347a2a2-92d5-4115-ad2e-7571d8fb1e25-extensions-socket-volume\") pod \"4347a2a2-92d5-4115-ad2e-7571d8fb1e25\" (UID: \"4347a2a2-92d5-4115-ad2e-7571d8fb1e25\") " Jul 01 22:27:59.830784 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.830730 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddf4g\" (UniqueName: \"kubernetes.io/projected/4347a2a2-92d5-4115-ad2e-7571d8fb1e25-kube-api-access-ddf4g\") pod \"4347a2a2-92d5-4115-ad2e-7571d8fb1e25\" (UID: \"4347a2a2-92d5-4115-ad2e-7571d8fb1e25\") " Jul 01 22:27:59.831411 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.831243 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4347a2a2-92d5-4115-ad2e-7571d8fb1e25-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4347a2a2-92d5-4115-ad2e-7571d8fb1e25" (UID: "4347a2a2-92d5-4115-ad2e-7571d8fb1e25"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 22:27:59.833162 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.832877 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rfpms" Jul 01 22:27:59.833162 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.833074 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4347a2a2-92d5-4115-ad2e-7571d8fb1e25-extensions-socket-volume\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:27:59.833387 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.833356 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be2fe675-dd8c-4f97-ad39-f1891ccb2cc8-kube-api-access-w865r" (OuterVolumeSpecName: "kube-api-access-w865r") pod "be2fe675-dd8c-4f97-ad39-f1891ccb2cc8" (UID: "be2fe675-dd8c-4f97-ad39-f1891ccb2cc8"). InnerVolumeSpecName "kube-api-access-w865r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 22:27:59.833949 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.833923 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4347a2a2-92d5-4115-ad2e-7571d8fb1e25-kube-api-access-ddf4g" (OuterVolumeSpecName: "kube-api-access-ddf4g") pod "4347a2a2-92d5-4115-ad2e-7571d8fb1e25" (UID: "4347a2a2-92d5-4115-ad2e-7571d8fb1e25"). InnerVolumeSpecName "kube-api-access-ddf4g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 22:27:59.880924 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.880898 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm"] Jul 01 22:27:59.882355 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:27:59.882327 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8743787b_5af3_4c74_946f_f07dc44de775.slice/crio-670c68656c80abe43de389dc0c490c310dc6af63faac2e20197290cd8618a810 WatchSource:0}: Error finding container 670c68656c80abe43de389dc0c490c310dc6af63faac2e20197290cd8618a810: Status 404 returned error can't find the container with id 670c68656c80abe43de389dc0c490c310dc6af63faac2e20197290cd8618a810 Jul 01 22:27:59.934196 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.934169 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddf4g\" (UniqueName: \"kubernetes.io/projected/4347a2a2-92d5-4115-ad2e-7571d8fb1e25-kube-api-access-ddf4g\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:27:59.934196 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.934197 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w865r\" (UniqueName: \"kubernetes.io/projected/be2fe675-dd8c-4f97-ad39-f1891ccb2cc8-kube-api-access-w865r\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:27:59.980960 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:27:59.980937 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rfpms"] Jul 01 22:27:59.983066 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:27:59.983046 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3766e89_67d4_4fa3_a3a6_3fcac443522b.slice/crio-480d5857dd17a196ea32dcc90ed50ed251b4a67c6ce84951e92a682dec6d39da WatchSource:0}: Error finding container 480d5857dd17a196ea32dcc90ed50ed251b4a67c6ce84951e92a682dec6d39da: Status 404 returned error can't find the container with id 480d5857dd17a196ea32dcc90ed50ed251b4a67c6ce84951e92a682dec6d39da Jul 01 22:28:00.559872 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:00.559771 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm" event={"ID":"8743787b-5af3-4c74-946f-f07dc44de775","Type":"ContainerStarted","Data":"2f96658e9dadec156ac5bf50c6bb605607267e7e9ef77347ac46e30d21a13ce3"} Jul 01 22:28:00.559872 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:00.559836 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm" Jul 01 22:28:00.559872 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:00.559862 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm" event={"ID":"8743787b-5af3-4c74-946f-f07dc44de775","Type":"ContainerStarted","Data":"670c68656c80abe43de389dc0c490c310dc6af63faac2e20197290cd8618a810"} Jul 01 22:28:00.561146 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:00.561125 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-77sjq" Jul 01 22:28:00.561392 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:00.561130 2580 scope.go:117] "RemoveContainer" containerID="e0faefc3e9ab18b1dac37e3ed0a9b578ae6d76e780e8224d802bf35ad886a4e4" Jul 01 22:28:00.562389 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:00.562317 2580 generic.go:356] "Generic (PLEG): container finished" podID="be2fe675-dd8c-4f97-ad39-f1891ccb2cc8" containerID="952d0f75364360dcfa55f857aed80e10ded63dea01f95e0d32b710a8484838de" exitCode=0 Jul 01 22:28:00.562389 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:00.562364 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rwcn2" Jul 01 22:28:00.564087 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:00.564062 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rfpms" event={"ID":"b3766e89-67d4-4fa3-a3a6-3fcac443522b","Type":"ContainerStarted","Data":"46a4c09a77957da7867eeed07c25b1b8bf8a51156d06dfb9a5b93cb5ae93b45d"} Jul 01 22:28:00.564189 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:00.564089 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v" Jul 01 22:28:00.564189 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:00.564115 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rfpms" event={"ID":"b3766e89-67d4-4fa3-a3a6-3fcac443522b","Type":"ContainerStarted","Data":"480d5857dd17a196ea32dcc90ed50ed251b4a67c6ce84951e92a682dec6d39da"} Jul 01 22:28:00.564299 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:00.564202 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rfpms" Jul 01 22:28:00.572078 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:00.572062 2580 scope.go:117] "RemoveContainer" containerID="952d0f75364360dcfa55f857aed80e10ded63dea01f95e0d32b710a8484838de" Jul 01 22:28:00.572244 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:00.572225 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v" Jul 01 22:28:00.579503 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:00.579487 2580 scope.go:117] "RemoveContainer" containerID="952d0f75364360dcfa55f857aed80e10ded63dea01f95e0d32b710a8484838de" Jul 01 22:28:00.579746 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:28:00.579730 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"952d0f75364360dcfa55f857aed80e10ded63dea01f95e0d32b710a8484838de\": container with ID starting with 952d0f75364360dcfa55f857aed80e10ded63dea01f95e0d32b710a8484838de not found: ID does not exist" containerID="952d0f75364360dcfa55f857aed80e10ded63dea01f95e0d32b710a8484838de" Jul 01 22:28:00.579799 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:00.579753 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"952d0f75364360dcfa55f857aed80e10ded63dea01f95e0d32b710a8484838de"} err="failed to get container status \"952d0f75364360dcfa55f857aed80e10ded63dea01f95e0d32b710a8484838de\": rpc error: code = NotFound desc = could not find container \"952d0f75364360dcfa55f857aed80e10ded63dea01f95e0d32b710a8484838de\": container with ID starting with 952d0f75364360dcfa55f857aed80e10ded63dea01f95e0d32b710a8484838de not found: ID does not exist" Jul 01 22:28:00.636310 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:00.636262 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm" podStartSLOduration=1.6362495099999999 podStartE2EDuration="1.63624951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:27:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:28:00.601676673 +0000 UTC m=+421.349117263" watchObservedRunningTime="2026-07-01 22:28:00.63624951 +0000 UTC m=+421.383690100" Jul 01 22:28:00.636434 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:00.636347 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rfpms" podStartSLOduration=1.636342725 podStartE2EDuration="1.636342725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:27:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:28:00.634496414 +0000 UTC m=+421.381937005" watchObservedRunningTime="2026-07-01 22:28:00.636342725 +0000 UTC m=+421.383783314" Jul 01 22:28:00.641218 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:00.641198 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fd6d26a-054a-4333-a843-165742c78b8c-extensions-socket-volume\") pod \"5fd6d26a-054a-4333-a843-165742c78b8c\" (UID: \"5fd6d26a-054a-4333-a843-165742c78b8c\") " Jul 01 22:28:00.641295 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:00.641266 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-krh9t\" (UniqueName: \"kubernetes.io/projected/5fd6d26a-054a-4333-a843-165742c78b8c-kube-api-access-krh9t\") pod \"5fd6d26a-054a-4333-a843-165742c78b8c\" (UID: \"5fd6d26a-054a-4333-a843-165742c78b8c\") " Jul 01 22:28:00.641500 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:00.641475 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fd6d26a-054a-4333-a843-165742c78b8c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5fd6d26a-054a-4333-a843-165742c78b8c" (UID: "5fd6d26a-054a-4333-a843-165742c78b8c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 22:28:00.643339 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:00.643311 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fd6d26a-054a-4333-a843-165742c78b8c-kube-api-access-krh9t" (OuterVolumeSpecName: "kube-api-access-krh9t") pod "5fd6d26a-054a-4333-a843-165742c78b8c" (UID: "5fd6d26a-054a-4333-a843-165742c78b8c"). InnerVolumeSpecName "kube-api-access-krh9t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 22:28:00.648078 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:28:00.648050 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-hcw8v\" is forbidden: User \"system:node:ip-10-0-129-238.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-238.ec2.internal' and this object" podUID="5fd6d26a-054a-4333-a843-165742c78b8c" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v" Jul 01 22:28:00.741917 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:00.741894 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5fd6d26a-054a-4333-a843-165742c78b8c-extensions-socket-volume\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:28:00.741917 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:00.741916 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krh9t\" (UniqueName: \"kubernetes.io/projected/5fd6d26a-054a-4333-a843-165742c78b8c-kube-api-access-krh9t\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:28:01.570368 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:01.570336 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v" Jul 01 22:28:01.573860 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:28:01.573835 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-hcw8v\" is forbidden: User \"system:node:ip-10-0-129-238.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-238.ec2.internal' and this object" podUID="5fd6d26a-054a-4333-a843-165742c78b8c" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v" Jul 01 22:28:01.578545 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:28:01.578519 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-hcw8v\" is forbidden: User \"system:node:ip-10-0-129-238.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-238.ec2.internal' and this object" podUID="5fd6d26a-054a-4333-a843-165742c78b8c" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hcw8v" Jul 01 22:28:01.746920 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:01.746898 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4347a2a2-92d5-4115-ad2e-7571d8fb1e25" path="/var/lib/kubelet/pods/4347a2a2-92d5-4115-ad2e-7571d8fb1e25/volumes" Jul 01 22:28:01.747234 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:01.747221 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fd6d26a-054a-4333-a843-165742c78b8c" path="/var/lib/kubelet/pods/5fd6d26a-054a-4333-a843-165742c78b8c/volumes" Jul 01 22:28:01.747414 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:01.747403 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be2fe675-dd8c-4f97-ad39-f1891ccb2cc8" path="/var/lib/kubelet/pods/be2fe675-dd8c-4f97-ad39-f1891ccb2cc8/volumes" Jul 01 22:28:11.572776 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:11.572732 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm" Jul 01 22:28:11.573149 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:11.572821 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-rfpms" Jul 01 22:28:12.043205 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:12.043179 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsl8h"] Jul 01 22:28:12.049625 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:12.049607 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsl8h" Jul 01 22:28:12.070088 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:12.070067 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsl8h"] Jul 01 22:28:12.229199 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:12.229132 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5103f362-b8c8-4b9c-9307-5c0ea11c6015-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rsl8h\" (UID: \"5103f362-b8c8-4b9c-9307-5c0ea11c6015\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsl8h" Jul 01 22:28:12.229339 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:12.229201 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5q7v\" (UniqueName: \"kubernetes.io/projected/5103f362-b8c8-4b9c-9307-5c0ea11c6015-kube-api-access-p5q7v\") pod \"kuadrant-operator-controller-manager-86878c769f-rsl8h\" (UID: \"5103f362-b8c8-4b9c-9307-5c0ea11c6015\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsl8h" Jul 01 22:28:12.330328 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:12.330287 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5103f362-b8c8-4b9c-9307-5c0ea11c6015-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rsl8h\" (UID: \"5103f362-b8c8-4b9c-9307-5c0ea11c6015\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsl8h" Jul 01 22:28:12.330476 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:12.330408 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5q7v\" (UniqueName: \"kubernetes.io/projected/5103f362-b8c8-4b9c-9307-5c0ea11c6015-kube-api-access-p5q7v\") pod \"kuadrant-operator-controller-manager-86878c769f-rsl8h\" (UID: \"5103f362-b8c8-4b9c-9307-5c0ea11c6015\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsl8h" Jul 01 22:28:12.330670 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:12.330650 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5103f362-b8c8-4b9c-9307-5c0ea11c6015-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rsl8h\" (UID: \"5103f362-b8c8-4b9c-9307-5c0ea11c6015\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsl8h" Jul 01 22:28:12.347857 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:12.347836 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5q7v\" (UniqueName: \"kubernetes.io/projected/5103f362-b8c8-4b9c-9307-5c0ea11c6015-kube-api-access-p5q7v\") pod \"kuadrant-operator-controller-manager-86878c769f-rsl8h\" (UID: \"5103f362-b8c8-4b9c-9307-5c0ea11c6015\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsl8h" Jul 01 22:28:12.359670 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:12.359646 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsl8h" Jul 01 22:28:12.509988 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:12.509957 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsl8h"] Jul 01 22:28:12.512724 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:28:12.512697 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5103f362_b8c8_4b9c_9307_5c0ea11c6015.slice/crio-392e8d043ea6d9e9347f6d15a4425329a6debc4b5475c78ddd96732473e80f5f WatchSource:0}: Error finding container 392e8d043ea6d9e9347f6d15a4425329a6debc4b5475c78ddd96732473e80f5f: Status 404 returned error can't find the container with id 392e8d043ea6d9e9347f6d15a4425329a6debc4b5475c78ddd96732473e80f5f Jul 01 22:28:12.628093 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:12.628061 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsl8h" event={"ID":"5103f362-b8c8-4b9c-9307-5c0ea11c6015","Type":"ContainerStarted","Data":"199518d70f169cf80b1ff3f6f28919ecf8500f90a18b09cb3e86e7ee5cc6c931"} Jul 01 22:28:12.628416 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:12.628136 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsl8h" event={"ID":"5103f362-b8c8-4b9c-9307-5c0ea11c6015","Type":"ContainerStarted","Data":"392e8d043ea6d9e9347f6d15a4425329a6debc4b5475c78ddd96732473e80f5f"} Jul 01 22:28:12.628416 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:12.628156 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsl8h" Jul 01 22:28:12.662133 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:12.662081 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsl8h" podStartSLOduration=0.662067008 podStartE2EDuration="662.067008ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:28:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:28:12.661173299 +0000 UTC m=+433.408613889" watchObservedRunningTime="2026-07-01 22:28:12.662067008 +0000 UTC m=+433.409507597" Jul 01 22:28:23.636393 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:23.636321 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rsl8h" Jul 01 22:28:23.691626 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:23.691596 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm"] Jul 01 22:28:23.691821 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:23.691796 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm" podUID="8743787b-5af3-4c74-946f-f07dc44de775" containerName="manager" containerID="cri-o://2f96658e9dadec156ac5bf50c6bb605607267e7e9ef77347ac46e30d21a13ce3" gracePeriod=10 Jul 01 22:28:23.941272 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:23.941243 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm" Jul 01 22:28:24.016984 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:24.016957 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ztc6x\" (UniqueName: \"kubernetes.io/projected/8743787b-5af3-4c74-946f-f07dc44de775-kube-api-access-ztc6x\") pod \"8743787b-5af3-4c74-946f-f07dc44de775\" (UID: \"8743787b-5af3-4c74-946f-f07dc44de775\") " Jul 01 22:28:24.017099 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:24.017009 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8743787b-5af3-4c74-946f-f07dc44de775-extensions-socket-volume\") pod \"8743787b-5af3-4c74-946f-f07dc44de775\" (UID: \"8743787b-5af3-4c74-946f-f07dc44de775\") " Jul 01 22:28:24.017358 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:24.017338 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8743787b-5af3-4c74-946f-f07dc44de775-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8743787b-5af3-4c74-946f-f07dc44de775" (UID: "8743787b-5af3-4c74-946f-f07dc44de775"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 22:28:24.018904 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:24.018884 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8743787b-5af3-4c74-946f-f07dc44de775-kube-api-access-ztc6x" (OuterVolumeSpecName: "kube-api-access-ztc6x") pod "8743787b-5af3-4c74-946f-f07dc44de775" (UID: "8743787b-5af3-4c74-946f-f07dc44de775"). InnerVolumeSpecName "kube-api-access-ztc6x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 22:28:24.118281 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:24.118257 2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8743787b-5af3-4c74-946f-f07dc44de775-extensions-socket-volume\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:28:24.118281 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:24.118277 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztc6x\" (UniqueName: \"kubernetes.io/projected/8743787b-5af3-4c74-946f-f07dc44de775-kube-api-access-ztc6x\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:28:24.689006 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:24.688973 2580 generic.go:356] "Generic (PLEG): container finished" podID="8743787b-5af3-4c74-946f-f07dc44de775" containerID="2f96658e9dadec156ac5bf50c6bb605607267e7e9ef77347ac46e30d21a13ce3" exitCode=0 Jul 01 22:28:24.689385 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:24.689018 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm" event={"ID":"8743787b-5af3-4c74-946f-f07dc44de775","Type":"ContainerDied","Data":"2f96658e9dadec156ac5bf50c6bb605607267e7e9ef77347ac46e30d21a13ce3"} Jul 01 22:28:24.689385 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:24.689056 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm" event={"ID":"8743787b-5af3-4c74-946f-f07dc44de775","Type":"ContainerDied","Data":"670c68656c80abe43de389dc0c490c310dc6af63faac2e20197290cd8618a810"} Jul 01 22:28:24.689385 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:24.689078 2580 scope.go:117] "RemoveContainer" containerID="2f96658e9dadec156ac5bf50c6bb605607267e7e9ef77347ac46e30d21a13ce3" Jul 01 22:28:24.689385 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:24.689021 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm" Jul 01 22:28:24.698235 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:24.698045 2580 scope.go:117] "RemoveContainer" containerID="2f96658e9dadec156ac5bf50c6bb605607267e7e9ef77347ac46e30d21a13ce3" Jul 01 22:28:24.698328 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:28:24.698308 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f96658e9dadec156ac5bf50c6bb605607267e7e9ef77347ac46e30d21a13ce3\": container with ID starting with 2f96658e9dadec156ac5bf50c6bb605607267e7e9ef77347ac46e30d21a13ce3 not found: ID does not exist" containerID="2f96658e9dadec156ac5bf50c6bb605607267e7e9ef77347ac46e30d21a13ce3" Jul 01 22:28:24.698374 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:24.698337 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f96658e9dadec156ac5bf50c6bb605607267e7e9ef77347ac46e30d21a13ce3"} err="failed to get container status \"2f96658e9dadec156ac5bf50c6bb605607267e7e9ef77347ac46e30d21a13ce3\": rpc error: code = NotFound desc = could not find container \"2f96658e9dadec156ac5bf50c6bb605607267e7e9ef77347ac46e30d21a13ce3\": container with ID starting with 2f96658e9dadec156ac5bf50c6bb605607267e7e9ef77347ac46e30d21a13ce3 not found: ID does not exist" Jul 01 22:28:24.714997 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:24.714976 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm"] Jul 01 22:28:24.718925 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:24.718906 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-m5kcm"] Jul 01 22:28:25.747153 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:25.747122 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8743787b-5af3-4c74-946f-f07dc44de775" path="/var/lib/kubelet/pods/8743787b-5af3-4c74-946f-f07dc44de775/volumes" Jul 01 22:28:28.031523 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.031488 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c"] Jul 01 22:28:28.032043 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.032024 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8743787b-5af3-4c74-946f-f07dc44de775" containerName="manager" Jul 01 22:28:28.032085 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.032046 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8743787b-5af3-4c74-946f-f07dc44de775" containerName="manager" Jul 01 22:28:28.032146 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.032134 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8743787b-5af3-4c74-946f-f07dc44de775" containerName="manager" Jul 01 22:28:28.036708 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.036691 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.039374 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.039346 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-bk4rf\"" Jul 01 22:28:28.046177 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.046155 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c"] Jul 01 22:28:28.046440 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.046421 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7640cec-b7f8-4db1-95ab-f929ffac82c8-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.046487 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.046450 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7640cec-b7f8-4db1-95ab-f929ffac82c8-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.046538 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.046486 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7640cec-b7f8-4db1-95ab-f929ffac82c8-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.046615 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.046593 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7640cec-b7f8-4db1-95ab-f929ffac82c8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.046671 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.046637 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7640cec-b7f8-4db1-95ab-f929ffac82c8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.046720 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.046696 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7640cec-b7f8-4db1-95ab-f929ffac82c8-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.046770 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.046730 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4qxh\" (UniqueName: \"kubernetes.io/projected/d7640cec-b7f8-4db1-95ab-f929ffac82c8-kube-api-access-h4qxh\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.046770 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.046753 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7640cec-b7f8-4db1-95ab-f929ffac82c8-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.046881 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.046791 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7640cec-b7f8-4db1-95ab-f929ffac82c8-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.147952 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.147923 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7640cec-b7f8-4db1-95ab-f929ffac82c8-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.148125 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.147967 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7640cec-b7f8-4db1-95ab-f929ffac82c8-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.148125 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.147989 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7640cec-b7f8-4db1-95ab-f929ffac82c8-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.148125 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.148012 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7640cec-b7f8-4db1-95ab-f929ffac82c8-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.148125 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.148048 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7640cec-b7f8-4db1-95ab-f929ffac82c8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.148125 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.148080 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7640cec-b7f8-4db1-95ab-f929ffac82c8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.148125 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.148121 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7640cec-b7f8-4db1-95ab-f929ffac82c8-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.148448 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.148151 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4qxh\" (UniqueName: \"kubernetes.io/projected/d7640cec-b7f8-4db1-95ab-f929ffac82c8-kube-api-access-h4qxh\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.148448 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.148182 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7640cec-b7f8-4db1-95ab-f929ffac82c8-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.148448 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.148356 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7640cec-b7f8-4db1-95ab-f929ffac82c8-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.148693 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.148443 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d7640cec-b7f8-4db1-95ab-f929ffac82c8-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.148693 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.148548 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d7640cec-b7f8-4db1-95ab-f929ffac82c8-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.148693 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.148627 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d7640cec-b7f8-4db1-95ab-f929ffac82c8-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.148824 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.148805 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d7640cec-b7f8-4db1-95ab-f929ffac82c8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.150424 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.150404 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d7640cec-b7f8-4db1-95ab-f929ffac82c8-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.150650 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.150632 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d7640cec-b7f8-4db1-95ab-f929ffac82c8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.156113 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.156090 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d7640cec-b7f8-4db1-95ab-f929ffac82c8-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.156741 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.156725 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4qxh\" (UniqueName: \"kubernetes.io/projected/d7640cec-b7f8-4db1-95ab-f929ffac82c8-kube-api-access-h4qxh\") pod \"maas-default-gateway-openshift-default-56654dcf7-xr74c\" (UID: \"d7640cec-b7f8-4db1-95ab-f929ffac82c8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.348204 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.348169 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:28.486808 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.486782 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c"] Jul 01 22:28:28.488784 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:28:28.488757 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7640cec_b7f8_4db1_95ab_f929ffac82c8.slice/crio-c154f8cc9ab763fc4792ea83f296cd8282fe63236b1c2af0a08cf3ef3a18c809 WatchSource:0}: Error finding container c154f8cc9ab763fc4792ea83f296cd8282fe63236b1c2af0a08cf3ef3a18c809: Status 404 returned error can't find the container with id c154f8cc9ab763fc4792ea83f296cd8282fe63236b1c2af0a08cf3ef3a18c809 Jul 01 22:28:28.490824 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.490789 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 01 22:28:28.490932 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.490869 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 01 22:28:28.490932 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.490913 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 01 22:28:28.712282 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.712209 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" event={"ID":"d7640cec-b7f8-4db1-95ab-f929ffac82c8","Type":"ContainerStarted","Data":"3d5c6b2ddc15cb8190726ac133e98947d74713362233af287bf77010237ff466"} Jul 01 22:28:28.712282 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.712262 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" event={"ID":"d7640cec-b7f8-4db1-95ab-f929ffac82c8","Type":"ContainerStarted","Data":"c154f8cc9ab763fc4792ea83f296cd8282fe63236b1c2af0a08cf3ef3a18c809"} Jul 01 22:28:28.737935 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:28.737887 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" podStartSLOduration=0.737872301 podStartE2EDuration="737.872301ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:28:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:28:28.736064897 +0000 UTC m=+449.483505487" watchObservedRunningTime="2026-07-01 22:28:28.737872301 +0000 UTC m=+449.485312891" Jul 01 22:28:29.349115 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:29.349085 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:29.353627 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:29.353603 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:29.717242 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:29.717154 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:29.718191 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:29.718170 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:28:44.339266 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:44.339236 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-psw9d"] Jul 01 22:28:44.343645 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:44.343624 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-psw9d" Jul 01 22:28:44.346339 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:44.346318 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6gzng\"" Jul 01 22:28:44.351477 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:44.351459 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-psw9d"] Jul 01 22:28:44.371366 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:44.371339 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzfc6\" (UniqueName: \"kubernetes.io/projected/e9538eac-76bc-408d-bd88-6a1401d3097b-kube-api-access-dzfc6\") pod \"authorino-58cd955675-psw9d\" (UID: \"e9538eac-76bc-408d-bd88-6a1401d3097b\") " pod="kuadrant-system/authorino-58cd955675-psw9d" Jul 01 22:28:44.472459 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:44.472426 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzfc6\" (UniqueName: \"kubernetes.io/projected/e9538eac-76bc-408d-bd88-6a1401d3097b-kube-api-access-dzfc6\") pod \"authorino-58cd955675-psw9d\" (UID: \"e9538eac-76bc-408d-bd88-6a1401d3097b\") " pod="kuadrant-system/authorino-58cd955675-psw9d" Jul 01 22:28:44.481097 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:44.481070 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzfc6\" (UniqueName: \"kubernetes.io/projected/e9538eac-76bc-408d-bd88-6a1401d3097b-kube-api-access-dzfc6\") pod \"authorino-58cd955675-psw9d\" (UID: \"e9538eac-76bc-408d-bd88-6a1401d3097b\") " pod="kuadrant-system/authorino-58cd955675-psw9d" Jul 01 22:28:44.559589 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:44.559547 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-b6psw"] Jul 01 22:28:44.562761 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:44.562745 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-b6psw" Jul 01 22:28:44.568655 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:44.568629 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b6psw"] Jul 01 22:28:44.653703 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:44.653674 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-psw9d" Jul 01 22:28:44.673876 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:44.673853 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d25c8\" (UniqueName: \"kubernetes.io/projected/85c2664f-f419-4ea7-a4d7-e86970951839-kube-api-access-d25c8\") pod \"authorino-75d556bd77-b6psw\" (UID: \"85c2664f-f419-4ea7-a4d7-e86970951839\") " pod="kuadrant-system/authorino-75d556bd77-b6psw" Jul 01 22:28:44.774289 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:44.774257 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d25c8\" (UniqueName: \"kubernetes.io/projected/85c2664f-f419-4ea7-a4d7-e86970951839-kube-api-access-d25c8\") pod \"authorino-75d556bd77-b6psw\" (UID: \"85c2664f-f419-4ea7-a4d7-e86970951839\") " pod="kuadrant-system/authorino-75d556bd77-b6psw" Jul 01 22:28:44.782662 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:44.782581 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d25c8\" (UniqueName: \"kubernetes.io/projected/85c2664f-f419-4ea7-a4d7-e86970951839-kube-api-access-d25c8\") pod \"authorino-75d556bd77-b6psw\" (UID: \"85c2664f-f419-4ea7-a4d7-e86970951839\") " pod="kuadrant-system/authorino-75d556bd77-b6psw" Jul 01 22:28:44.785661 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:44.785639 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-psw9d"] Jul 01 22:28:44.787703 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:28:44.787681 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode9538eac_76bc_408d_bd88_6a1401d3097b.slice/crio-e2e115ecd742476af436e74b5b3a28fbe3d22a7afe32c2e8a9e177a827e30bdd WatchSource:0}: Error finding container e2e115ecd742476af436e74b5b3a28fbe3d22a7afe32c2e8a9e177a827e30bdd: Status 404 returned error can't find the container with id e2e115ecd742476af436e74b5b3a28fbe3d22a7afe32c2e8a9e177a827e30bdd Jul 01 22:28:44.873238 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:44.873219 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-b6psw" Jul 01 22:28:44.999962 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:44.999937 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b6psw"] Jul 01 22:28:45.002137 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:28:45.002114 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod85c2664f_f419_4ea7_a4d7_e86970951839.slice/crio-f22c2ce9a44e58c5a2fcdffc46f3688b9f446987646c2acbf702659966319687 WatchSource:0}: Error finding container f22c2ce9a44e58c5a2fcdffc46f3688b9f446987646c2acbf702659966319687: Status 404 returned error can't find the container with id f22c2ce9a44e58c5a2fcdffc46f3688b9f446987646c2acbf702659966319687 Jul 01 22:28:45.802648 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:45.802610 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-psw9d" event={"ID":"e9538eac-76bc-408d-bd88-6a1401d3097b","Type":"ContainerStarted","Data":"e2e115ecd742476af436e74b5b3a28fbe3d22a7afe32c2e8a9e177a827e30bdd"} Jul 01 22:28:45.805238 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:45.805208 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-b6psw" event={"ID":"85c2664f-f419-4ea7-a4d7-e86970951839","Type":"ContainerStarted","Data":"f22c2ce9a44e58c5a2fcdffc46f3688b9f446987646c2acbf702659966319687"} Jul 01 22:28:48.824236 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:48.824190 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-psw9d" event={"ID":"e9538eac-76bc-408d-bd88-6a1401d3097b","Type":"ContainerStarted","Data":"e00e5d650cf802c5a0d4216869eeccf676e6bed425a0539eec2891b3b99defeb"} Jul 01 22:28:48.825876 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:48.825849 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-b6psw" event={"ID":"85c2664f-f419-4ea7-a4d7-e86970951839","Type":"ContainerStarted","Data":"9e5314b4c91507315e2d41b5dc53abc2dad9202a23f62872b10ef86df682567e"} Jul 01 22:28:48.841371 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:48.841333 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-psw9d" podStartSLOduration=1.406314099 podStartE2EDuration="4.841321673s" totalImagesPullingTime="3.435007574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:28:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:28:48.83959058 +0000 UTC m=+469.587031171" watchObservedRunningTime="2026-07-01 22:28:48.841321673 +0000 UTC m=+469.588762263" Jul 01 22:28:48.857177 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:48.857137 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-b6psw" podStartSLOduration=1.6254706909999999 podStartE2EDuration="4.857124197s" totalImagesPullingTime="3.231653506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:28:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:28:48.855773803 +0000 UTC m=+469.603214394" watchObservedRunningTime="2026-07-01 22:28:48.857124197 +0000 UTC m=+469.604564787" Jul 01 22:28:48.892791 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:48.892750 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-psw9d"] Jul 01 22:28:50.836161 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:50.836124 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-psw9d" podUID="e9538eac-76bc-408d-bd88-6a1401d3097b" containerName="authorino" containerID="cri-o://e00e5d650cf802c5a0d4216869eeccf676e6bed425a0539eec2891b3b99defeb" gracePeriod=30 Jul 01 22:28:51.077830 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:51.077806 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-psw9d" Jul 01 22:28:51.125055 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:51.125033 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzfc6\" (UniqueName: \"kubernetes.io/projected/e9538eac-76bc-408d-bd88-6a1401d3097b-kube-api-access-dzfc6\") pod \"e9538eac-76bc-408d-bd88-6a1401d3097b\" (UID: \"e9538eac-76bc-408d-bd88-6a1401d3097b\") " Jul 01 22:28:51.126953 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:51.126925 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9538eac-76bc-408d-bd88-6a1401d3097b-kube-api-access-dzfc6" (OuterVolumeSpecName: "kube-api-access-dzfc6") pod "e9538eac-76bc-408d-bd88-6a1401d3097b" (UID: "e9538eac-76bc-408d-bd88-6a1401d3097b"). InnerVolumeSpecName "kube-api-access-dzfc6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 22:28:51.226113 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:51.226090 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzfc6\" (UniqueName: \"kubernetes.io/projected/e9538eac-76bc-408d-bd88-6a1401d3097b-kube-api-access-dzfc6\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:28:51.841503 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:51.841470 2580 generic.go:356] "Generic (PLEG): container finished" podID="e9538eac-76bc-408d-bd88-6a1401d3097b" containerID="e00e5d650cf802c5a0d4216869eeccf676e6bed425a0539eec2891b3b99defeb" exitCode=0 Jul 01 22:28:51.841861 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:51.841518 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-psw9d" Jul 01 22:28:51.841861 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:51.841571 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-psw9d" event={"ID":"e9538eac-76bc-408d-bd88-6a1401d3097b","Type":"ContainerDied","Data":"e00e5d650cf802c5a0d4216869eeccf676e6bed425a0539eec2891b3b99defeb"} Jul 01 22:28:51.841861 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:51.841617 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-psw9d" event={"ID":"e9538eac-76bc-408d-bd88-6a1401d3097b","Type":"ContainerDied","Data":"e2e115ecd742476af436e74b5b3a28fbe3d22a7afe32c2e8a9e177a827e30bdd"} Jul 01 22:28:51.841861 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:51.841636 2580 scope.go:117] "RemoveContainer" containerID="e00e5d650cf802c5a0d4216869eeccf676e6bed425a0539eec2891b3b99defeb" Jul 01 22:28:51.849807 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:51.849791 2580 scope.go:117] "RemoveContainer" containerID="e00e5d650cf802c5a0d4216869eeccf676e6bed425a0539eec2891b3b99defeb" Jul 01 22:28:51.850052 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:28:51.850032 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e00e5d650cf802c5a0d4216869eeccf676e6bed425a0539eec2891b3b99defeb\": container with ID starting with e00e5d650cf802c5a0d4216869eeccf676e6bed425a0539eec2891b3b99defeb not found: ID does not exist" containerID="e00e5d650cf802c5a0d4216869eeccf676e6bed425a0539eec2891b3b99defeb" Jul 01 22:28:51.850111 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:51.850064 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e00e5d650cf802c5a0d4216869eeccf676e6bed425a0539eec2891b3b99defeb"} err="failed to get container status \"e00e5d650cf802c5a0d4216869eeccf676e6bed425a0539eec2891b3b99defeb\": rpc error: code = NotFound desc = could not find container \"e00e5d650cf802c5a0d4216869eeccf676e6bed425a0539eec2891b3b99defeb\": container with ID starting with e00e5d650cf802c5a0d4216869eeccf676e6bed425a0539eec2891b3b99defeb not found: ID does not exist" Jul 01 22:28:51.859072 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:51.859051 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-psw9d"] Jul 01 22:28:51.862618 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:51.862601 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-psw9d"] Jul 01 22:28:53.747036 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:28:53.746986 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9538eac-76bc-408d-bd88-6a1401d3097b" path="/var/lib/kubelet/pods/e9538eac-76bc-408d-bd88-6a1401d3097b/volumes" Jul 01 22:29:39.098358 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:39.098330 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 22:29:39.098758 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:39.098662 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9538eac-76bc-408d-bd88-6a1401d3097b" containerName="authorino" Jul 01 22:29:39.098758 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:39.098673 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9538eac-76bc-408d-bd88-6a1401d3097b" containerName="authorino" Jul 01 22:29:39.098758 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:39.098744 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9538eac-76bc-408d-bd88-6a1401d3097b" containerName="authorino" Jul 01 22:29:39.101659 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:39.101644 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 01 22:29:39.104569 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:39.104535 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jul 01 22:29:39.104693 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:39.104538 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jul 01 22:29:39.105789 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:39.105771 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-2lhws\"" Jul 01 22:29:39.105897 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:39.105803 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jul 01 22:29:39.112100 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:39.112080 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 22:29:39.175046 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:39.175019 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bfrz\" (UniqueName: \"kubernetes.io/projected/01484b49-950f-4ef4-87bd-a50d59539c50-kube-api-access-9bfrz\") pod \"maas-keycloak-0\" (UID: \"01484b49-950f-4ef4-87bd-a50d59539c50\") " pod="keycloak-system/maas-keycloak-0" Jul 01 22:29:39.276310 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:39.276284 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bfrz\" (UniqueName: \"kubernetes.io/projected/01484b49-950f-4ef4-87bd-a50d59539c50-kube-api-access-9bfrz\") pod \"maas-keycloak-0\" (UID: \"01484b49-950f-4ef4-87bd-a50d59539c50\") " pod="keycloak-system/maas-keycloak-0" Jul 01 22:29:39.285328 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:39.285305 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bfrz\" (UniqueName: \"kubernetes.io/projected/01484b49-950f-4ef4-87bd-a50d59539c50-kube-api-access-9bfrz\") pod \"maas-keycloak-0\" (UID: \"01484b49-950f-4ef4-87bd-a50d59539c50\") " pod="keycloak-system/maas-keycloak-0" Jul 01 22:29:39.411617 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:39.411543 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 01 22:29:39.542124 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:39.542104 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 22:29:39.543929 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:29:39.543899 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01484b49_950f_4ef4_87bd_a50d59539c50.slice/crio-e50023ae7db573be4055ce8414c390bc6e28457d4c9070c6cf2c2c2cde1708e4 WatchSource:0}: Error finding container e50023ae7db573be4055ce8414c390bc6e28457d4c9070c6cf2c2c2cde1708e4: Status 404 returned error can't find the container with id e50023ae7db573be4055ce8414c390bc6e28457d4c9070c6cf2c2c2cde1708e4 Jul 01 22:29:40.071724 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:40.071681 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"01484b49-950f-4ef4-87bd-a50d59539c50","Type":"ContainerStarted","Data":"e50023ae7db573be4055ce8414c390bc6e28457d4c9070c6cf2c2c2cde1708e4"} Jul 01 22:29:44.099810 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:44.099771 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"01484b49-950f-4ef4-87bd-a50d59539c50","Type":"ContainerStarted","Data":"521953481cba94d964fe8ddc95abb4fbcfa543eb4701a25c285c1acb4b97e6f8"} Jul 01 22:29:44.121404 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:44.121331 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=0.862289422 podStartE2EDuration="5.12131182s" totalImagesPullingTime="4.259022398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:29:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:29:44.120083919 +0000 UTC m=+524.867524510" watchObservedRunningTime="2026-07-01 22:29:44.12131182 +0000 UTC m=+524.868752411" Jul 01 22:29:44.412397 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:44.412299 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jul 01 22:29:44.413690 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:44.413657 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="01484b49-950f-4ef4-87bd-a50d59539c50" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.40:9000/health/started\": dial tcp 10.132.0.40:9000: connect: connection refused" Jul 01 22:29:45.412792 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:45.412744 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="01484b49-950f-4ef4-87bd-a50d59539c50" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.40:9000/health/started\": dial tcp 10.132.0.40:9000: connect: connection refused" Jul 01 22:29:46.412448 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:46.412394 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="01484b49-950f-4ef4-87bd-a50d59539c50" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.40:9000/health/started\": dial tcp 10.132.0.40:9000: connect: connection refused" Jul 01 22:29:47.412945 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:47.412891 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="01484b49-950f-4ef4-87bd-a50d59539c50" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.40:9000/health/started\": dial tcp 10.132.0.40:9000: connect: connection refused" Jul 01 22:29:48.413065 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:48.413018 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="01484b49-950f-4ef4-87bd-a50d59539c50" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.40:9000/health/started\": dial tcp 10.132.0.40:9000: connect: connection refused" Jul 01 22:29:49.412644 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:49.412546 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="01484b49-950f-4ef4-87bd-a50d59539c50" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.40:9000/health/started\": dial tcp 10.132.0.40:9000: connect: connection refused" Jul 01 22:29:49.413083 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:49.413057 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jul 01 22:29:50.412302 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:50.412249 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="01484b49-950f-4ef4-87bd-a50d59539c50" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.40:9000/health/started\": dial tcp 10.132.0.40:9000: connect: connection refused" Jul 01 22:29:51.412504 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:51.412451 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="01484b49-950f-4ef4-87bd-a50d59539c50" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.40:9000/health/started\": dial tcp 10.132.0.40:9000: connect: connection refused" Jul 01 22:29:52.412938 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:52.412844 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="01484b49-950f-4ef4-87bd-a50d59539c50" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.40:9000/health/started\": dial tcp 10.132.0.40:9000: connect: connection refused" Jul 01 22:29:53.412233 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:53.412191 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="01484b49-950f-4ef4-87bd-a50d59539c50" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.40:9000/health/started\": dial tcp 10.132.0.40:9000: connect: connection refused" Jul 01 22:29:54.412239 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:54.412192 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="01484b49-950f-4ef4-87bd-a50d59539c50" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.40:9000/health/started\": dial tcp 10.132.0.40:9000: connect: connection refused" Jul 01 22:29:55.412672 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:55.412631 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="01484b49-950f-4ef4-87bd-a50d59539c50" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.40:9000/health/started\": dial tcp 10.132.0.40:9000: connect: connection refused" Jul 01 22:29:56.412625 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:56.412545 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="01484b49-950f-4ef4-87bd-a50d59539c50" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.40:9000/health/started\": dial tcp 10.132.0.40:9000: connect: connection refused" Jul 01 22:29:57.524944 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:57.524889 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jul 01 22:29:57.542781 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:29:57.542718 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="01484b49-950f-4ef4-87bd-a50d59539c50" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 22:30:07.532544 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:07.532506 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jul 01 22:30:08.396655 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:08.396624 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-p5kvw"] Jul 01 22:30:08.403450 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:08.403429 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-p5kvw" Jul 01 22:30:08.408434 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:08.408414 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-p5kvw"] Jul 01 22:30:08.548511 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:08.548482 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97pt7\" (UniqueName: \"kubernetes.io/projected/a43ef56a-9c8b-4c5d-9d5b-fa0d7b2101bd-kube-api-access-97pt7\") pod \"authorino-5d4b67c446-p5kvw\" (UID: \"a43ef56a-9c8b-4c5d-9d5b-fa0d7b2101bd\") " pod="kuadrant-system/authorino-5d4b67c446-p5kvw" Jul 01 22:30:08.629099 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:08.629070 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-p5kvw"] Jul 01 22:30:08.629279 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:30:08.629260 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-97pt7], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-p5kvw" podUID="a43ef56a-9c8b-4c5d-9d5b-fa0d7b2101bd" Jul 01 22:30:08.649214 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:08.649155 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97pt7\" (UniqueName: \"kubernetes.io/projected/a43ef56a-9c8b-4c5d-9d5b-fa0d7b2101bd-kube-api-access-97pt7\") pod \"authorino-5d4b67c446-p5kvw\" (UID: \"a43ef56a-9c8b-4c5d-9d5b-fa0d7b2101bd\") " pod="kuadrant-system/authorino-5d4b67c446-p5kvw" Jul 01 22:30:08.654114 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:08.654092 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f7d4cc9fb-mhbbz"] Jul 01 22:30:08.656173 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:08.656158 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f7d4cc9fb-mhbbz" Jul 01 22:30:08.663861 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:08.663839 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97pt7\" (UniqueName: \"kubernetes.io/projected/a43ef56a-9c8b-4c5d-9d5b-fa0d7b2101bd-kube-api-access-97pt7\") pod \"authorino-5d4b67c446-p5kvw\" (UID: \"a43ef56a-9c8b-4c5d-9d5b-fa0d7b2101bd\") " pod="kuadrant-system/authorino-5d4b67c446-p5kvw" Jul 01 22:30:08.664777 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:08.664753 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f7d4cc9fb-mhbbz"] Jul 01 22:30:08.749771 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:08.749741 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgx8s\" (UniqueName: \"kubernetes.io/projected/8a29163d-d33d-40a4-9937-8bcbe5bcfdbc-kube-api-access-dgx8s\") pod \"authorino-6f7d4cc9fb-mhbbz\" (UID: \"8a29163d-d33d-40a4-9937-8bcbe5bcfdbc\") " pod="kuadrant-system/authorino-6f7d4cc9fb-mhbbz" Jul 01 22:30:08.850462 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:08.850436 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgx8s\" (UniqueName: \"kubernetes.io/projected/8a29163d-d33d-40a4-9937-8bcbe5bcfdbc-kube-api-access-dgx8s\") pod \"authorino-6f7d4cc9fb-mhbbz\" (UID: \"8a29163d-d33d-40a4-9937-8bcbe5bcfdbc\") " pod="kuadrant-system/authorino-6f7d4cc9fb-mhbbz" Jul 01 22:30:08.859965 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:08.859938 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgx8s\" (UniqueName: \"kubernetes.io/projected/8a29163d-d33d-40a4-9937-8bcbe5bcfdbc-kube-api-access-dgx8s\") pod \"authorino-6f7d4cc9fb-mhbbz\" (UID: \"8a29163d-d33d-40a4-9937-8bcbe5bcfdbc\") " pod="kuadrant-system/authorino-6f7d4cc9fb-mhbbz" Jul 01 22:30:08.974992 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:08.974927 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f7d4cc9fb-mhbbz" Jul 01 22:30:09.101210 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:09.101183 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f7d4cc9fb-mhbbz"] Jul 01 22:30:09.103827 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:30:09.103801 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a29163d_d33d_40a4_9937_8bcbe5bcfdbc.slice/crio-4486c7fe2ff1ab686079071d53c594d2bec4c78ce4a8252f36ebf72d87cd551c WatchSource:0}: Error finding container 4486c7fe2ff1ab686079071d53c594d2bec4c78ce4a8252f36ebf72d87cd551c: Status 404 returned error can't find the container with id 4486c7fe2ff1ab686079071d53c594d2bec4c78ce4a8252f36ebf72d87cd551c Jul 01 22:30:09.224577 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:09.224538 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f7d4cc9fb-mhbbz"] Jul 01 22:30:09.255114 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:09.255061 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-c55db47d4-tt55r"] Jul 01 22:30:09.255215 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:09.255199 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-p5kvw" Jul 01 22:30:09.257926 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:09.257907 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f7d4cc9fb-mhbbz" event={"ID":"8a29163d-d33d-40a4-9937-8bcbe5bcfdbc","Type":"ContainerStarted","Data":"4486c7fe2ff1ab686079071d53c594d2bec4c78ce4a8252f36ebf72d87cd551c"} Jul 01 22:30:09.258011 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:09.257984 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c55db47d4-tt55r" Jul 01 22:30:09.260936 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:09.260920 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-p5kvw" Jul 01 22:30:09.261235 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:09.261223 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 01 22:30:09.265511 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:09.265483 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c55db47d4-tt55r"] Jul 01 22:30:09.354613 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:09.354592 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97pt7\" (UniqueName: \"kubernetes.io/projected/a43ef56a-9c8b-4c5d-9d5b-fa0d7b2101bd-kube-api-access-97pt7\") pod \"a43ef56a-9c8b-4c5d-9d5b-fa0d7b2101bd\" (UID: \"a43ef56a-9c8b-4c5d-9d5b-fa0d7b2101bd\") " Jul 01 22:30:09.355159 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:09.355104 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/16f6c220-69a7-4b82-bb85-6dbe0149ee8e-tls-cert\") pod \"authorino-c55db47d4-tt55r\" (UID: \"16f6c220-69a7-4b82-bb85-6dbe0149ee8e\") " pod="kuadrant-system/authorino-c55db47d4-tt55r" Jul 01 22:30:09.355293 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:09.355201 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8c2v\" (UniqueName: \"kubernetes.io/projected/16f6c220-69a7-4b82-bb85-6dbe0149ee8e-kube-api-access-g8c2v\") pod \"authorino-c55db47d4-tt55r\" (UID: \"16f6c220-69a7-4b82-bb85-6dbe0149ee8e\") " pod="kuadrant-system/authorino-c55db47d4-tt55r" Jul 01 22:30:09.361247 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:09.361221 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a43ef56a-9c8b-4c5d-9d5b-fa0d7b2101bd-kube-api-access-97pt7" (OuterVolumeSpecName: "kube-api-access-97pt7") pod "a43ef56a-9c8b-4c5d-9d5b-fa0d7b2101bd" (UID: "a43ef56a-9c8b-4c5d-9d5b-fa0d7b2101bd"). InnerVolumeSpecName "kube-api-access-97pt7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 22:30:09.455864 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:09.455839 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/16f6c220-69a7-4b82-bb85-6dbe0149ee8e-tls-cert\") pod \"authorino-c55db47d4-tt55r\" (UID: \"16f6c220-69a7-4b82-bb85-6dbe0149ee8e\") " pod="kuadrant-system/authorino-c55db47d4-tt55r" Jul 01 22:30:09.455960 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:09.455872 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8c2v\" (UniqueName: \"kubernetes.io/projected/16f6c220-69a7-4b82-bb85-6dbe0149ee8e-kube-api-access-g8c2v\") pod \"authorino-c55db47d4-tt55r\" (UID: \"16f6c220-69a7-4b82-bb85-6dbe0149ee8e\") " pod="kuadrant-system/authorino-c55db47d4-tt55r" Jul 01 22:30:09.455960 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:09.455927 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97pt7\" (UniqueName: \"kubernetes.io/projected/a43ef56a-9c8b-4c5d-9d5b-fa0d7b2101bd-kube-api-access-97pt7\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:30:09.458262 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:09.458243 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/16f6c220-69a7-4b82-bb85-6dbe0149ee8e-tls-cert\") pod \"authorino-c55db47d4-tt55r\" (UID: \"16f6c220-69a7-4b82-bb85-6dbe0149ee8e\") " pod="kuadrant-system/authorino-c55db47d4-tt55r" Jul 01 22:30:09.464405 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:09.464386 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8c2v\" (UniqueName: \"kubernetes.io/projected/16f6c220-69a7-4b82-bb85-6dbe0149ee8e-kube-api-access-g8c2v\") pod \"authorino-c55db47d4-tt55r\" (UID: \"16f6c220-69a7-4b82-bb85-6dbe0149ee8e\") " pod="kuadrant-system/authorino-c55db47d4-tt55r" Jul 01 22:30:09.568148 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:09.568098 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c55db47d4-tt55r" Jul 01 22:30:09.704513 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:09.704427 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-c55db47d4-tt55r"] Jul 01 22:30:09.707022 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:30:09.706974 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16f6c220_69a7_4b82_bb85_6dbe0149ee8e.slice/crio-06e7c411d9a7ff9d2010059f819ebd863ff42600d625d6cfd4e6feb87b7c8a66 WatchSource:0}: Error finding container 06e7c411d9a7ff9d2010059f819ebd863ff42600d625d6cfd4e6feb87b7c8a66: Status 404 returned error can't find the container with id 06e7c411d9a7ff9d2010059f819ebd863ff42600d625d6cfd4e6feb87b7c8a66 Jul 01 22:30:10.261490 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:10.261453 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f7d4cc9fb-mhbbz" event={"ID":"8a29163d-d33d-40a4-9937-8bcbe5bcfdbc","Type":"ContainerStarted","Data":"79312dd71abd11474a8105e2c5c34452bb300ca2ef9bf98b3fedbe75c94f3107"} Jul 01 22:30:10.261731 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:10.261498 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-6f7d4cc9fb-mhbbz" podUID="8a29163d-d33d-40a4-9937-8bcbe5bcfdbc" containerName="authorino" containerID="cri-o://79312dd71abd11474a8105e2c5c34452bb300ca2ef9bf98b3fedbe75c94f3107" gracePeriod=30 Jul 01 22:30:10.263162 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:10.263137 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c55db47d4-tt55r" event={"ID":"16f6c220-69a7-4b82-bb85-6dbe0149ee8e","Type":"ContainerStarted","Data":"918569331ad1a2ce1d97329f5f950b9a499a6e02c7246ccbca1045df7a7ea057"} Jul 01 22:30:10.263162 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:10.263166 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-p5kvw" Jul 01 22:30:10.263162 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:10.263173 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c55db47d4-tt55r" event={"ID":"16f6c220-69a7-4b82-bb85-6dbe0149ee8e","Type":"ContainerStarted","Data":"06e7c411d9a7ff9d2010059f819ebd863ff42600d625d6cfd4e6feb87b7c8a66"} Jul 01 22:30:10.280969 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:10.280922 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6f7d4cc9fb-mhbbz" podStartSLOduration=1.773264532 podStartE2EDuration="2.280905588s" totalImagesPullingTime="507.641056ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:30:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:30:10.280275671 +0000 UTC m=+551.027716258" watchObservedRunningTime="2026-07-01 22:30:10.280905588 +0000 UTC m=+551.028346179" Jul 01 22:30:10.305070 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:10.305045 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-p5kvw"] Jul 01 22:30:10.308001 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:10.307978 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-p5kvw"] Jul 01 22:30:10.323329 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:10.323289 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-c55db47d4-tt55r" podStartSLOduration=0.915052722 podStartE2EDuration="1.323276418s" totalImagesPullingTime="408.223696ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:30:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:30:10.321722903 +0000 UTC m=+551.069163494" watchObservedRunningTime="2026-07-01 22:30:10.323276418 +0000 UTC m=+551.070717008" Jul 01 22:30:10.347020 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:10.346988 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b6psw"] Jul 01 22:30:10.347166 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:10.347147 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-b6psw" podUID="85c2664f-f419-4ea7-a4d7-e86970951839" containerName="authorino" containerID="cri-o://9e5314b4c91507315e2d41b5dc53abc2dad9202a23f62872b10ef86df682567e" gracePeriod=30 Jul 01 22:30:10.601904 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:10.601876 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f7d4cc9fb-mhbbz" Jul 01 22:30:10.609372 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:10.609350 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-b6psw" Jul 01 22:30:10.665586 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:10.665546 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgx8s\" (UniqueName: \"kubernetes.io/projected/8a29163d-d33d-40a4-9937-8bcbe5bcfdbc-kube-api-access-dgx8s\") pod \"8a29163d-d33d-40a4-9937-8bcbe5bcfdbc\" (UID: \"8a29163d-d33d-40a4-9937-8bcbe5bcfdbc\") " Jul 01 22:30:10.665710 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:10.665597 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d25c8\" (UniqueName: \"kubernetes.io/projected/85c2664f-f419-4ea7-a4d7-e86970951839-kube-api-access-d25c8\") pod \"85c2664f-f419-4ea7-a4d7-e86970951839\" (UID: \"85c2664f-f419-4ea7-a4d7-e86970951839\") " Jul 01 22:30:10.667666 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:10.667635 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a29163d-d33d-40a4-9937-8bcbe5bcfdbc-kube-api-access-dgx8s" (OuterVolumeSpecName: "kube-api-access-dgx8s") pod "8a29163d-d33d-40a4-9937-8bcbe5bcfdbc" (UID: "8a29163d-d33d-40a4-9937-8bcbe5bcfdbc"). InnerVolumeSpecName "kube-api-access-dgx8s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 22:30:10.667761 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:10.667666 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85c2664f-f419-4ea7-a4d7-e86970951839-kube-api-access-d25c8" (OuterVolumeSpecName: "kube-api-access-d25c8") pod "85c2664f-f419-4ea7-a4d7-e86970951839" (UID: "85c2664f-f419-4ea7-a4d7-e86970951839"). InnerVolumeSpecName "kube-api-access-d25c8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 22:30:10.766677 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:10.766621 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgx8s\" (UniqueName: \"kubernetes.io/projected/8a29163d-d33d-40a4-9937-8bcbe5bcfdbc-kube-api-access-dgx8s\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:30:10.766677 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:10.766643 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d25c8\" (UniqueName: \"kubernetes.io/projected/85c2664f-f419-4ea7-a4d7-e86970951839-kube-api-access-d25c8\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:30:11.269243 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:11.269209 2580 generic.go:356] "Generic (PLEG): container finished" podID="85c2664f-f419-4ea7-a4d7-e86970951839" containerID="9e5314b4c91507315e2d41b5dc53abc2dad9202a23f62872b10ef86df682567e" exitCode=0 Jul 01 22:30:11.269419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:11.269253 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-b6psw" Jul 01 22:30:11.269419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:11.269293 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-b6psw" event={"ID":"85c2664f-f419-4ea7-a4d7-e86970951839","Type":"ContainerDied","Data":"9e5314b4c91507315e2d41b5dc53abc2dad9202a23f62872b10ef86df682567e"} Jul 01 22:30:11.269419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:11.269339 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-b6psw" event={"ID":"85c2664f-f419-4ea7-a4d7-e86970951839","Type":"ContainerDied","Data":"f22c2ce9a44e58c5a2fcdffc46f3688b9f446987646c2acbf702659966319687"} Jul 01 22:30:11.269419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:11.269363 2580 scope.go:117] "RemoveContainer" containerID="9e5314b4c91507315e2d41b5dc53abc2dad9202a23f62872b10ef86df682567e" Jul 01 22:30:11.270781 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:11.270687 2580 generic.go:356] "Generic (PLEG): container finished" podID="8a29163d-d33d-40a4-9937-8bcbe5bcfdbc" containerID="79312dd71abd11474a8105e2c5c34452bb300ca2ef9bf98b3fedbe75c94f3107" exitCode=0 Jul 01 22:30:11.270781 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:11.270741 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f7d4cc9fb-mhbbz" Jul 01 22:30:11.270781 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:11.270770 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f7d4cc9fb-mhbbz" event={"ID":"8a29163d-d33d-40a4-9937-8bcbe5bcfdbc","Type":"ContainerDied","Data":"79312dd71abd11474a8105e2c5c34452bb300ca2ef9bf98b3fedbe75c94f3107"} Jul 01 22:30:11.270966 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:11.270813 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f7d4cc9fb-mhbbz" event={"ID":"8a29163d-d33d-40a4-9937-8bcbe5bcfdbc","Type":"ContainerDied","Data":"4486c7fe2ff1ab686079071d53c594d2bec4c78ce4a8252f36ebf72d87cd551c"} Jul 01 22:30:11.278669 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:11.278503 2580 scope.go:117] "RemoveContainer" containerID="9e5314b4c91507315e2d41b5dc53abc2dad9202a23f62872b10ef86df682567e" Jul 01 22:30:11.278903 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:30:11.278802 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e5314b4c91507315e2d41b5dc53abc2dad9202a23f62872b10ef86df682567e\": container with ID starting with 9e5314b4c91507315e2d41b5dc53abc2dad9202a23f62872b10ef86df682567e not found: ID does not exist" containerID="9e5314b4c91507315e2d41b5dc53abc2dad9202a23f62872b10ef86df682567e" Jul 01 22:30:11.278903 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:11.278839 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e5314b4c91507315e2d41b5dc53abc2dad9202a23f62872b10ef86df682567e"} err="failed to get container status \"9e5314b4c91507315e2d41b5dc53abc2dad9202a23f62872b10ef86df682567e\": rpc error: code = NotFound desc = could not find container \"9e5314b4c91507315e2d41b5dc53abc2dad9202a23f62872b10ef86df682567e\": container with ID starting with 9e5314b4c91507315e2d41b5dc53abc2dad9202a23f62872b10ef86df682567e not found: ID does not exist" Jul 01 22:30:11.278903 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:11.278861 2580 scope.go:117] "RemoveContainer" containerID="79312dd71abd11474a8105e2c5c34452bb300ca2ef9bf98b3fedbe75c94f3107" Jul 01 22:30:11.287238 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:11.287222 2580 scope.go:117] "RemoveContainer" containerID="79312dd71abd11474a8105e2c5c34452bb300ca2ef9bf98b3fedbe75c94f3107" Jul 01 22:30:11.287468 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:30:11.287451 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79312dd71abd11474a8105e2c5c34452bb300ca2ef9bf98b3fedbe75c94f3107\": container with ID starting with 79312dd71abd11474a8105e2c5c34452bb300ca2ef9bf98b3fedbe75c94f3107 not found: ID does not exist" containerID="79312dd71abd11474a8105e2c5c34452bb300ca2ef9bf98b3fedbe75c94f3107" Jul 01 22:30:11.287512 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:11.287475 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79312dd71abd11474a8105e2c5c34452bb300ca2ef9bf98b3fedbe75c94f3107"} err="failed to get container status \"79312dd71abd11474a8105e2c5c34452bb300ca2ef9bf98b3fedbe75c94f3107\": rpc error: code = NotFound desc = could not find container \"79312dd71abd11474a8105e2c5c34452bb300ca2ef9bf98b3fedbe75c94f3107\": container with ID starting with 79312dd71abd11474a8105e2c5c34452bb300ca2ef9bf98b3fedbe75c94f3107 not found: ID does not exist" Jul 01 22:30:11.294264 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:11.294243 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f7d4cc9fb-mhbbz"] Jul 01 22:30:11.299094 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:11.299075 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f7d4cc9fb-mhbbz"] Jul 01 22:30:11.310133 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:11.310111 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b6psw"] Jul 01 22:30:11.313169 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:11.313149 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-b6psw"] Jul 01 22:30:11.748042 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:11.748011 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85c2664f-f419-4ea7-a4d7-e86970951839" path="/var/lib/kubelet/pods/85c2664f-f419-4ea7-a4d7-e86970951839/volumes" Jul 01 22:30:11.748373 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:11.748308 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a29163d-d33d-40a4-9937-8bcbe5bcfdbc" path="/var/lib/kubelet/pods/8a29163d-d33d-40a4-9937-8bcbe5bcfdbc/volumes" Jul 01 22:30:11.748609 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:11.748596 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a43ef56a-9c8b-4c5d-9d5b-fa0d7b2101bd" path="/var/lib/kubelet/pods/a43ef56a-9c8b-4c5d-9d5b-fa0d7b2101bd/volumes" Jul 01 22:30:50.024616 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:50.024579 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-7mngm"] Jul 01 22:30:50.025057 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:50.024922 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85c2664f-f419-4ea7-a4d7-e86970951839" containerName="authorino" Jul 01 22:30:50.025057 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:50.024932 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="85c2664f-f419-4ea7-a4d7-e86970951839" containerName="authorino" Jul 01 22:30:50.025057 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:50.024942 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a29163d-d33d-40a4-9937-8bcbe5bcfdbc" containerName="authorino" Jul 01 22:30:50.025057 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:50.024948 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a29163d-d33d-40a4-9937-8bcbe5bcfdbc" containerName="authorino" Jul 01 22:30:50.025057 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:50.025008 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="85c2664f-f419-4ea7-a4d7-e86970951839" containerName="authorino" Jul 01 22:30:50.025057 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:50.025018 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a29163d-d33d-40a4-9937-8bcbe5bcfdbc" containerName="authorino" Jul 01 22:30:50.028209 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:50.028190 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" Jul 01 22:30:50.030901 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:50.030871 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 01 22:30:50.031015 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:50.030964 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-w2gd8\"" Jul 01 22:30:50.038115 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:50.038095 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-7mngm"] Jul 01 22:30:50.160977 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:50.160943 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzhdc\" (UniqueName: \"kubernetes.io/projected/579297ec-d799-4cd5-8db9-11e27a5f2f28-kube-api-access-mzhdc\") pod \"payload-processing-7bbcb654f4-7mngm\" (UID: \"579297ec-d799-4cd5-8db9-11e27a5f2f28\") " pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" Jul 01 22:30:50.161179 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:50.161041 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/579297ec-d799-4cd5-8db9-11e27a5f2f28-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-7mngm\" (UID: \"579297ec-d799-4cd5-8db9-11e27a5f2f28\") " pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" Jul 01 22:30:50.261740 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:50.261706 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzhdc\" (UniqueName: \"kubernetes.io/projected/579297ec-d799-4cd5-8db9-11e27a5f2f28-kube-api-access-mzhdc\") pod \"payload-processing-7bbcb654f4-7mngm\" (UID: \"579297ec-d799-4cd5-8db9-11e27a5f2f28\") " pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" Jul 01 22:30:50.261898 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:50.261746 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/579297ec-d799-4cd5-8db9-11e27a5f2f28-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-7mngm\" (UID: \"579297ec-d799-4cd5-8db9-11e27a5f2f28\") " pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" Jul 01 22:30:50.262419 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:50.262401 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/579297ec-d799-4cd5-8db9-11e27a5f2f28-plugins-config-volume\") pod \"payload-processing-7bbcb654f4-7mngm\" (UID: \"579297ec-d799-4cd5-8db9-11e27a5f2f28\") " pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" Jul 01 22:30:50.272910 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:50.272885 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzhdc\" (UniqueName: \"kubernetes.io/projected/579297ec-d799-4cd5-8db9-11e27a5f2f28-kube-api-access-mzhdc\") pod \"payload-processing-7bbcb654f4-7mngm\" (UID: \"579297ec-d799-4cd5-8db9-11e27a5f2f28\") " pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" Jul 01 22:30:50.338941 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:50.338873 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" Jul 01 22:30:50.468004 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:50.467978 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bbcb654f4-7mngm"] Jul 01 22:30:50.470009 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:30:50.469985 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod579297ec_d799_4cd5_8db9_11e27a5f2f28.slice/crio-42ae3958b30e9f9231085f5e6a24fbeabe32edf3a81d921683f4449d983ef571 WatchSource:0}: Error finding container 42ae3958b30e9f9231085f5e6a24fbeabe32edf3a81d921683f4449d983ef571: Status 404 returned error can't find the container with id 42ae3958b30e9f9231085f5e6a24fbeabe32edf3a81d921683f4449d983ef571 Jul 01 22:30:51.473940 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:51.473900 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" event={"ID":"579297ec-d799-4cd5-8db9-11e27a5f2f28","Type":"ContainerStarted","Data":"42ae3958b30e9f9231085f5e6a24fbeabe32edf3a81d921683f4449d983ef571"} Jul 01 22:30:53.486720 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:53.486624 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" event={"ID":"579297ec-d799-4cd5-8db9-11e27a5f2f28","Type":"ContainerStarted","Data":"b84d83ad5f9c1669f6adfcb9803ef7a607223a4e4f6590206e83ade217920221"} Jul 01 22:30:53.487126 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:53.486807 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" Jul 01 22:30:53.507916 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:53.507863 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" podStartSLOduration=0.787796789 podStartE2EDuration="3.50785016s" totalImagesPullingTime="2.720053371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:30:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:30:53.506215553 +0000 UTC m=+594.253656143" watchObservedRunningTime="2026-07-01 22:30:53.50785016 +0000 UTC m=+594.255290750" Jul 01 22:30:58.513055 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:58.513023 2580 generic.go:356] "Generic (PLEG): container finished" podID="d7640cec-b7f8-4db1-95ab-f929ffac82c8" containerID="3d5c6b2ddc15cb8190726ac133e98947d74713362233af287bf77010237ff466" exitCode=0 Jul 01 22:30:58.513449 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:58.513099 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" event={"ID":"d7640cec-b7f8-4db1-95ab-f929ffac82c8","Type":"ContainerDied","Data":"3d5c6b2ddc15cb8190726ac133e98947d74713362233af287bf77010237ff466"} Jul 01 22:30:58.513449 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:58.513429 2580 scope.go:117] "RemoveContainer" containerID="3d5c6b2ddc15cb8190726ac133e98947d74713362233af287bf77010237ff466" Jul 01 22:30:58.514523 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:58.514494 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 01 22:30:58.514604 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:58.514591 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 01 22:30:58.514656 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:58.514640 2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 01 22:30:59.522147 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:59.522117 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" event={"ID":"d7640cec-b7f8-4db1-95ab-f929ffac82c8","Type":"ContainerStarted","Data":"8a787646ce5fafb93be5569f1f96612bd86749bcb7117eab620c8292f4f0e392"} Jul 01 22:30:59.568051 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:59.568016 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 22:30:59.568231 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:59.568205 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="01484b49-950f-4ef4-87bd-a50d59539c50" containerName="keycloak" containerID="cri-o://521953481cba94d964fe8ddc95abb4fbcfa543eb4701a25c285c1acb4b97e6f8" gracePeriod=30 Jul 01 22:30:59.717072 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:59.717041 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:30:59.720181 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:59.720157 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8xh6_18708697-4b5c-4f52-9878-e369f8aa9eb2/console-operator/2.log" Jul 01 22:30:59.721079 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:30:59.721056 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8xh6_18708697-4b5c-4f52-9878-e369f8aa9eb2/console-operator/2.log" Jul 01 22:31:00.348808 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:00.348770 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:31:00.350394 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:00.350352 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-xr74c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 01 22:31:00.350565 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:00.350447 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" podUID="d7640cec-b7f8-4db1-95ab-f929ffac82c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 01 22:31:01.261776 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.261714 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 01 22:31:01.262989 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.262970 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bfrz\" (UniqueName: \"kubernetes.io/projected/01484b49-950f-4ef4-87bd-a50d59539c50-kube-api-access-9bfrz\") pod \"01484b49-950f-4ef4-87bd-a50d59539c50\" (UID: \"01484b49-950f-4ef4-87bd-a50d59539c50\") " Jul 01 22:31:01.264983 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.264961 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01484b49-950f-4ef4-87bd-a50d59539c50-kube-api-access-9bfrz" (OuterVolumeSpecName: "kube-api-access-9bfrz") pod "01484b49-950f-4ef4-87bd-a50d59539c50" (UID: "01484b49-950f-4ef4-87bd-a50d59539c50"). InnerVolumeSpecName "kube-api-access-9bfrz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 22:31:01.350574 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.350050 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-xr74c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 01 22:31:01.350574 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.350106 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" podUID="d7640cec-b7f8-4db1-95ab-f929ffac82c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 01 22:31:01.364146 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.364120 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bfrz\" (UniqueName: \"kubernetes.io/projected/01484b49-950f-4ef4-87bd-a50d59539c50-kube-api-access-9bfrz\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:31:01.534860 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.534770 2580 generic.go:356] "Generic (PLEG): container finished" podID="01484b49-950f-4ef4-87bd-a50d59539c50" containerID="521953481cba94d964fe8ddc95abb4fbcfa543eb4701a25c285c1acb4b97e6f8" exitCode=143 Jul 01 22:31:01.535146 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.534943 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"01484b49-950f-4ef4-87bd-a50d59539c50","Type":"ContainerDied","Data":"521953481cba94d964fe8ddc95abb4fbcfa543eb4701a25c285c1acb4b97e6f8"} Jul 01 22:31:01.535230 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.535185 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"01484b49-950f-4ef4-87bd-a50d59539c50","Type":"ContainerDied","Data":"e50023ae7db573be4055ce8414c390bc6e28457d4c9070c6cf2c2c2cde1708e4"} Jul 01 22:31:01.535230 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.535226 2580 scope.go:117] "RemoveContainer" containerID="521953481cba94d964fe8ddc95abb4fbcfa543eb4701a25c285c1acb4b97e6f8" Jul 01 22:31:01.535352 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.535035 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 01 22:31:01.560125 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.560107 2580 scope.go:117] "RemoveContainer" containerID="521953481cba94d964fe8ddc95abb4fbcfa543eb4701a25c285c1acb4b97e6f8" Jul 01 22:31:01.560395 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:31:01.560377 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"521953481cba94d964fe8ddc95abb4fbcfa543eb4701a25c285c1acb4b97e6f8\": container with ID starting with 521953481cba94d964fe8ddc95abb4fbcfa543eb4701a25c285c1acb4b97e6f8 not found: ID does not exist" containerID="521953481cba94d964fe8ddc95abb4fbcfa543eb4701a25c285c1acb4b97e6f8" Jul 01 22:31:01.560446 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.560401 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"521953481cba94d964fe8ddc95abb4fbcfa543eb4701a25c285c1acb4b97e6f8"} err="failed to get container status \"521953481cba94d964fe8ddc95abb4fbcfa543eb4701a25c285c1acb4b97e6f8\": rpc error: code = NotFound desc = could not find container \"521953481cba94d964fe8ddc95abb4fbcfa543eb4701a25c285c1acb4b97e6f8\": container with ID starting with 521953481cba94d964fe8ddc95abb4fbcfa543eb4701a25c285c1acb4b97e6f8 not found: ID does not exist" Jul 01 22:31:01.580829 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.580807 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 22:31:01.585747 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.585729 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 22:31:01.617634 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.617614 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 22:31:01.617966 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.617953 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01484b49-950f-4ef4-87bd-a50d59539c50" containerName="keycloak" Jul 01 22:31:01.617966 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.617968 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="01484b49-950f-4ef4-87bd-a50d59539c50" containerName="keycloak" Jul 01 22:31:01.618053 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.618041 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="01484b49-950f-4ef4-87bd-a50d59539c50" containerName="keycloak" Jul 01 22:31:01.623143 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.623128 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 01 22:31:01.631002 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.630966 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jul 01 22:31:01.631111 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.631082 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jul 01 22:31:01.631235 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.631213 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jul 01 22:31:01.632784 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.632758 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jul 01 22:31:01.632982 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.632962 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-2lhws\"" Jul 01 22:31:01.637220 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.637189 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 22:31:01.665961 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.665934 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/7f7e6eba-739c-4f15-965d-8f3ebd15589d-test-realms\") pod \"maas-keycloak-0\" (UID: \"7f7e6eba-739c-4f15-965d-8f3ebd15589d\") " pod="keycloak-system/maas-keycloak-0" Jul 01 22:31:01.666076 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.666000 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxjxg\" (UniqueName: \"kubernetes.io/projected/7f7e6eba-739c-4f15-965d-8f3ebd15589d-kube-api-access-gxjxg\") pod \"maas-keycloak-0\" (UID: \"7f7e6eba-739c-4f15-965d-8f3ebd15589d\") " pod="keycloak-system/maas-keycloak-0" Jul 01 22:31:01.750922 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.750888 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01484b49-950f-4ef4-87bd-a50d59539c50" path="/var/lib/kubelet/pods/01484b49-950f-4ef4-87bd-a50d59539c50/volumes" Jul 01 22:31:01.766983 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.766960 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxjxg\" (UniqueName: \"kubernetes.io/projected/7f7e6eba-739c-4f15-965d-8f3ebd15589d-kube-api-access-gxjxg\") pod \"maas-keycloak-0\" (UID: \"7f7e6eba-739c-4f15-965d-8f3ebd15589d\") " pod="keycloak-system/maas-keycloak-0" Jul 01 22:31:01.767110 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.767011 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/7f7e6eba-739c-4f15-965d-8f3ebd15589d-test-realms\") pod \"maas-keycloak-0\" (UID: \"7f7e6eba-739c-4f15-965d-8f3ebd15589d\") " pod="keycloak-system/maas-keycloak-0" Jul 01 22:31:01.767656 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.767638 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/7f7e6eba-739c-4f15-965d-8f3ebd15589d-test-realms\") pod \"maas-keycloak-0\" (UID: \"7f7e6eba-739c-4f15-965d-8f3ebd15589d\") " pod="keycloak-system/maas-keycloak-0" Jul 01 22:31:01.777157 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.777128 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxjxg\" (UniqueName: \"kubernetes.io/projected/7f7e6eba-739c-4f15-965d-8f3ebd15589d-kube-api-access-gxjxg\") pod \"maas-keycloak-0\" (UID: \"7f7e6eba-739c-4f15-965d-8f3ebd15589d\") " pod="keycloak-system/maas-keycloak-0" Jul 01 22:31:01.942037 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:01.942002 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 01 22:31:02.104103 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:02.104074 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 22:31:02.105945 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:31:02.105907 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f7e6eba_739c_4f15_965d_8f3ebd15589d.slice/crio-fbfce4d8f8625702dbeb1f7febd480bc0511d713e3707ffa4632e0b40b9f1b11 WatchSource:0}: Error finding container fbfce4d8f8625702dbeb1f7febd480bc0511d713e3707ffa4632e0b40b9f1b11: Status 404 returned error can't find the container with id fbfce4d8f8625702dbeb1f7febd480bc0511d713e3707ffa4632e0b40b9f1b11 Jul 01 22:31:02.349186 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:02.349149 2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-xr74c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 01 22:31:02.349753 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:02.349716 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" podUID="d7640cec-b7f8-4db1-95ab-f929ffac82c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 01 22:31:02.544731 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:02.544693 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"7f7e6eba-739c-4f15-965d-8f3ebd15589d","Type":"ContainerStarted","Data":"9b3abe12d47572ef0c89d8e21c7f6eb5b7101a1a939c9449b9c8830429bf9770"} Jul 01 22:31:02.544863 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:02.544793 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"7f7e6eba-739c-4f15-965d-8f3ebd15589d","Type":"ContainerStarted","Data":"fbfce4d8f8625702dbeb1f7febd480bc0511d713e3707ffa4632e0b40b9f1b11"} Jul 01 22:31:02.569288 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:02.569243 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.221471878 podStartE2EDuration="1.569230151s" totalImagesPullingTime="347.758273ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:31:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:31:02.569076114 +0000 UTC m=+603.316516705" watchObservedRunningTime="2026-07-01 22:31:02.569230151 +0000 UTC m=+603.316670738" Jul 01 22:31:02.944436 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:02.944401 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jul 01 22:31:02.944804 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:02.944776 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7f7e6eba-739c-4f15-965d-8f3ebd15589d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 01 22:31:03.353412 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:03.353381 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:31:03.354602 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:03.354582 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-xr74c" Jul 01 22:31:03.942876 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:03.942832 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7f7e6eba-739c-4f15-965d-8f3ebd15589d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 01 22:31:04.494035 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:04.494005 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" Jul 01 22:31:04.943536 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:04.943484 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7f7e6eba-739c-4f15-965d-8f3ebd15589d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 01 22:31:05.943331 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:05.943278 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7f7e6eba-739c-4f15-965d-8f3ebd15589d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 01 22:31:06.942970 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:06.942924 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7f7e6eba-739c-4f15-965d-8f3ebd15589d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 01 22:31:07.942843 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:07.942795 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7f7e6eba-739c-4f15-965d-8f3ebd15589d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 01 22:31:08.942722 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:08.942638 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7f7e6eba-739c-4f15-965d-8f3ebd15589d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 01 22:31:09.943329 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:09.943282 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7f7e6eba-739c-4f15-965d-8f3ebd15589d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 01 22:31:10.942840 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:10.942794 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7f7e6eba-739c-4f15-965d-8f3ebd15589d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 01 22:31:11.942476 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:11.942432 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jul 01 22:31:11.943036 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:11.942756 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7f7e6eba-739c-4f15-965d-8f3ebd15589d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 01 22:31:12.942742 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:12.942695 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7f7e6eba-739c-4f15-965d-8f3ebd15589d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 01 22:31:13.943386 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:13.943337 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7f7e6eba-739c-4f15-965d-8f3ebd15589d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 01 22:31:14.943114 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:14.943071 2580 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="7f7e6eba-739c-4f15-965d-8f3ebd15589d" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.45:9000/health/started\": dial tcp 10.132.0.45:9000: connect: connection refused" Jul 01 22:31:16.066988 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:16.066936 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jul 01 22:31:16.083977 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:16.083916 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="7f7e6eba-739c-4f15-965d-8f3ebd15589d" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 22:31:26.073216 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:26.073124 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jul 01 22:31:36.782694 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:36.782660 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5695859946-dctmr"] Jul 01 22:31:36.786453 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:36.786430 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5695859946-dctmr" Jul 01 22:31:36.789586 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:36.789538 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jul 01 22:31:36.798216 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:36.798190 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5695859946-dctmr"] Jul 01 22:31:36.892423 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:36.892393 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/532a2b2c-e23f-43f6-83e5-64295dd0562d-tls-cert\") pod \"authorino-5695859946-dctmr\" (UID: \"532a2b2c-e23f-43f6-83e5-64295dd0562d\") " pod="kuadrant-system/authorino-5695859946-dctmr" Jul 01 22:31:36.892538 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:36.892428 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/532a2b2c-e23f-43f6-83e5-64295dd0562d-oidc-ca\") pod \"authorino-5695859946-dctmr\" (UID: \"532a2b2c-e23f-43f6-83e5-64295dd0562d\") " pod="kuadrant-system/authorino-5695859946-dctmr" Jul 01 22:31:36.892538 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:36.892491 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9g7p\" (UniqueName: \"kubernetes.io/projected/532a2b2c-e23f-43f6-83e5-64295dd0562d-kube-api-access-h9g7p\") pod \"authorino-5695859946-dctmr\" (UID: \"532a2b2c-e23f-43f6-83e5-64295dd0562d\") " pod="kuadrant-system/authorino-5695859946-dctmr" Jul 01 22:31:36.993666 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:36.993640 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/532a2b2c-e23f-43f6-83e5-64295dd0562d-tls-cert\") pod \"authorino-5695859946-dctmr\" (UID: \"532a2b2c-e23f-43f6-83e5-64295dd0562d\") " pod="kuadrant-system/authorino-5695859946-dctmr" Jul 01 22:31:36.993767 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:36.993676 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/532a2b2c-e23f-43f6-83e5-64295dd0562d-oidc-ca\") pod \"authorino-5695859946-dctmr\" (UID: \"532a2b2c-e23f-43f6-83e5-64295dd0562d\") " pod="kuadrant-system/authorino-5695859946-dctmr" Jul 01 22:31:36.993767 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:36.993697 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9g7p\" (UniqueName: \"kubernetes.io/projected/532a2b2c-e23f-43f6-83e5-64295dd0562d-kube-api-access-h9g7p\") pod \"authorino-5695859946-dctmr\" (UID: \"532a2b2c-e23f-43f6-83e5-64295dd0562d\") " pod="kuadrant-system/authorino-5695859946-dctmr" Jul 01 22:31:36.994239 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:36.994219 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/532a2b2c-e23f-43f6-83e5-64295dd0562d-oidc-ca\") pod \"authorino-5695859946-dctmr\" (UID: \"532a2b2c-e23f-43f6-83e5-64295dd0562d\") " pod="kuadrant-system/authorino-5695859946-dctmr" Jul 01 22:31:36.996105 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:36.996077 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/532a2b2c-e23f-43f6-83e5-64295dd0562d-tls-cert\") pod \"authorino-5695859946-dctmr\" (UID: \"532a2b2c-e23f-43f6-83e5-64295dd0562d\") " pod="kuadrant-system/authorino-5695859946-dctmr" Jul 01 22:31:37.002214 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:37.002190 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9g7p\" (UniqueName: \"kubernetes.io/projected/532a2b2c-e23f-43f6-83e5-64295dd0562d-kube-api-access-h9g7p\") pod \"authorino-5695859946-dctmr\" (UID: \"532a2b2c-e23f-43f6-83e5-64295dd0562d\") " pod="kuadrant-system/authorino-5695859946-dctmr" Jul 01 22:31:37.095653 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:37.095631 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5695859946-dctmr" Jul 01 22:31:37.227801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:37.227772 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5695859946-dctmr"] Jul 01 22:31:37.231023 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:31:37.230987 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod532a2b2c_e23f_43f6_83e5_64295dd0562d.slice/crio-1fa108147e76d98f5dffd33120cc020e31b2c4ea62a2bef3f38ec5c1e678b0e2 WatchSource:0}: Error finding container 1fa108147e76d98f5dffd33120cc020e31b2c4ea62a2bef3f38ec5c1e678b0e2: Status 404 returned error can't find the container with id 1fa108147e76d98f5dffd33120cc020e31b2c4ea62a2bef3f38ec5c1e678b0e2 Jul 01 22:31:37.232267 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:37.232252 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 22:31:37.748574 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:37.748528 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5695859946-dctmr" event={"ID":"532a2b2c-e23f-43f6-83e5-64295dd0562d","Type":"ContainerStarted","Data":"1fa108147e76d98f5dffd33120cc020e31b2c4ea62a2bef3f38ec5c1e678b0e2"} Jul 01 22:31:38.752976 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:38.752939 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5695859946-dctmr" event={"ID":"532a2b2c-e23f-43f6-83e5-64295dd0562d","Type":"ContainerStarted","Data":"a71f66852e116d095a85ce5e89d191fe7790cc4e76f47dc6df806456ae6705ff"} Jul 01 22:31:38.772240 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:38.772155 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5695859946-dctmr" podStartSLOduration=2.230016175 podStartE2EDuration="2.772138709s" totalImagesPullingTime="542.122534ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:31:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:31:38.77192138 +0000 UTC m=+639.519361971" watchObservedRunningTime="2026-07-01 22:31:38.772138709 +0000 UTC m=+639.519579300" Jul 01 22:31:38.804256 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:38.804226 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c55db47d4-tt55r"] Jul 01 22:31:38.804450 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:38.804425 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-c55db47d4-tt55r" podUID="16f6c220-69a7-4b82-bb85-6dbe0149ee8e" containerName="authorino" containerID="cri-o://918569331ad1a2ce1d97329f5f950b9a499a6e02c7246ccbca1045df7a7ea057" gracePeriod=30 Jul 01 22:31:39.077988 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:39.077968 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c55db47d4-tt55r" Jul 01 22:31:39.212519 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:39.212492 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8c2v\" (UniqueName: \"kubernetes.io/projected/16f6c220-69a7-4b82-bb85-6dbe0149ee8e-kube-api-access-g8c2v\") pod \"16f6c220-69a7-4b82-bb85-6dbe0149ee8e\" (UID: \"16f6c220-69a7-4b82-bb85-6dbe0149ee8e\") " Jul 01 22:31:39.212675 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:39.212582 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/16f6c220-69a7-4b82-bb85-6dbe0149ee8e-tls-cert\") pod \"16f6c220-69a7-4b82-bb85-6dbe0149ee8e\" (UID: \"16f6c220-69a7-4b82-bb85-6dbe0149ee8e\") " Jul 01 22:31:39.214535 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:39.214506 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16f6c220-69a7-4b82-bb85-6dbe0149ee8e-kube-api-access-g8c2v" (OuterVolumeSpecName: "kube-api-access-g8c2v") pod "16f6c220-69a7-4b82-bb85-6dbe0149ee8e" (UID: "16f6c220-69a7-4b82-bb85-6dbe0149ee8e"). InnerVolumeSpecName "kube-api-access-g8c2v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 22:31:39.222904 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:39.222845 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16f6c220-69a7-4b82-bb85-6dbe0149ee8e-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "16f6c220-69a7-4b82-bb85-6dbe0149ee8e" (UID: "16f6c220-69a7-4b82-bb85-6dbe0149ee8e"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 22:31:39.313278 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:39.313250 2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/16f6c220-69a7-4b82-bb85-6dbe0149ee8e-tls-cert\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:31:39.313367 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:39.313278 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8c2v\" (UniqueName: \"kubernetes.io/projected/16f6c220-69a7-4b82-bb85-6dbe0149ee8e-kube-api-access-g8c2v\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:31:39.758668 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:39.758640 2580 generic.go:356] "Generic (PLEG): container finished" podID="16f6c220-69a7-4b82-bb85-6dbe0149ee8e" containerID="918569331ad1a2ce1d97329f5f950b9a499a6e02c7246ccbca1045df7a7ea057" exitCode=0 Jul 01 22:31:39.759063 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:39.758686 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-c55db47d4-tt55r" Jul 01 22:31:39.759063 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:39.758736 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c55db47d4-tt55r" event={"ID":"16f6c220-69a7-4b82-bb85-6dbe0149ee8e","Type":"ContainerDied","Data":"918569331ad1a2ce1d97329f5f950b9a499a6e02c7246ccbca1045df7a7ea057"} Jul 01 22:31:39.759063 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:39.758784 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-c55db47d4-tt55r" event={"ID":"16f6c220-69a7-4b82-bb85-6dbe0149ee8e","Type":"ContainerDied","Data":"06e7c411d9a7ff9d2010059f819ebd863ff42600d625d6cfd4e6feb87b7c8a66"} Jul 01 22:31:39.759063 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:39.758808 2580 scope.go:117] "RemoveContainer" containerID="918569331ad1a2ce1d97329f5f950b9a499a6e02c7246ccbca1045df7a7ea057" Jul 01 22:31:39.767105 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:39.767085 2580 scope.go:117] "RemoveContainer" containerID="918569331ad1a2ce1d97329f5f950b9a499a6e02c7246ccbca1045df7a7ea057" Jul 01 22:31:39.767344 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:31:39.767327 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"918569331ad1a2ce1d97329f5f950b9a499a6e02c7246ccbca1045df7a7ea057\": container with ID starting with 918569331ad1a2ce1d97329f5f950b9a499a6e02c7246ccbca1045df7a7ea057 not found: ID does not exist" containerID="918569331ad1a2ce1d97329f5f950b9a499a6e02c7246ccbca1045df7a7ea057" Jul 01 22:31:39.767387 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:39.767351 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"918569331ad1a2ce1d97329f5f950b9a499a6e02c7246ccbca1045df7a7ea057"} err="failed to get container status \"918569331ad1a2ce1d97329f5f950b9a499a6e02c7246ccbca1045df7a7ea057\": rpc error: code = NotFound desc = could not find container \"918569331ad1a2ce1d97329f5f950b9a499a6e02c7246ccbca1045df7a7ea057\": container with ID starting with 918569331ad1a2ce1d97329f5f950b9a499a6e02c7246ccbca1045df7a7ea057 not found: ID does not exist" Jul 01 22:31:39.775101 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:39.775078 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-c55db47d4-tt55r"] Jul 01 22:31:39.777298 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:39.777279 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-c55db47d4-tt55r"] Jul 01 22:31:41.747476 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:41.747443 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16f6c220-69a7-4b82-bb85-6dbe0149ee8e" path="/var/lib/kubelet/pods/16f6c220-69a7-4b82-bb85-6dbe0149ee8e/volumes" Jul 01 22:31:53.379332 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.379299 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg"] Jul 01 22:31:53.379701 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.379681 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16f6c220-69a7-4b82-bb85-6dbe0149ee8e" containerName="authorino" Jul 01 22:31:53.379701 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.379693 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="16f6c220-69a7-4b82-bb85-6dbe0149ee8e" containerName="authorino" Jul 01 22:31:53.379770 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.379758 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="16f6c220-69a7-4b82-bb85-6dbe0149ee8e" containerName="authorino" Jul 01 22:31:53.384131 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.384109 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:31:53.387445 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.387422 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-f7bz4\"" Jul 01 22:31:53.387622 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.387602 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 01 22:31:53.387929 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.387912 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 01 22:31:53.387998 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.387985 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 01 22:31:53.395962 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.395943 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg"] Jul 01 22:31:53.434624 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.434602 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0758543e-6104-413b-b620-f49699af55e9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg\" (UID: \"0758543e-6104-413b-b620-f49699af55e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:31:53.434734 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.434635 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0758543e-6104-413b-b620-f49699af55e9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg\" (UID: \"0758543e-6104-413b-b620-f49699af55e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:31:53.434734 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.434659 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0758543e-6104-413b-b620-f49699af55e9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg\" (UID: \"0758543e-6104-413b-b620-f49699af55e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:31:53.434734 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.434689 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t96ws\" (UniqueName: \"kubernetes.io/projected/0758543e-6104-413b-b620-f49699af55e9-kube-api-access-t96ws\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg\" (UID: \"0758543e-6104-413b-b620-f49699af55e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:31:53.434858 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.434743 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0758543e-6104-413b-b620-f49699af55e9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg\" (UID: \"0758543e-6104-413b-b620-f49699af55e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:31:53.434858 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.434819 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0758543e-6104-413b-b620-f49699af55e9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg\" (UID: \"0758543e-6104-413b-b620-f49699af55e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:31:53.536167 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.536133 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0758543e-6104-413b-b620-f49699af55e9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg\" (UID: \"0758543e-6104-413b-b620-f49699af55e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:31:53.536307 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.536176 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t96ws\" (UniqueName: \"kubernetes.io/projected/0758543e-6104-413b-b620-f49699af55e9-kube-api-access-t96ws\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg\" (UID: \"0758543e-6104-413b-b620-f49699af55e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:31:53.536307 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.536201 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0758543e-6104-413b-b620-f49699af55e9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg\" (UID: \"0758543e-6104-413b-b620-f49699af55e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:31:53.536307 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.536276 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0758543e-6104-413b-b620-f49699af55e9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg\" (UID: \"0758543e-6104-413b-b620-f49699af55e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:31:53.536421 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.536308 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0758543e-6104-413b-b620-f49699af55e9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg\" (UID: \"0758543e-6104-413b-b620-f49699af55e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:31:53.536421 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.536337 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0758543e-6104-413b-b620-f49699af55e9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg\" (UID: \"0758543e-6104-413b-b620-f49699af55e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:31:53.536733 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.536707 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0758543e-6104-413b-b620-f49699af55e9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg\" (UID: \"0758543e-6104-413b-b620-f49699af55e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:31:53.536899 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.536751 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0758543e-6104-413b-b620-f49699af55e9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg\" (UID: \"0758543e-6104-413b-b620-f49699af55e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:31:53.536899 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.536790 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0758543e-6104-413b-b620-f49699af55e9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg\" (UID: \"0758543e-6104-413b-b620-f49699af55e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:31:53.538587 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.538547 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0758543e-6104-413b-b620-f49699af55e9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg\" (UID: \"0758543e-6104-413b-b620-f49699af55e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:31:53.538676 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.538635 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0758543e-6104-413b-b620-f49699af55e9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg\" (UID: \"0758543e-6104-413b-b620-f49699af55e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:31:53.551088 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.551069 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t96ws\" (UniqueName: \"kubernetes.io/projected/0758543e-6104-413b-b620-f49699af55e9-kube-api-access-t96ws\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg\" (UID: \"0758543e-6104-413b-b620-f49699af55e9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:31:53.694949 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.694873 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:31:53.836817 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:53.836795 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg"] Jul 01 22:31:53.839114 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:31:53.839062 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0758543e_6104_413b_b620_f49699af55e9.slice/crio-133042585afa49bb2ebddac00bd15e50cdbb84a255e9a34fe6deef5bbe34acb8 WatchSource:0}: Error finding container 133042585afa49bb2ebddac00bd15e50cdbb84a255e9a34fe6deef5bbe34acb8: Status 404 returned error can't find the container with id 133042585afa49bb2ebddac00bd15e50cdbb84a255e9a34fe6deef5bbe34acb8 Jul 01 22:31:54.829596 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:54.829538 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" event={"ID":"0758543e-6104-413b-b620-f49699af55e9","Type":"ContainerStarted","Data":"133042585afa49bb2ebddac00bd15e50cdbb84a255e9a34fe6deef5bbe34acb8"} Jul 01 22:31:55.835983 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:55.835945 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" event={"ID":"0758543e-6104-413b-b620-f49699af55e9","Type":"ContainerStarted","Data":"1c4fde20a7e27cb4fb41e951f95bd7fc7fc4b06843cf7acd9d29d3f4765eb530"} Jul 01 22:31:55.857269 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:31:55.857221 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" podStartSLOduration=1.617790835 podStartE2EDuration="2.857206406s" totalImagesPullingTime="1.239415571s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:31:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:31:55.856231005 +0000 UTC m=+656.603671597" watchObservedRunningTime="2026-07-01 22:31:55.857206406 +0000 UTC m=+656.604646995" Jul 01 22:32:03.695444 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:03.695401 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:32:03.695847 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:03.695463 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:32:03.707699 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:03.707673 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:32:03.885516 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:03.885490 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg" Jul 01 22:32:22.489995 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.489966 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6"] Jul 01 22:32:22.494881 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.494861 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:32:22.497425 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.497409 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 01 22:32:22.507222 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.507198 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6"] Jul 01 22:32:22.573700 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.573669 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b37990b-0603-43e4-b97b-b1a765a362e4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6\" (UID: \"5b37990b-0603-43e4-b97b-b1a765a362e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:32:22.573837 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.573719 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b37990b-0603-43e4-b97b-b1a765a362e4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6\" (UID: \"5b37990b-0603-43e4-b97b-b1a765a362e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:32:22.573837 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.573747 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b37990b-0603-43e4-b97b-b1a765a362e4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6\" (UID: \"5b37990b-0603-43e4-b97b-b1a765a362e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:32:22.573837 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.573769 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b37990b-0603-43e4-b97b-b1a765a362e4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6\" (UID: \"5b37990b-0603-43e4-b97b-b1a765a362e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:32:22.574005 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.573837 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b37990b-0603-43e4-b97b-b1a765a362e4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6\" (UID: \"5b37990b-0603-43e4-b97b-b1a765a362e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:32:22.574005 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.573936 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45knq\" (UniqueName: \"kubernetes.io/projected/5b37990b-0603-43e4-b97b-b1a765a362e4-kube-api-access-45knq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6\" (UID: \"5b37990b-0603-43e4-b97b-b1a765a362e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:32:22.674694 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.674663 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b37990b-0603-43e4-b97b-b1a765a362e4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6\" (UID: \"5b37990b-0603-43e4-b97b-b1a765a362e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:32:22.674844 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.674716 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45knq\" (UniqueName: \"kubernetes.io/projected/5b37990b-0603-43e4-b97b-b1a765a362e4-kube-api-access-45knq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6\" (UID: \"5b37990b-0603-43e4-b97b-b1a765a362e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:32:22.674844 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.674753 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b37990b-0603-43e4-b97b-b1a765a362e4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6\" (UID: \"5b37990b-0603-43e4-b97b-b1a765a362e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:32:22.674844 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.674778 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b37990b-0603-43e4-b97b-b1a765a362e4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6\" (UID: \"5b37990b-0603-43e4-b97b-b1a765a362e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:32:22.674844 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.674794 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b37990b-0603-43e4-b97b-b1a765a362e4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6\" (UID: \"5b37990b-0603-43e4-b97b-b1a765a362e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:32:22.674844 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.674811 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b37990b-0603-43e4-b97b-b1a765a362e4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6\" (UID: \"5b37990b-0603-43e4-b97b-b1a765a362e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:32:22.675422 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.675319 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5b37990b-0603-43e4-b97b-b1a765a362e4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6\" (UID: \"5b37990b-0603-43e4-b97b-b1a765a362e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:32:22.675422 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.675391 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5b37990b-0603-43e4-b97b-b1a765a362e4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6\" (UID: \"5b37990b-0603-43e4-b97b-b1a765a362e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:32:22.675500 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.675431 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5b37990b-0603-43e4-b97b-b1a765a362e4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6\" (UID: \"5b37990b-0603-43e4-b97b-b1a765a362e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:32:22.677026 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.677003 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5b37990b-0603-43e4-b97b-b1a765a362e4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6\" (UID: \"5b37990b-0603-43e4-b97b-b1a765a362e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:32:22.677341 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.677324 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5b37990b-0603-43e4-b97b-b1a765a362e4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6\" (UID: \"5b37990b-0603-43e4-b97b-b1a765a362e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:32:22.683046 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.683025 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45knq\" (UniqueName: \"kubernetes.io/projected/5b37990b-0603-43e4-b97b-b1a765a362e4-kube-api-access-45knq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6\" (UID: \"5b37990b-0603-43e4-b97b-b1a765a362e4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:32:22.804983 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.804926 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:32:22.947522 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.947489 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6"] Jul 01 22:32:22.950160 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:32:22.950132 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b37990b_0603_43e4_b97b_b1a765a362e4.slice/crio-81ab53a16bbe7ab533edc1bb9f394235596792a75366787cfbc51fa62a487b1e WatchSource:0}: Error finding container 81ab53a16bbe7ab533edc1bb9f394235596792a75366787cfbc51fa62a487b1e: Status 404 returned error can't find the container with id 81ab53a16bbe7ab533edc1bb9f394235596792a75366787cfbc51fa62a487b1e Jul 01 22:32:22.964595 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:22.964547 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" event={"ID":"5b37990b-0603-43e4-b97b-b1a765a362e4","Type":"ContainerStarted","Data":"81ab53a16bbe7ab533edc1bb9f394235596792a75366787cfbc51fa62a487b1e"} Jul 01 22:32:23.970904 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:23.970865 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" event={"ID":"5b37990b-0603-43e4-b97b-b1a765a362e4","Type":"ContainerStarted","Data":"31cc89b142bd3989cb091945de9526ee54dc1a3c8f2f7d859177226ee670d927"} Jul 01 22:32:23.991891 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:23.991834 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" podStartSLOduration=1.828898398 podStartE2EDuration="1.991820641s" totalImagesPullingTime="162.922243ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:32:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:32:23.989323038 +0000 UTC m=+684.736763630" watchObservedRunningTime="2026-07-01 22:32:23.991820641 +0000 UTC m=+684.739261231" Jul 01 22:32:32.805091 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:32.805054 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:32:32.805769 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:32.805112 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:32:32.817228 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:32.817206 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:32:33.024739 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:32:33.024714 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6" Jul 01 22:33:54.447519 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:54.447483 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7d74f89dc9-nlgnc"] Jul 01 22:33:54.451050 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:54.451028 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d74f89dc9-nlgnc" Jul 01 22:33:54.464142 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:54.463327 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d74f89dc9-nlgnc"] Jul 01 22:33:54.510696 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:54.510667 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/09cfd04d-e167-46b3-b3a0-ab8de081f8a5-tls-cert\") pod \"authorino-7d74f89dc9-nlgnc\" (UID: \"09cfd04d-e167-46b3-b3a0-ab8de081f8a5\") " pod="kuadrant-system/authorino-7d74f89dc9-nlgnc" Jul 01 22:33:54.510845 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:54.510711 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vtpr\" (UniqueName: \"kubernetes.io/projected/09cfd04d-e167-46b3-b3a0-ab8de081f8a5-kube-api-access-4vtpr\") pod \"authorino-7d74f89dc9-nlgnc\" (UID: \"09cfd04d-e167-46b3-b3a0-ab8de081f8a5\") " pod="kuadrant-system/authorino-7d74f89dc9-nlgnc" Jul 01 22:33:54.510845 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:54.510758 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/09cfd04d-e167-46b3-b3a0-ab8de081f8a5-oidc-ca\") pod \"authorino-7d74f89dc9-nlgnc\" (UID: \"09cfd04d-e167-46b3-b3a0-ab8de081f8a5\") " pod="kuadrant-system/authorino-7d74f89dc9-nlgnc" Jul 01 22:33:54.611102 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:54.611072 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/09cfd04d-e167-46b3-b3a0-ab8de081f8a5-tls-cert\") pod \"authorino-7d74f89dc9-nlgnc\" (UID: \"09cfd04d-e167-46b3-b3a0-ab8de081f8a5\") " pod="kuadrant-system/authorino-7d74f89dc9-nlgnc" Jul 01 22:33:54.611256 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:54.611113 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vtpr\" (UniqueName: \"kubernetes.io/projected/09cfd04d-e167-46b3-b3a0-ab8de081f8a5-kube-api-access-4vtpr\") pod \"authorino-7d74f89dc9-nlgnc\" (UID: \"09cfd04d-e167-46b3-b3a0-ab8de081f8a5\") " pod="kuadrant-system/authorino-7d74f89dc9-nlgnc" Jul 01 22:33:54.611256 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:54.611147 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/09cfd04d-e167-46b3-b3a0-ab8de081f8a5-oidc-ca\") pod \"authorino-7d74f89dc9-nlgnc\" (UID: \"09cfd04d-e167-46b3-b3a0-ab8de081f8a5\") " pod="kuadrant-system/authorino-7d74f89dc9-nlgnc" Jul 01 22:33:54.611743 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:54.611724 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/09cfd04d-e167-46b3-b3a0-ab8de081f8a5-oidc-ca\") pod \"authorino-7d74f89dc9-nlgnc\" (UID: \"09cfd04d-e167-46b3-b3a0-ab8de081f8a5\") " pod="kuadrant-system/authorino-7d74f89dc9-nlgnc" Jul 01 22:33:54.613566 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:54.613513 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/09cfd04d-e167-46b3-b3a0-ab8de081f8a5-tls-cert\") pod \"authorino-7d74f89dc9-nlgnc\" (UID: \"09cfd04d-e167-46b3-b3a0-ab8de081f8a5\") " pod="kuadrant-system/authorino-7d74f89dc9-nlgnc" Jul 01 22:33:54.619437 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:54.619417 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vtpr\" (UniqueName: \"kubernetes.io/projected/09cfd04d-e167-46b3-b3a0-ab8de081f8a5-kube-api-access-4vtpr\") pod \"authorino-7d74f89dc9-nlgnc\" (UID: \"09cfd04d-e167-46b3-b3a0-ab8de081f8a5\") " pod="kuadrant-system/authorino-7d74f89dc9-nlgnc" Jul 01 22:33:54.760445 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:54.760385 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d74f89dc9-nlgnc" Jul 01 22:33:54.894377 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:54.894350 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d74f89dc9-nlgnc"] Jul 01 22:33:54.896534 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:33:54.896507 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09cfd04d_e167_46b3_b3a0_ab8de081f8a5.slice/crio-86f2e2b25625084128bae44e70c8d4a85378d76ab7a3868566ade607893148de WatchSource:0}: Error finding container 86f2e2b25625084128bae44e70c8d4a85378d76ab7a3868566ade607893148de: Status 404 returned error can't find the container with id 86f2e2b25625084128bae44e70c8d4a85378d76ab7a3868566ade607893148de Jul 01 22:33:55.416547 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:55.416519 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d74f89dc9-nlgnc" event={"ID":"09cfd04d-e167-46b3-b3a0-ab8de081f8a5","Type":"ContainerStarted","Data":"a87f641c3b929602d4fcbd6b26932be64f754b8d15d8e84ed595c3d7a006fb8b"} Jul 01 22:33:55.416547 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:55.416569 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d74f89dc9-nlgnc" event={"ID":"09cfd04d-e167-46b3-b3a0-ab8de081f8a5","Type":"ContainerStarted","Data":"86f2e2b25625084128bae44e70c8d4a85378d76ab7a3868566ade607893148de"} Jul 01 22:33:55.436840 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:55.436787 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7d74f89dc9-nlgnc" podStartSLOduration=1.007540741 podStartE2EDuration="1.436772541s" totalImagesPullingTime="429.2318ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 22:33:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:33:55.436185824 +0000 UTC m=+776.183626414" watchObservedRunningTime="2026-07-01 22:33:55.436772541 +0000 UTC m=+776.184213131" Jul 01 22:33:55.469407 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:55.469366 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5695859946-dctmr"] Jul 01 22:33:55.469895 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:55.469628 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5695859946-dctmr" podUID="532a2b2c-e23f-43f6-83e5-64295dd0562d" containerName="authorino" containerID="cri-o://a71f66852e116d095a85ce5e89d191fe7790cc4e76f47dc6df806456ae6705ff" gracePeriod=30 Jul 01 22:33:55.749112 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:55.749084 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5695859946-dctmr" Jul 01 22:33:55.822411 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:55.822383 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9g7p\" (UniqueName: \"kubernetes.io/projected/532a2b2c-e23f-43f6-83e5-64295dd0562d-kube-api-access-h9g7p\") pod \"532a2b2c-e23f-43f6-83e5-64295dd0562d\" (UID: \"532a2b2c-e23f-43f6-83e5-64295dd0562d\") " Jul 01 22:33:55.822604 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:55.822432 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/532a2b2c-e23f-43f6-83e5-64295dd0562d-oidc-ca\") pod \"532a2b2c-e23f-43f6-83e5-64295dd0562d\" (UID: \"532a2b2c-e23f-43f6-83e5-64295dd0562d\") " Jul 01 22:33:55.822604 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:55.822470 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/532a2b2c-e23f-43f6-83e5-64295dd0562d-tls-cert\") pod \"532a2b2c-e23f-43f6-83e5-64295dd0562d\" (UID: \"532a2b2c-e23f-43f6-83e5-64295dd0562d\") " Jul 01 22:33:55.824821 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:55.824778 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/532a2b2c-e23f-43f6-83e5-64295dd0562d-kube-api-access-h9g7p" (OuterVolumeSpecName: "kube-api-access-h9g7p") pod "532a2b2c-e23f-43f6-83e5-64295dd0562d" (UID: "532a2b2c-e23f-43f6-83e5-64295dd0562d"). InnerVolumeSpecName "kube-api-access-h9g7p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 22:33:55.827774 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:55.827736 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/532a2b2c-e23f-43f6-83e5-64295dd0562d-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "532a2b2c-e23f-43f6-83e5-64295dd0562d" (UID: "532a2b2c-e23f-43f6-83e5-64295dd0562d"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 22:33:55.833019 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:55.832966 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/532a2b2c-e23f-43f6-83e5-64295dd0562d-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "532a2b2c-e23f-43f6-83e5-64295dd0562d" (UID: "532a2b2c-e23f-43f6-83e5-64295dd0562d"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 22:33:55.923741 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:55.923709 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9g7p\" (UniqueName: \"kubernetes.io/projected/532a2b2c-e23f-43f6-83e5-64295dd0562d-kube-api-access-h9g7p\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:33:55.923741 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:55.923737 2580 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/532a2b2c-e23f-43f6-83e5-64295dd0562d-oidc-ca\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:33:55.923741 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:55.923748 2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/532a2b2c-e23f-43f6-83e5-64295dd0562d-tls-cert\") on node \"ip-10-0-129-238.ec2.internal\" DevicePath \"\"" Jul 01 22:33:56.424033 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:56.423996 2580 generic.go:356] "Generic (PLEG): container finished" podID="532a2b2c-e23f-43f6-83e5-64295dd0562d" containerID="a71f66852e116d095a85ce5e89d191fe7790cc4e76f47dc6df806456ae6705ff" exitCode=0 Jul 01 22:33:56.424236 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:56.424046 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5695859946-dctmr" Jul 01 22:33:56.424236 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:56.424087 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5695859946-dctmr" event={"ID":"532a2b2c-e23f-43f6-83e5-64295dd0562d","Type":"ContainerDied","Data":"a71f66852e116d095a85ce5e89d191fe7790cc4e76f47dc6df806456ae6705ff"} Jul 01 22:33:56.424236 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:56.424133 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5695859946-dctmr" event={"ID":"532a2b2c-e23f-43f6-83e5-64295dd0562d","Type":"ContainerDied","Data":"1fa108147e76d98f5dffd33120cc020e31b2c4ea62a2bef3f38ec5c1e678b0e2"} Jul 01 22:33:56.424236 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:56.424158 2580 scope.go:117] "RemoveContainer" containerID="a71f66852e116d095a85ce5e89d191fe7790cc4e76f47dc6df806456ae6705ff" Jul 01 22:33:56.436392 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:56.436366 2580 scope.go:117] "RemoveContainer" containerID="a71f66852e116d095a85ce5e89d191fe7790cc4e76f47dc6df806456ae6705ff" Jul 01 22:33:56.436936 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:33:56.436726 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a71f66852e116d095a85ce5e89d191fe7790cc4e76f47dc6df806456ae6705ff\": container with ID starting with a71f66852e116d095a85ce5e89d191fe7790cc4e76f47dc6df806456ae6705ff not found: ID does not exist" containerID="a71f66852e116d095a85ce5e89d191fe7790cc4e76f47dc6df806456ae6705ff" Jul 01 22:33:56.436936 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:56.436764 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a71f66852e116d095a85ce5e89d191fe7790cc4e76f47dc6df806456ae6705ff"} err="failed to get container status \"a71f66852e116d095a85ce5e89d191fe7790cc4e76f47dc6df806456ae6705ff\": rpc error: code = NotFound desc = could not find container \"a71f66852e116d095a85ce5e89d191fe7790cc4e76f47dc6df806456ae6705ff\": container with ID starting with a71f66852e116d095a85ce5e89d191fe7790cc4e76f47dc6df806456ae6705ff not found: ID does not exist" Jul 01 22:33:56.465342 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:56.465312 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5695859946-dctmr"] Jul 01 22:33:56.482200 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:56.482172 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5695859946-dctmr"] Jul 01 22:33:57.752103 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:33:57.752069 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="532a2b2c-e23f-43f6-83e5-64295dd0562d" path="/var/lib/kubelet/pods/532a2b2c-e23f-43f6-83e5-64295dd0562d/volumes" Jul 01 22:34:02.458882 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:02.458849 2580 generic.go:356] "Generic (PLEG): container finished" podID="579297ec-d799-4cd5-8db9-11e27a5f2f28" containerID="b84d83ad5f9c1669f6adfcb9803ef7a607223a4e4f6590206e83ade217920221" exitCode=2 Jul 01 22:34:02.459246 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:02.458926 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" event={"ID":"579297ec-d799-4cd5-8db9-11e27a5f2f28","Type":"ContainerDied","Data":"b84d83ad5f9c1669f6adfcb9803ef7a607223a4e4f6590206e83ade217920221"} Jul 01 22:34:02.459293 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:02.459265 2580 scope.go:117] "RemoveContainer" containerID="b84d83ad5f9c1669f6adfcb9803ef7a607223a4e4f6590206e83ade217920221" Jul 01 22:34:03.465757 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:03.465721 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" event={"ID":"579297ec-d799-4cd5-8db9-11e27a5f2f28","Type":"ContainerStarted","Data":"33afd11eb02aba6d82b3911d1e2b2cc4e5120583f66ac1f83a5ae11dcd86aa72"} Jul 01 22:34:03.466107 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:03.465928 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" Jul 01 22:34:14.472026 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:14.471996 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" Jul 01 22:34:35.629069 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:35.628994 2580 generic.go:356] "Generic (PLEG): container finished" podID="579297ec-d799-4cd5-8db9-11e27a5f2f28" containerID="33afd11eb02aba6d82b3911d1e2b2cc4e5120583f66ac1f83a5ae11dcd86aa72" exitCode=2 Jul 01 22:34:35.629659 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:35.629068 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" event={"ID":"579297ec-d799-4cd5-8db9-11e27a5f2f28","Type":"ContainerDied","Data":"33afd11eb02aba6d82b3911d1e2b2cc4e5120583f66ac1f83a5ae11dcd86aa72"} Jul 01 22:34:35.629659 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:35.629156 2580 scope.go:117] "RemoveContainer" containerID="b84d83ad5f9c1669f6adfcb9803ef7a607223a4e4f6590206e83ade217920221" Jul 01 22:34:35.629659 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:35.629421 2580 scope.go:117] "RemoveContainer" containerID="33afd11eb02aba6d82b3911d1e2b2cc4e5120583f66ac1f83a5ae11dcd86aa72" Jul 01 22:34:35.629659 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:34:35.629628 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 10s restarting failed container=payload-processing pod=payload-processing-7bbcb654f4-7mngm_openshift-ingress(579297ec-d799-4cd5-8db9-11e27a5f2f28)\"" pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" podUID="579297ec-d799-4cd5-8db9-11e27a5f2f28" Jul 01 22:34:35.800496 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:35.800468 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7d74f89dc9-nlgnc_09cfd04d-e167-46b3-b3a0-ab8de081f8a5/authorino/0.log" Jul 01 22:34:40.563356 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:40.563319 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-k7spv_2b92f5a0-0a01-4676-8adf-ad047c2f6ca4/manager/0.log" Jul 01 22:34:41.167858 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:41.167833 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b6ffc8659-cqbln_e5c96b43-1669-4a57-8a32-a8eb28e35d00/copy-manifests/0.log" Jul 01 22:34:41.187455 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:41.187423 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b6ffc8659-cqbln_e5c96b43-1669-4a57-8a32-a8eb28e35d00/manager/0.log" Jul 01 22:34:42.676534 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:42.676499 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7d74f89dc9-nlgnc_09cfd04d-e167-46b3-b3a0-ab8de081f8a5/authorino/0.log" Jul 01 22:34:42.915005 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:42.914972 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-ft7mh_fbdf2bcc-979e-4fda-b2aa-ca6894cd0f51/manager/0.log" Jul 01 22:34:43.147109 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:43.147081 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-86kxj_c92f9cce-4542-45af-8c59-850e8df59df6/registry-server/0.log" Jul 01 22:34:43.275287 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:43.275261 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-rsl8h_5103f362-b8c8-4b9c-9307-5c0ea11c6015/manager/0.log" Jul 01 22:34:43.529346 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:43.529274 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-rfpms_b3766e89-67d4-4fa3-a3a6-3fcac443522b/manager/0.log" Jul 01 22:34:43.888312 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:43.888283 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qz8f9_4b18eaaa-c9ea-429d-b752-ec0f9284a4c0/istio-proxy/0.log" Jul 01 22:34:44.350164 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:44.350129 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xr74c_d7640cec-b7f8-4db1-95ab-f929ffac82c8/istio-proxy/1.log" Jul 01 22:34:44.470652 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:44.470624 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" Jul 01 22:34:44.471050 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:44.471019 2580 scope.go:117] "RemoveContainer" containerID="33afd11eb02aba6d82b3911d1e2b2cc4e5120583f66ac1f83a5ae11dcd86aa72" Jul 01 22:34:44.471282 ip-10-0-129-238 kubenswrapper[2580]: E0701 22:34:44.471264 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 10s restarting failed container=payload-processing pod=payload-processing-7bbcb654f4-7mngm_openshift-ingress(579297ec-d799-4cd5-8db9-11e27a5f2f28)\"" pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" podUID="579297ec-d799-4cd5-8db9-11e27a5f2f28" Jul 01 22:34:44.582783 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:44.582752 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-7mngm_579297ec-d799-4cd5-8db9-11e27a5f2f28/payload-processing/1.log" Jul 01 22:34:44.696024 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:44.695951 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-dd9c9df89-c2lhf_39a42bc6-115f-4ce9-ba14-a35288061f5f/router/0.log" Jul 01 22:34:45.070101 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:45.070017 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-xbxtg_0758543e-6104-413b-b620-f49699af55e9/main/0.log" Jul 01 22:34:45.438542 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:45.438515 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbhjfn6_5b37990b-0603-43e4-b97b-b1a765a362e4/main/0.log" Jul 01 22:34:50.339224 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:50.339181 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" Jul 01 22:34:50.339631 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:50.339544 2580 scope.go:117] "RemoveContainer" containerID="33afd11eb02aba6d82b3911d1e2b2cc4e5120583f66ac1f83a5ae11dcd86aa72" Jul 01 22:34:50.713173 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:50.713141 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" event={"ID":"579297ec-d799-4cd5-8db9-11e27a5f2f28","Type":"ContainerStarted","Data":"108244f5e1baa61713529eb0588937a5512301f6b562a244bff94aa5d71fa458"} Jul 01 22:34:50.713370 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:50.713353 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" Jul 01 22:34:52.382016 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:52.381989 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-d5xct_f700798a-ec9b-4ee0-8a62-d5ff885135cd/global-pull-secret-syncer/0.log" Jul 01 22:34:52.545886 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:52.545863 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vz9dp_8694157d-ebfe-44aa-ae98-4ef443d09ec2/konnectivity-agent/0.log" Jul 01 22:34:52.594958 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:52.594922 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-238.ec2.internal_7d254b613571f06d3af95550e0820f84/haproxy/0.log" Jul 01 22:34:57.204904 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:57.204873 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7d74f89dc9-nlgnc_09cfd04d-e167-46b3-b3a0-ab8de081f8a5/authorino/0.log" Jul 01 22:34:57.266010 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:57.265987 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-ft7mh_fbdf2bcc-979e-4fda-b2aa-ca6894cd0f51/manager/0.log" Jul 01 22:34:57.343719 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:57.343693 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-86kxj_c92f9cce-4542-45af-8c59-850e8df59df6/registry-server/0.log" Jul 01 22:34:57.394128 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:57.394104 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-rsl8h_5103f362-b8c8-4b9c-9307-5c0ea11c6015/manager/0.log" Jul 01 22:34:57.468592 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:57.468520 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-rfpms_b3766e89-67d4-4fa3-a3a6-3fcac443522b/manager/0.log" Jul 01 22:34:59.349679 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:59.349648 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-54cb84584-mxlf8_2af46b0f-9553-4f48-abe5-76bdb3ae6a01/cluster-monitoring-operator/0.log" Jul 01 22:34:59.697626 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:59.697597 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mzhlc_2eb01e25-2de7-4fd6-aac5-8682ce27a267/node-exporter/0.log" Jul 01 22:34:59.727611 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:59.727590 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mzhlc_2eb01e25-2de7-4fd6-aac5-8682ce27a267/kube-rbac-proxy/0.log" Jul 01 22:34:59.753506 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:34:59.753484 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mzhlc_2eb01e25-2de7-4fd6-aac5-8682ce27a267/init-textfile/0.log" Jul 01 22:35:01.719871 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:01.719837 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bbcb654f4-7mngm" Jul 01 22:35:01.736956 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:01.736936 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cdb5968b9-8mthg_67818314-407c-407b-bd1c-5870354be6bd/networking-console-plugin/0.log" Jul 01 22:35:02.373070 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.373039 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8xh6_18708697-4b5c-4f52-9878-e369f8aa9eb2/console-operator/2.log" Jul 01 22:35:02.379838 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.379814 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-z8xh6_18708697-4b5c-4f52-9878-e369f8aa9eb2/console-operator/3.log" Jul 01 22:35:02.486145 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.486123 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c"] Jul 01 22:35:02.486479 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.486467 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="532a2b2c-e23f-43f6-83e5-64295dd0562d" containerName="authorino" Jul 01 22:35:02.486578 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.486480 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="532a2b2c-e23f-43f6-83e5-64295dd0562d" containerName="authorino" Jul 01 22:35:02.486578 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.486566 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="532a2b2c-e23f-43f6-83e5-64295dd0562d" containerName="authorino" Jul 01 22:35:02.489831 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.489815 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c" Jul 01 22:35:02.492435 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.492416 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-spcv6\"/\"openshift-service-ca.crt\"" Jul 01 22:35:02.493468 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.493451 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-spcv6\"/\"default-dockercfg-v9l84\"" Jul 01 22:35:02.493571 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.493451 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-spcv6\"/\"kube-root-ca.crt\"" Jul 01 22:35:02.505804 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.505785 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c"] Jul 01 22:35:02.559432 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.559408 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t975l\" (UniqueName: \"kubernetes.io/projected/a4cb4fff-f56f-4274-b3de-1d774fb86d2f-kube-api-access-t975l\") pod \"perf-node-gather-daemonset-t4f9c\" (UID: \"a4cb4fff-f56f-4274-b3de-1d774fb86d2f\") " pod="openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c" Jul 01 22:35:02.559619 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.559436 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a4cb4fff-f56f-4274-b3de-1d774fb86d2f-podres\") pod \"perf-node-gather-daemonset-t4f9c\" (UID: \"a4cb4fff-f56f-4274-b3de-1d774fb86d2f\") " pod="openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c" Jul 01 22:35:02.559619 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.559454 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a4cb4fff-f56f-4274-b3de-1d774fb86d2f-proc\") pod \"perf-node-gather-daemonset-t4f9c\" (UID: \"a4cb4fff-f56f-4274-b3de-1d774fb86d2f\") " pod="openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c" Jul 01 22:35:02.559619 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.559527 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4cb4fff-f56f-4274-b3de-1d774fb86d2f-sys\") pod \"perf-node-gather-daemonset-t4f9c\" (UID: \"a4cb4fff-f56f-4274-b3de-1d774fb86d2f\") " pod="openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c" Jul 01 22:35:02.559784 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.559652 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a4cb4fff-f56f-4274-b3de-1d774fb86d2f-lib-modules\") pod \"perf-node-gather-daemonset-t4f9c\" (UID: \"a4cb4fff-f56f-4274-b3de-1d774fb86d2f\") " pod="openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c" Jul 01 22:35:02.660459 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.660393 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4cb4fff-f56f-4274-b3de-1d774fb86d2f-sys\") pod \"perf-node-gather-daemonset-t4f9c\" (UID: \"a4cb4fff-f56f-4274-b3de-1d774fb86d2f\") " pod="openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c" Jul 01 22:35:02.660459 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.660439 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a4cb4fff-f56f-4274-b3de-1d774fb86d2f-lib-modules\") pod \"perf-node-gather-daemonset-t4f9c\" (UID: \"a4cb4fff-f56f-4274-b3de-1d774fb86d2f\") " pod="openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c" Jul 01 22:35:02.660685 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.660472 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t975l\" (UniqueName: \"kubernetes.io/projected/a4cb4fff-f56f-4274-b3de-1d774fb86d2f-kube-api-access-t975l\") pod \"perf-node-gather-daemonset-t4f9c\" (UID: \"a4cb4fff-f56f-4274-b3de-1d774fb86d2f\") " pod="openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c" Jul 01 22:35:02.660685 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.660488 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a4cb4fff-f56f-4274-b3de-1d774fb86d2f-podres\") pod \"perf-node-gather-daemonset-t4f9c\" (UID: \"a4cb4fff-f56f-4274-b3de-1d774fb86d2f\") " pod="openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c" Jul 01 22:35:02.660685 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.660507 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a4cb4fff-f56f-4274-b3de-1d774fb86d2f-proc\") pod \"perf-node-gather-daemonset-t4f9c\" (UID: \"a4cb4fff-f56f-4274-b3de-1d774fb86d2f\") " pod="openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c" Jul 01 22:35:02.660685 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.660510 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a4cb4fff-f56f-4274-b3de-1d774fb86d2f-sys\") pod \"perf-node-gather-daemonset-t4f9c\" (UID: \"a4cb4fff-f56f-4274-b3de-1d774fb86d2f\") " pod="openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c" Jul 01 22:35:02.660685 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.660606 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a4cb4fff-f56f-4274-b3de-1d774fb86d2f-proc\") pod \"perf-node-gather-daemonset-t4f9c\" (UID: \"a4cb4fff-f56f-4274-b3de-1d774fb86d2f\") " pod="openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c" Jul 01 22:35:02.660685 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.660638 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a4cb4fff-f56f-4274-b3de-1d774fb86d2f-podres\") pod \"perf-node-gather-daemonset-t4f9c\" (UID: \"a4cb4fff-f56f-4274-b3de-1d774fb86d2f\") " pod="openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c" Jul 01 22:35:02.660685 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.660684 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a4cb4fff-f56f-4274-b3de-1d774fb86d2f-lib-modules\") pod \"perf-node-gather-daemonset-t4f9c\" (UID: \"a4cb4fff-f56f-4274-b3de-1d774fb86d2f\") " pod="openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c" Jul 01 22:35:02.674638 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.674616 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t975l\" (UniqueName: \"kubernetes.io/projected/a4cb4fff-f56f-4274-b3de-1d774fb86d2f-kube-api-access-t975l\") pod \"perf-node-gather-daemonset-t4f9c\" (UID: \"a4cb4fff-f56f-4274-b3de-1d774fb86d2f\") " pod="openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c" Jul 01 22:35:02.800029 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.800008 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c" Jul 01 22:35:02.936196 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:02.936172 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c"] Jul 01 22:35:02.937875 ip-10-0-129-238 kubenswrapper[2580]: W0701 22:35:02.937852 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda4cb4fff_f56f_4274_b3de_1d774fb86d2f.slice/crio-f1e9a151055417ff063c2da6171fbb1c31713ea324c9e1fa2d55bc385530203f WatchSource:0}: Error finding container f1e9a151055417ff063c2da6171fbb1c31713ea324c9e1fa2d55bc385530203f: Status 404 returned error can't find the container with id f1e9a151055417ff063c2da6171fbb1c31713ea324c9e1fa2d55bc385530203f Jul 01 22:35:03.701084 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:03.701058 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-69d9f679db-p95p7_5f7fd383-088b-42e4-bf37-f6ae25e5e263/volume-data-source-validator/0.log" Jul 01 22:35:03.779928 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:03.779895 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c" event={"ID":"a4cb4fff-f56f-4274-b3de-1d774fb86d2f","Type":"ContainerStarted","Data":"5302728a8a4b6c27a4363d6ea2a58cd09d39907ca19926f45e282d84b268f7f7"} Jul 01 22:35:03.780084 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:03.779948 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c" event={"ID":"a4cb4fff-f56f-4274-b3de-1d774fb86d2f","Type":"ContainerStarted","Data":"f1e9a151055417ff063c2da6171fbb1c31713ea324c9e1fa2d55bc385530203f"} Jul 01 22:35:03.780084 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:03.779976 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c" Jul 01 22:35:03.798437 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:03.798396 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c" podStartSLOduration=1.798382084 podStartE2EDuration="1.798382084s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 22:35:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 22:35:03.79745506 +0000 UTC m=+844.544895650" watchObservedRunningTime="2026-07-01 22:35:03.798382084 +0000 UTC m=+844.545822674" Jul 01 22:35:04.794261 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:04.794226 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7zjsw_0ed3959f-efbe-43a9-a6f3-d7516f19dcd0/dns/0.log" Jul 01 22:35:04.820077 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:04.820054 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7zjsw_0ed3959f-efbe-43a9-a6f3-d7516f19dcd0/kube-rbac-proxy/0.log" Jul 01 22:35:04.956564 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:04.956528 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ws2ns_33732b5f-89f5-4e70-a763-cfb5cda678e0/dns-node-resolver/0.log" Jul 01 22:35:05.567535 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:05.567509 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wmmbz_fbbd6b24-4540-40b3-b0e8-a9d0b67559f5/node-ca/0.log" Jul 01 22:35:06.524538 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:06.524507 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qz8f9_4b18eaaa-c9ea-429d-b752-ec0f9284a4c0/istio-proxy/0.log" Jul 01 22:35:06.656840 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:06.656782 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xr74c_d7640cec-b7f8-4db1-95ab-f929ffac82c8/istio-proxy/1.log" Jul 01 22:35:06.657276 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:06.657254 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xr74c_d7640cec-b7f8-4db1-95ab-f929ffac82c8/istio-proxy/0.log" Jul 01 22:35:06.710608 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:06.710581 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-7mngm_579297ec-d799-4cd5-8db9-11e27a5f2f28/payload-processing/2.log" Jul 01 22:35:06.711801 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:06.711782 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-7mngm_579297ec-d799-4cd5-8db9-11e27a5f2f28/payload-processing/1.log" Jul 01 22:35:06.738525 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:06.738506 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-dd9c9df89-c2lhf_39a42bc6-115f-4ce9-ba14-a35288061f5f/router/0.log" Jul 01 22:35:07.409825 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:07.409796 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5sqqv_e405fa2c-a5db-47a7-9de0-909629fafdd8/serve-healthcheck-canary/0.log" Jul 01 22:35:07.928078 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:07.928049 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5f67b55555-lzdgs_c496dde0-19fa-46ee-82e2-41871f3fa2b7/insights-operator/0.log" Jul 01 22:35:07.930185 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:07.930165 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5f67b55555-lzdgs_c496dde0-19fa-46ee-82e2-41871f3fa2b7/insights-operator/1.log" Jul 01 22:35:07.951727 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:07.951705 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2st4x_629aff78-cbe4-45e4-8277-3cb809af23ca/kube-rbac-proxy/0.log" Jul 01 22:35:07.975440 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:07.975420 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2st4x_629aff78-cbe4-45e4-8277-3cb809af23ca/exporter/0.log" Jul 01 22:35:07.998816 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:07.998781 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2st4x_629aff78-cbe4-45e4-8277-3cb809af23ca/extractor/0.log" Jul 01 22:35:09.794344 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:09.794314 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-spcv6/perf-node-gather-daemonset-t4f9c" Jul 01 22:35:10.368920 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:10.368892 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-k7spv_2b92f5a0-0a01-4676-8adf-ad047c2f6ca4/manager/0.log" Jul 01 22:35:10.638338 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:10.638253 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b6ffc8659-cqbln_e5c96b43-1669-4a57-8a32-a8eb28e35d00/manager/0.log" Jul 01 22:35:10.664878 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:10.664857 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b6ffc8659-cqbln_e5c96b43-1669-4a57-8a32-a8eb28e35d00/copy-manifests/0.log" Jul 01 22:35:12.259102 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:12.259069 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-658788d8fc-rzxrn_8283436a-43f3-4e1b-bc39-814874332817/manager/0.log" Jul 01 22:35:13.105459 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:13.105412 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qz8f9_4b18eaaa-c9ea-429d-b752-ec0f9284a4c0/istio-proxy/0.log" Jul 01 22:35:13.223422 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:13.223397 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xr74c_d7640cec-b7f8-4db1-95ab-f929ffac82c8/istio-proxy/0.log" Jul 01 22:35:13.224063 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:13.224038 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xr74c_d7640cec-b7f8-4db1-95ab-f929ffac82c8/istio-proxy/1.log" Jul 01 22:35:13.276737 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:13.276707 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-7mngm_579297ec-d799-4cd5-8db9-11e27a5f2f28/payload-processing/2.log" Jul 01 22:35:13.277905 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:13.277883 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-7mngm_579297ec-d799-4cd5-8db9-11e27a5f2f28/payload-processing/1.log" Jul 01 22:35:13.291959 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:13.291936 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-dd9c9df89-c2lhf_39a42bc6-115f-4ce9-ba14-a35288061f5f/router/0.log" Jul 01 22:35:14.466413 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:14.466315 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4qz8f9_4b18eaaa-c9ea-429d-b752-ec0f9284a4c0/istio-proxy/0.log" Jul 01 22:35:14.573373 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:14.573342 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xr74c_d7640cec-b7f8-4db1-95ab-f929ffac82c8/istio-proxy/1.log" Jul 01 22:35:14.573741 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:14.573712 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-xr74c_d7640cec-b7f8-4db1-95ab-f929ffac82c8/istio-proxy/0.log" Jul 01 22:35:14.608433 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:14.608412 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-7mngm_579297ec-d799-4cd5-8db9-11e27a5f2f28/payload-processing/2.log" Jul 01 22:35:14.609622 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:14.609602 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bbcb654f4-7mngm_579297ec-d799-4cd5-8db9-11e27a5f2f28/payload-processing/1.log" Jul 01 22:35:14.626300 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:14.626282 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-dd9c9df89-c2lhf_39a42bc6-115f-4ce9-ba14-a35288061f5f/router/0.log" Jul 01 22:35:17.408270 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:17.408228 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-2758l_6570158d-3c50-41c9-93ca-0b912e915894/migrator/0.log" Jul 01 22:35:17.430130 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:17.430103 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-786f9fbc5-2758l_6570158d-3c50-41c9-93ca-0b912e915894/graceful-termination/0.log" Jul 01 22:35:17.863179 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:17.863147 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-79964ff6f9-fw5wx_a4590f39-429a-43f3-8c4b-5d2e7fc4b378/kube-storage-version-migrator-operator/1.log" Jul 01 22:35:17.864768 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:17.864742 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-79964ff6f9-fw5wx_a4590f39-429a-43f3-8c4b-5d2e7fc4b378/kube-storage-version-migrator-operator/0.log" Jul 01 22:35:19.227168 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:19.227141 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p44zj_4e8d667b-1ec1-4147-9238-6f016882b971/kube-multus-additional-cni-plugins/0.log" Jul 01 22:35:19.251823 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:19.251789 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p44zj_4e8d667b-1ec1-4147-9238-6f016882b971/egress-router-binary-copy/0.log" Jul 01 22:35:19.274406 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:19.274381 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p44zj_4e8d667b-1ec1-4147-9238-6f016882b971/cni-plugins/0.log" Jul 01 22:35:19.299293 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:19.299269 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p44zj_4e8d667b-1ec1-4147-9238-6f016882b971/bond-cni-plugin/0.log" Jul 01 22:35:19.326077 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:19.326056 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p44zj_4e8d667b-1ec1-4147-9238-6f016882b971/routeoverride-cni/0.log" Jul 01 22:35:19.349191 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:19.349169 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p44zj_4e8d667b-1ec1-4147-9238-6f016882b971/whereabouts-cni-bincopy/0.log" Jul 01 22:35:19.373762 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:19.373737 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p44zj_4e8d667b-1ec1-4147-9238-6f016882b971/whereabouts-cni/0.log" Jul 01 22:35:19.436069 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:19.436047 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-d4j26_f7388d18-cc3b-407c-98ee-5543c413b77e/kube-multus/0.log" Jul 01 22:35:19.610038 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:19.610007 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-kfd7x_246c052d-11a5-4032-9505-a1628e462a19/network-metrics-daemon/0.log" Jul 01 22:35:19.629972 ip-10-0-129-238 kubenswrapper[2580]: I0701 22:35:19.629944 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-kfd7x_246c052d-11a5-4032-9505-a1628e462a19/kube-rbac-proxy/0.log"