Jul 07 16:16:24.917456 ip-10-0-129-174 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 07 16:16:24.917472 ip-10-0-129-174 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 07 16:16:24.917483 ip-10-0-129-174 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 07 16:16:24.917821 ip-10-0-129-174 systemd[1]: Failed to start Kubernetes Kubelet. Jul 07 16:16:35.149550 ip-10-0-129-174 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 07 16:16:35.149597 ip-10-0-129-174 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot aa2e6ed4ebc8456580f79ce5945b9999 -- Jul 07 16:18:10.419768 ip-10-0-129-174 systemd[1]: Starting Kubernetes Kubelet... Jul 07 16:18:10.860839 ip-10-0-129-174 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 16:18:10.860839 ip-10-0-129-174 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 16:18:10.860839 ip-10-0-129-174 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 16:18:10.860839 ip-10-0-129-174 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 16:18:10.860839 ip-10-0-129-174 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 16:18:10.860839 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.851031 2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 16:18:10.860839 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855447 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 16:18:10.860839 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855461 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 16:18:10.860839 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855464 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 16:18:10.860839 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855467 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 16:18:10.860839 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855470 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 16:18:10.860839 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855473 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 16:18:10.860839 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855476 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 16:18:10.860839 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855480 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 16:18:10.875995 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855484 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 16:18:10.875995 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855488 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 16:18:10.875995 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855491 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 16:18:10.875995 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855500 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 16:18:10.875995 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855503 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 16:18:10.875995 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855505 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 16:18:10.875995 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855508 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 16:18:10.875995 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855511 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 16:18:10.875995 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855513 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 16:18:10.875995 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855516 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 16:18:10.875995 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855519 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 16:18:10.875995 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855522 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 16:18:10.875995 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855525 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 16:18:10.875995 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855527 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 16:18:10.875995 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855530 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 16:18:10.875995 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855532 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 16:18:10.875995 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855535 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 16:18:10.875995 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855537 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 16:18:10.875995 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855540 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 16:18:10.877840 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855544 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 16:18:10.877840 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855547 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 16:18:10.877840 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855549 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 16:18:10.877840 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855552 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 16:18:10.877840 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855555 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 16:18:10.877840 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855558 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 16:18:10.877840 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855560 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 16:18:10.877840 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855563 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 16:18:10.877840 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855566 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 16:18:10.877840 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855569 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 16:18:10.877840 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855572 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 16:18:10.877840 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855574 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 16:18:10.877840 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855577 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 16:18:10.877840 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855580 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 16:18:10.877840 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855583 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 16:18:10.877840 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855586 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 16:18:10.877840 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855589 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 16:18:10.877840 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855593 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 16:18:10.877840 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855596 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 16:18:10.880651 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855598 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 16:18:10.880651 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855601 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 16:18:10.880651 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855603 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 16:18:10.880651 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855606 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 16:18:10.880651 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855608 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 16:18:10.880651 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855611 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 16:18:10.880651 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855613 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 16:18:10.880651 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855616 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 16:18:10.880651 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855619 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 07 16:18:10.880651 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855621 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 16:18:10.880651 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855624 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 16:18:10.880651 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855626 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 16:18:10.880651 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855630 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 16:18:10.880651 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855635 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 16:18:10.880651 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855639 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 16:18:10.880651 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855644 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 16:18:10.880651 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855647 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 16:18:10.880651 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855651 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 16:18:10.880651 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855656 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 16:18:10.880651 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855660 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 16:18:10.895126 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855664 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 16:18:10.895126 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855667 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 16:18:10.895126 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855669 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 16:18:10.895126 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855672 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 16:18:10.895126 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855675 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 16:18:10.895126 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855678 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 16:18:10.895126 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855681 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 16:18:10.895126 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855684 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 16:18:10.895126 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855686 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 16:18:10.895126 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855691 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 16:18:10.895126 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855695 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 16:18:10.895126 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855698 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 16:18:10.895126 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855701 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 16:18:10.895126 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855704 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 16:18:10.895126 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855706 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 16:18:10.895126 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855710 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 16:18:10.895126 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855713 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 16:18:10.895126 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855717 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 16:18:10.895126 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855721 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 16:18:10.895126 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855723 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 16:18:10.897772 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855726 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 16:18:10.897772 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855728 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 16:18:10.897772 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855731 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 16:18:10.897772 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855734 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 16:18:10.897772 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855737 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 16:18:10.897772 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855739 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 16:18:10.897772 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855742 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 16:18:10.897772 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855745 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 16:18:10.897772 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855747 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 16:18:10.897772 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855751 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 16:18:10.897772 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855754 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 16:18:10.897772 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855757 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 16:18:10.897772 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.855760 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 16:18:10.897772 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856502 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 16:18:10.897772 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856508 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 16:18:10.897772 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856511 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 16:18:10.897772 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856514 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 16:18:10.897772 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856517 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 16:18:10.897772 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856520 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 16:18:10.899476 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856523 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 16:18:10.899476 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856526 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 16:18:10.899476 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856528 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 16:18:10.899476 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856531 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 16:18:10.899476 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856533 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 16:18:10.899476 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856536 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 16:18:10.899476 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856541 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 16:18:10.899476 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856544 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 16:18:10.899476 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856548 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 16:18:10.899476 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856551 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 16:18:10.899476 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856554 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 16:18:10.899476 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856557 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 16:18:10.899476 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856561 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 16:18:10.899476 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856564 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 16:18:10.899476 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856567 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 16:18:10.899476 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856570 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 16:18:10.899476 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856573 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 16:18:10.899476 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856575 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 16:18:10.899476 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856578 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 16:18:10.915973 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856581 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 16:18:10.915973 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856584 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 16:18:10.915973 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856587 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 16:18:10.915973 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856589 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 16:18:10.915973 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856591 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 16:18:10.915973 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856594 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 16:18:10.915973 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856597 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 16:18:10.915973 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856599 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 16:18:10.915973 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856603 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 16:18:10.915973 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856606 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 16:18:10.915973 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856608 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 07 16:18:10.915973 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856611 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 16:18:10.915973 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856614 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 16:18:10.915973 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856617 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 16:18:10.915973 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856619 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 16:18:10.915973 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856623 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 16:18:10.915973 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856626 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 16:18:10.915973 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856629 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 16:18:10.915973 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856631 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 16:18:10.915973 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856634 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 16:18:10.917869 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856638 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 16:18:10.917869 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856641 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 16:18:10.917869 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856644 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 16:18:10.917869 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856647 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 16:18:10.917869 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856649 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 16:18:10.917869 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856652 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 16:18:10.917869 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856654 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 16:18:10.917869 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856657 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 16:18:10.917869 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856659 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 16:18:10.917869 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856662 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 16:18:10.917869 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856665 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 16:18:10.917869 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856668 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 16:18:10.917869 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856671 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 16:18:10.917869 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856673 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 16:18:10.917869 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856676 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 16:18:10.917869 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856678 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 16:18:10.917869 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856681 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 16:18:10.917869 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856685 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 16:18:10.917869 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856687 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 16:18:10.917869 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856690 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 16:18:10.920472 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856693 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 16:18:10.920472 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856695 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 16:18:10.920472 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856698 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 16:18:10.920472 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856701 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 16:18:10.920472 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856703 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 16:18:10.920472 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856706 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 16:18:10.920472 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856709 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 16:18:10.920472 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856713 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 16:18:10.920472 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856716 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 16:18:10.920472 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856719 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 16:18:10.920472 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856722 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 16:18:10.920472 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856724 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 16:18:10.920472 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856727 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 16:18:10.920472 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856730 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 16:18:10.920472 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856732 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 16:18:10.920472 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856735 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 16:18:10.920472 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856738 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 16:18:10.920472 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856740 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 16:18:10.920472 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856743 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 16:18:10.953611 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856746 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 16:18:10.953611 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856748 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 16:18:10.953611 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856751 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 16:18:10.953611 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856754 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 16:18:10.953611 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856756 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 16:18:10.953611 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856759 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 16:18:10.953611 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856761 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 16:18:10.953611 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856764 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 16:18:10.953611 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856766 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 16:18:10.953611 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856769 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 16:18:10.953611 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856771 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 16:18:10.953611 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856774 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 16:18:10.953611 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856776 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 16:18:10.953611 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856779 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 16:18:10.953611 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.856781 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 16:18:10.953611 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857179 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 16:18:10.953611 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857185 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 16:18:10.953611 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857189 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 16:18:10.925532 ip-10-0-129-174 systemd[1]: Started Kubernetes Kubelet. Jul 07 16:18:10.969020 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857192 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 16:18:10.969020 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857196 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 16:18:10.969020 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857199 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 16:18:10.969020 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857202 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 16:18:10.969020 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857205 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 16:18:10.969020 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857208 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 16:18:10.969020 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857211 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 16:18:10.969020 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857213 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 16:18:10.969020 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857216 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 16:18:10.969020 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857219 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 16:18:10.969020 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857224 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 16:18:10.969020 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857226 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 16:18:10.969020 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857229 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 16:18:10.969020 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857232 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 16:18:10.969020 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857235 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 16:18:10.969020 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857237 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 16:18:10.969020 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857241 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 16:18:10.969020 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857244 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 16:18:10.969020 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857246 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 16:18:10.969020 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857249 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 16:18:11.007425 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857251 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 16:18:11.007425 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857254 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 16:18:11.007425 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857257 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 16:18:11.007425 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857259 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 16:18:11.007425 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857262 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 16:18:11.007425 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857264 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 16:18:11.007425 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857267 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 16:18:11.007425 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857270 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 16:18:11.007425 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857273 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 16:18:11.007425 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857275 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 16:18:11.007425 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857278 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 16:18:11.007425 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857280 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 16:18:11.007425 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857283 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 16:18:11.007425 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857286 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 16:18:11.007425 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857289 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 16:18:11.007425 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857291 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 16:18:11.007425 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857295 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 16:18:11.007425 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857297 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 16:18:11.007425 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857300 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 16:18:11.007425 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857302 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 16:18:11.025943 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857305 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 16:18:11.025943 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857307 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 16:18:11.025943 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857312 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 16:18:11.025943 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857315 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 16:18:11.025943 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857317 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 16:18:11.025943 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857320 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 16:18:11.025943 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857322 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 16:18:11.025943 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857325 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 16:18:11.025943 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857327 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 16:18:11.025943 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857330 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 16:18:11.025943 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857332 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 16:18:11.025943 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857335 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 16:18:11.025943 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857337 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 16:18:11.025943 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857339 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 16:18:11.025943 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857342 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 16:18:11.025943 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857345 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 16:18:11.025943 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857347 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 16:18:11.025943 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857350 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 16:18:11.025943 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857354 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 16:18:11.025943 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857358 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 16:18:11.188132 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857361 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 16:18:11.188132 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857364 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 16:18:11.188132 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857380 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 16:18:11.188132 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857383 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 07 16:18:11.188132 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857386 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 16:18:11.188132 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857389 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 16:18:11.188132 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857392 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 16:18:11.188132 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857395 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 16:18:11.188132 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857398 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 16:18:11.188132 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857401 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 16:18:11.188132 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857404 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 16:18:11.188132 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857407 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 16:18:11.188132 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857409 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 16:18:11.188132 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857412 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 16:18:11.188132 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857417 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 16:18:11.188132 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857420 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 16:18:11.188132 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857422 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 16:18:11.188132 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857425 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 16:18:11.188132 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857427 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 16:18:11.188132 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857431 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 16:18:11.190998 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857434 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 16:18:11.190998 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857436 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 16:18:11.190998 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857439 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 16:18:11.190998 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857442 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 16:18:11.190998 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857444 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 16:18:11.190998 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857447 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 16:18:11.190998 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857450 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 16:18:11.190998 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857453 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 16:18:11.190998 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857458 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 16:18:11.190998 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857461 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 16:18:11.190998 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857463 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 16:18:11.190998 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857466 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 16:18:11.190998 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857468 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 16:18:11.190998 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857471 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 16:18:11.190998 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857473 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 16:18:11.190998 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.857476 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 16:18:11.190998 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857555 2564 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 16:18:11.190998 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857562 2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 16:18:11.190998 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857569 2564 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 16:18:11.190998 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857574 2564 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 16:18:11.190998 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857579 2564 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857583 2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857587 2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857591 2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857594 2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857597 2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857602 2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857605 2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857608 2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857611 2564 flags.go:64] FLAG: --cgroup-root="" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857614 2564 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857617 2564 flags.go:64] FLAG: --client-ca-file="" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857620 2564 flags.go:64] FLAG: --cloud-provider="external" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857623 2564 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857627 2564 flags.go:64] FLAG: --cluster-domain="" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857630 2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857633 2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857636 2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857640 2564 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857644 2564 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857649 2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857652 2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857655 2564 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857658 2564 flags.go:64] FLAG: --contention-profiling="false" Jul 07 16:18:11.193398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857661 2564 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857664 2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857667 2564 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857670 2564 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857674 2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857677 2564 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857680 2564 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857683 2564 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857686 2564 flags.go:64] FLAG: --enable-server="true" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857690 2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857694 2564 flags.go:64] FLAG: --event-burst="100" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857697 2564 flags.go:64] FLAG: --event-qps="50" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857700 2564 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857703 2564 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857708 2564 flags.go:64] FLAG: --eviction-hard="" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857712 2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857714 2564 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857718 2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857721 2564 flags.go:64] FLAG: --eviction-soft="" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857724 2564 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857727 2564 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857729 2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857732 2564 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857735 2564 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857738 2564 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 16:18:11.195412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857741 2564 flags.go:64] FLAG: --feature-gates="" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857745 2564 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857748 2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857752 2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857755 2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857758 2564 flags.go:64] FLAG: --healthz-port="10248" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857761 2564 flags.go:64] FLAG: --help="false" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857764 2564 flags.go:64] FLAG: --hostname-override="ip-10-0-129-174.ec2.internal" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857767 2564 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857770 2564 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857773 2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857776 2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857780 2564 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857783 2564 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857786 2564 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857789 2564 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857792 2564 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857795 2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857798 2564 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857802 2564 flags.go:64] FLAG: --kube-reserved="" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857805 2564 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857809 2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857812 2564 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857815 2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 16:18:11.197821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857818 2564 flags.go:64] FLAG: --lock-file="" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857821 2564 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857823 2564 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857826 2564 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857831 2564 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857834 2564 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857837 2564 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857840 2564 flags.go:64] FLAG: --logging-format="text" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857843 2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857846 2564 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857849 2564 flags.go:64] FLAG: --manifest-url="" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857853 2564 flags.go:64] FLAG: --manifest-url-header="" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857858 2564 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857861 2564 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857865 2564 flags.go:64] FLAG: --max-pods="110" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857868 2564 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857871 2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857874 2564 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857877 2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857880 2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857883 2564 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857886 2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857895 2564 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857898 2564 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857900 2564 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 16:18:11.216228 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857904 2564 flags.go:64] FLAG: --pod-cidr="" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857908 2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857914 2564 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857917 2564 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857922 2564 flags.go:64] FLAG: --pods-per-core="0" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857933 2564 flags.go:64] FLAG: --port="10250" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857936 2564 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857939 2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0707c6ba4b09e97ee" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857943 2564 flags.go:64] FLAG: --qos-reserved="" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857945 2564 flags.go:64] FLAG: --read-only-port="10255" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857948 2564 flags.go:64] FLAG: --register-node="true" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857951 2564 flags.go:64] FLAG: --register-with-taints="" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857955 2564 flags.go:64] FLAG: --registry-burst="10" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857958 2564 flags.go:64] FLAG: --registry-qps="5" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857961 2564 flags.go:64] FLAG: --reserved-cpus="" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857963 2564 flags.go:64] FLAG: --reserved-memory="" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857967 2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857970 2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857973 2564 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857977 2564 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857981 2564 flags.go:64] FLAG: --runonce="false" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857985 2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857990 2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857993 2564 flags.go:64] FLAG: --seccomp-default="false" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857996 2564 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 16:18:11.219934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.857999 2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.858002 2564 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.858005 2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.858009 2564 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.858014 2564 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.858017 2564 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.858020 2564 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.858024 2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.858027 2564 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.858031 2564 flags.go:64] FLAG: --system-cgroups="" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.858035 2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.858042 2564 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.858045 2564 flags.go:64] FLAG: --tls-cert-file="" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.858050 2564 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.858054 2564 flags.go:64] FLAG: --tls-min-version="" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.858057 2564 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.858061 2564 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.858064 2564 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.858067 2564 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.858071 2564 flags.go:64] FLAG: --v="2" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.858075 2564 flags.go:64] FLAG: --version="false" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.858080 2564 flags.go:64] FLAG: --vmodule="" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.858085 2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.858089 2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858233 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 16:18:11.224248 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858238 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 16:18:11.227547 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858243 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 16:18:11.227547 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858246 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 16:18:11.227547 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858250 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 16:18:11.227547 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858254 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 16:18:11.227547 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858257 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 16:18:11.227547 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858260 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 16:18:11.227547 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858264 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 16:18:11.227547 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858267 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 16:18:11.227547 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858270 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 16:18:11.227547 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858273 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 16:18:11.227547 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858276 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 16:18:11.227547 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858279 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 16:18:11.227547 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858283 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 16:18:11.227547 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858286 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 16:18:11.227547 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858289 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 16:18:11.227547 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858293 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 16:18:11.227547 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858297 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 16:18:11.227547 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858302 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 16:18:11.227547 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858305 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 16:18:11.227547 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858309 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 16:18:11.228860 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858313 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 16:18:11.228860 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858316 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 16:18:11.228860 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858319 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 16:18:11.228860 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858322 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 16:18:11.228860 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858325 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 16:18:11.228860 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858328 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 16:18:11.228860 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858332 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 16:18:11.228860 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858336 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 16:18:11.228860 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858339 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 16:18:11.228860 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858342 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 16:18:11.228860 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858346 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 16:18:11.228860 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858349 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 16:18:11.228860 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858353 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 16:18:11.228860 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858357 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 16:18:11.228860 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858360 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 16:18:11.228860 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858363 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 16:18:11.228860 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858383 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 16:18:11.228860 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858387 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 16:18:11.228860 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858390 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 16:18:11.228860 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858394 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 07 16:18:11.230002 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858397 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 16:18:11.230002 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858400 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 16:18:11.230002 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858404 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 16:18:11.230002 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858407 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 16:18:11.230002 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858411 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 16:18:11.230002 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858415 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 16:18:11.230002 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858420 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 16:18:11.230002 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858424 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 16:18:11.230002 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858427 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 16:18:11.230002 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858454 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 16:18:11.230002 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858472 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 16:18:11.230002 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858477 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 16:18:11.230002 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858481 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 16:18:11.230002 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858483 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 16:18:11.230002 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858486 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 16:18:11.230002 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858490 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 16:18:11.230002 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858492 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 16:18:11.230002 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858495 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 16:18:11.230002 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858498 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 16:18:11.230002 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858500 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 16:18:11.231320 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858504 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 16:18:11.231320 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858507 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 16:18:11.231320 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858509 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 16:18:11.231320 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858512 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 16:18:11.231320 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858515 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 16:18:11.231320 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858518 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 16:18:11.231320 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858521 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 16:18:11.231320 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858524 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 16:18:11.231320 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858529 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 16:18:11.231320 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858533 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 16:18:11.231320 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858538 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 16:18:11.231320 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858542 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 16:18:11.231320 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858545 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 16:18:11.231320 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858549 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 16:18:11.231320 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858552 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 16:18:11.231320 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858555 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 16:18:11.231320 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858558 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 16:18:11.231320 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858561 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 16:18:11.231320 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858564 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 16:18:11.232209 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858568 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 16:18:11.232209 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858571 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 16:18:11.232209 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858575 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 16:18:11.232209 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858580 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 16:18:11.232209 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858583 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 16:18:11.232209 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858586 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 16:18:11.232209 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858589 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 16:18:11.232209 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858592 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 16:18:11.232209 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858595 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 16:18:11.232209 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858598 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 16:18:11.232209 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858601 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 16:18:11.232209 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858604 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 16:18:11.232209 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858606 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 16:18:11.232209 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858609 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 16:18:11.232209 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858612 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 16:18:11.232209 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858615 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 16:18:11.232209 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858617 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 16:18:11.232209 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.858620 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 16:18:11.233683 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.859464 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 16:18:11.233683 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.866983 2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 16:18:11.233683 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.867005 2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 16:18:11.233683 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.867025 2564 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 16:18:11.233683 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.867033 2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 16:18:11.233683 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867100 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 16:18:11.233683 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867106 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 16:18:11.233683 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867109 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 16:18:11.233683 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867112 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 16:18:11.233683 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867115 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 16:18:11.233683 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867117 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 16:18:11.233683 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867120 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 16:18:11.233683 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867123 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 16:18:11.233683 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867126 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 16:18:11.233683 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867129 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 16:18:11.234758 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867132 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 16:18:11.234758 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867135 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 16:18:11.234758 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867138 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 16:18:11.234758 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867141 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 16:18:11.234758 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867143 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 16:18:11.234758 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867146 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 16:18:11.234758 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867149 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 16:18:11.234758 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867152 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 16:18:11.234758 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867155 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 16:18:11.234758 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867158 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 16:18:11.234758 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867161 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 16:18:11.234758 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867164 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 16:18:11.234758 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867167 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 16:18:11.234758 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867169 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 16:18:11.234758 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867172 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 16:18:11.234758 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867175 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 16:18:11.234758 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867177 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 16:18:11.234758 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867180 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 16:18:11.234758 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867183 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 16:18:11.234758 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867192 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 16:18:11.236048 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867195 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 16:18:11.236048 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867197 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 16:18:11.236048 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867200 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 16:18:11.236048 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867202 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 16:18:11.236048 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867205 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 16:18:11.236048 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867208 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 16:18:11.236048 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867210 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 16:18:11.236048 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867213 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 16:18:11.236048 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867215 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 16:18:11.236048 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867218 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 16:18:11.236048 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867220 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 16:18:11.236048 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867223 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 16:18:11.236048 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867226 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 16:18:11.236048 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867228 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 16:18:11.236048 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867231 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 16:18:11.236048 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867233 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 16:18:11.236048 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867236 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 16:18:11.236048 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867238 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 16:18:11.236048 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867241 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 16:18:11.236921 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867243 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 16:18:11.236921 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867246 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 16:18:11.236921 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867249 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 16:18:11.236921 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867252 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 16:18:11.236921 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867255 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 16:18:11.236921 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867258 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 16:18:11.236921 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867260 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 16:18:11.236921 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867264 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 16:18:11.236921 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867268 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 16:18:11.236921 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867271 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 16:18:11.236921 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867274 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 16:18:11.236921 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867276 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 16:18:11.236921 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867279 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 16:18:11.236921 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867287 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 16:18:11.236921 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867290 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 16:18:11.236921 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867293 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 16:18:11.236921 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867295 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 16:18:11.236921 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867297 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 16:18:11.236921 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867300 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 16:18:11.238219 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867303 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 16:18:11.238219 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867305 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 16:18:11.238219 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867308 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 16:18:11.238219 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867310 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 16:18:11.238219 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867312 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 16:18:11.238219 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867316 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 16:18:11.238219 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867320 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 16:18:11.238219 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867323 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 07 16:18:11.238219 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867326 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 16:18:11.238219 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867329 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 16:18:11.238219 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867331 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 16:18:11.238219 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867335 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 16:18:11.238219 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867338 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 16:18:11.238219 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867341 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 16:18:11.238219 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867344 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 16:18:11.238219 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867347 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 16:18:11.238219 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867350 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 16:18:11.238219 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867353 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 16:18:11.238219 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867355 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 16:18:11.238219 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867358 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 16:18:11.239107 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867360 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 16:18:11.239107 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867362 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 16:18:11.239107 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867365 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 16:18:11.239107 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867384 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 16:18:11.239107 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867387 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 16:18:11.239107 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867390 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 16:18:11.239107 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867392 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 16:18:11.239107 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867400 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 16:18:11.239107 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867403 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 16:18:11.239107 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867406 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 16:18:11.239107 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867408 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 16:18:11.239107 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.867414 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 16:18:11.239107 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867568 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 16:18:11.239107 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867573 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 16:18:11.239107 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867576 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 16:18:11.240616 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867579 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 16:18:11.240616 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867582 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 16:18:11.240616 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867585 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 16:18:11.240616 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867588 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 16:18:11.240616 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867590 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 16:18:11.240616 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867593 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 16:18:11.240616 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867596 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 16:18:11.240616 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867599 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 16:18:11.240616 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867603 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 16:18:11.240616 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867607 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 16:18:11.240616 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867611 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 16:18:11.240616 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867614 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 16:18:11.240616 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867616 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 16:18:11.240616 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867619 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 16:18:11.240616 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867623 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 16:18:11.240616 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867626 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 16:18:11.240616 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867629 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 16:18:11.240616 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867632 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 16:18:11.240616 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867635 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 16:18:11.241685 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867638 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 16:18:11.241685 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867640 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 16:18:11.241685 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867643 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 16:18:11.241685 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867646 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 16:18:11.241685 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867649 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 16:18:11.241685 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867652 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 16:18:11.241685 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867661 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 16:18:11.241685 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867664 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 16:18:11.241685 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867666 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 16:18:11.241685 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867669 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 16:18:11.241685 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867671 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 16:18:11.241685 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867674 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 16:18:11.241685 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867676 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 16:18:11.241685 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867679 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 16:18:11.241685 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867681 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 16:18:11.241685 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867684 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 16:18:11.241685 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867686 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 16:18:11.241685 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867689 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 16:18:11.241685 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867691 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 16:18:11.241685 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867694 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 16:18:11.242567 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867697 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 16:18:11.242567 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867699 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 16:18:11.242567 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867702 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 16:18:11.242567 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867704 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 16:18:11.242567 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867707 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 16:18:11.242567 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867710 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 16:18:11.242567 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867713 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 16:18:11.242567 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867715 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 16:18:11.242567 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867718 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 07 16:18:11.242567 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867720 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 16:18:11.242567 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867723 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 16:18:11.242567 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867725 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 16:18:11.242567 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867728 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 16:18:11.242567 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867730 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 16:18:11.242567 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867733 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 16:18:11.242567 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867735 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 16:18:11.242567 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867738 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 16:18:11.242567 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867741 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 16:18:11.242567 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867744 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 16:18:11.242567 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867753 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 16:18:11.243442 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867756 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 16:18:11.243442 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867759 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 16:18:11.243442 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867762 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 16:18:11.243442 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867764 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 16:18:11.243442 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867766 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 16:18:11.243442 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867769 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 16:18:11.243442 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867772 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 16:18:11.243442 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867774 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 16:18:11.243442 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867780 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 16:18:11.243442 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867783 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 16:18:11.243442 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867785 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 16:18:11.243442 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867788 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 16:18:11.243442 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867791 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 16:18:11.243442 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867793 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 16:18:11.243442 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867796 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 16:18:11.243442 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867798 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 16:18:11.243442 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867800 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 16:18:11.243442 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867803 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 16:18:11.243442 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867806 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 16:18:11.244932 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867808 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 16:18:11.244932 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867811 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 16:18:11.244932 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867814 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 16:18:11.244932 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867816 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 16:18:11.244932 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867818 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 16:18:11.244932 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867821 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 16:18:11.244932 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867823 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 16:18:11.244932 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867826 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 16:18:11.244932 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867829 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 16:18:11.244932 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867831 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 16:18:11.244932 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867833 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 16:18:11.244932 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867836 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 16:18:11.244932 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867838 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 16:18:11.244932 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867848 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 16:18:11.244932 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867850 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 16:18:11.244932 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867853 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 16:18:11.244932 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867855 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 16:18:11.244932 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:10.867857 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 16:18:11.245796 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.867862 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 16:18:11.245796 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.868029 2564 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 16:18:11.245796 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.870066 2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 16:18:11.245796 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.871678 2564 server.go:1019] "Starting client certificate rotation" Jul 07 16:18:11.245796 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.871782 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 16:18:11.245796 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.871818 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 16:18:11.245796 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.874675 2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 16:18:11.245796 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.876589 2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 16:18:11.245796 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.887218 2564 log.go:25] "Validated CRI v1 runtime API" Jul 07 16:18:11.245796 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.890205 2564 log.go:25] "Validated CRI v1 image API" Jul 07 16:18:11.245796 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.891052 2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 16:18:11.245796 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.895956 2564 fs.go:135] Filesystem UUIDs: map[1066ea4c-f6c9-4e93-b4fe-6790e49d97ff:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9061d9dc-3eaf-436b-a29c-1658a99b7d2e:/dev/nvme0n1p3] Jul 07 16:18:11.247075 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.895975 2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 16:18:11.247284 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.902597 2564 manager.go:216] Machine: {Timestamp:2026-07-07 16:18:10.900516367 +0000 UTC m=+0.353864812 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099812 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec218dd4011351df9532900d89a2a405 SystemUUID:ec218dd4-0113-51df-9532-900d89a2a405 BootID:aa2e6ed4-ebc8-4565-80f7-9ce5945b9999 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:48:9b:dc:41:77 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:48:9b:dc:41:77 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:14:88:52:c3:e0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 16:18:11.247284 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.902704 2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 16:18:11.247284 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.902796 2564 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 16:18:11.247284 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.904223 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 16:18:11.247284 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.905915 2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 16:18:11.247284 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.905950 2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-174.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 16:18:11.247284 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.907181 2564 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.907192 2564 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.907200 2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.907804 2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.909129 2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.911971 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.912083 2564 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.912110 2564 kubelet.go:486] "Attempting to sync node with API server" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.912120 2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.912137 2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.912147 2564 kubelet.go:398] "Adding apiserver pod source" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.912156 2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.912633 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.912656 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.916249 2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.918342 2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.918385 2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.921109 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.921129 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.921137 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.921143 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.921152 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.921158 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.921166 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.921171 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.921205 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.921212 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.921227 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.921233 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.921263 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.921271 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:10.923489 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-174.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.923530 2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-174.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:10.923567 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.924474 2564 server.go:1268] "Started kubelet" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.924570 2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.924627 2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.924944 2564 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.927287 2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 16:18:11.248326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.928709 2564 server.go:310] "Adding debug handlers to kubelet server" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:10.930457 2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.934918 2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.934941 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:10.935029 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-174.ec2.internal\" not found" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.935864 2564 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.935867 2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.935876 2564 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.936335 2564 reconstruct.go:97] "Volume reconstruction finished" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.936354 2564 reconciler.go:29] "Reconciler: start to sync state" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.936362 2564 factory.go:55] Registering systemd factory Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.936495 2564 factory.go:223] Registration of the systemd container factory successfully Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:10.935483 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-174.ec2.internal.18c00e391e62bdfc default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-174.ec2.internal,UID:ip-10-0-129-174.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-174.ec2.internal,},FirstTimestamp:2026-07-07 16:18:10.924428796 +0000 UTC m=+0.377777241,LastTimestamp:2026-07-07 16:18:10.924428796 +0000 UTC m=+0.377777241,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-174.ec2.internal,}" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.937254 2564 factory.go:153] Registering CRI-O factory Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.937273 2564 factory.go:223] Registration of the crio container factory successfully Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.937322 2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.937353 2564 factory.go:103] Registering Raw factory Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.937388 2564 manager.go:1196] Started watching for new ooms in manager Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.937825 2564 manager.go:349] Starting recovery of all containers Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:10.947431 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:10.947632 2564 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-174.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.948512 2564 manager.go:354] Recovery completed Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.952473 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.954408 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-174.ec2.internal" event="NodeHasSufficientMemory" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.954450 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-174.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.954460 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-174.ec2.internal" event="NodeHasSufficientPID" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.955015 2564 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.955027 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.955026 2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.955527 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.956126 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-174.ec2.internal" event="NodeHasSufficientMemory" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.956150 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-174.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.956159 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-174.ec2.internal" event="NodeHasSufficientPID" Jul 07 16:18:11.250055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.956739 2564 policy_none.go:49] "None policy: Start" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.956753 2564 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.956763 2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.957654 2564 policy_none.go:47] "Start" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.964128 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-d5r85" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:10.966984 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-174.ec2.internal.18c00e39202c9678 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-174.ec2.internal,UID:ip-10-0-129-174.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-129-174.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-129-174.ec2.internal,},FirstTimestamp:2026-07-07 16:18:10.954434168 +0000 UTC m=+0.407782612,LastTimestamp:2026-07-07 16:18:10.954434168 +0000 UTC m=+0.407782612,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-174.ec2.internal,}" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:10.973692 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-d5r85" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:11.038871 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-174.ec2.internal\" not found" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.050143 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.051287 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.051321 2564 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.051342 2564 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:11.051398 2564 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.054088 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.061990 2564 manager.go:337] "Starting Device Plugin manager" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:11.062023 2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.062035 2564 server.go:87] "Starting device plugin registration server" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.062311 2564 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.062322 2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.062447 2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.062526 2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.062536 2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:11.062931 2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:11.062980 2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-174.ec2.internal\" not found" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.151810 2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-174.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-174.ec2.internal"] Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.151885 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.152951 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-174.ec2.internal" event="NodeHasSufficientMemory" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.152994 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-174.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.153010 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-174.ec2.internal" event="NodeHasSufficientPID" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:11.155046 2564 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-174.ec2.internal\" not found" node="ip-10-0-129-174.ec2.internal" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.155295 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 16:18:11.251814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.155419 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-174.ec2.internal" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.155453 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.156108 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-174.ec2.internal" event="NodeHasSufficientMemory" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.156130 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-174.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.156144 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-174.ec2.internal" event="NodeHasSufficientPID" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.156159 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-174.ec2.internal" event="NodeHasSufficientMemory" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.156182 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-174.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.156192 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-174.ec2.internal" event="NodeHasSufficientPID" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.158284 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-174.ec2.internal" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.158307 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.158989 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-174.ec2.internal" event="NodeHasSufficientMemory" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.159015 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-174.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.159029 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-174.ec2.internal" event="NodeHasSufficientPID" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.162476 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.163259 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-174.ec2.internal" event="NodeHasSufficientMemory" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.163287 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-174.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.163303 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-174.ec2.internal" event="NodeHasSufficientPID" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.163330 2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-174.ec2.internal" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.176592 2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-174.ec2.internal" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:11.176618 2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-174.ec2.internal\": node \"ip-10-0-129-174.ec2.internal\" not found" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:11.184362 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-174.ec2.internal\" not found" node="ip-10-0-129-174.ec2.internal" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:11.208294 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-174.ec2.internal\" not found" node="ip-10-0-129-174.ec2.internal" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:11.212575 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-174.ec2.internal\" not found" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.240026 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e0639c46a9cdb652f922e26862f5abff-config\") pod \"kube-apiserver-proxy-ip-10-0-129-174.ec2.internal\" (UID: \"e0639c46a9cdb652f922e26862f5abff\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-174.ec2.internal" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.240052 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/900907d9d63ddcde1a226020b0a143ac-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-174.ec2.internal\" (UID: \"900907d9d63ddcde1a226020b0a143ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-174.ec2.internal" Jul 07 16:18:11.253146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.240072 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/900907d9d63ddcde1a226020b0a143ac-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-174.ec2.internal\" (UID: \"900907d9d63ddcde1a226020b0a143ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-174.ec2.internal" Jul 07 16:18:11.491876 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:11.312645 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-174.ec2.internal\" not found" Jul 07 16:18:11.491876 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.341054 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/900907d9d63ddcde1a226020b0a143ac-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-174.ec2.internal\" (UID: \"900907d9d63ddcde1a226020b0a143ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-174.ec2.internal" Jul 07 16:18:11.491876 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.341084 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/900907d9d63ddcde1a226020b0a143ac-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-174.ec2.internal\" (UID: \"900907d9d63ddcde1a226020b0a143ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-174.ec2.internal" Jul 07 16:18:11.491876 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.341101 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e0639c46a9cdb652f922e26862f5abff-config\") pod \"kube-apiserver-proxy-ip-10-0-129-174.ec2.internal\" (UID: \"e0639c46a9cdb652f922e26862f5abff\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-174.ec2.internal" Jul 07 16:18:11.491876 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.341138 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e0639c46a9cdb652f922e26862f5abff-config\") pod \"kube-apiserver-proxy-ip-10-0-129-174.ec2.internal\" (UID: \"e0639c46a9cdb652f922e26862f5abff\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-174.ec2.internal" Jul 07 16:18:11.491876 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.341147 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/900907d9d63ddcde1a226020b0a143ac-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-174.ec2.internal\" (UID: \"900907d9d63ddcde1a226020b0a143ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-174.ec2.internal" Jul 07 16:18:11.491876 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.341154 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/900907d9d63ddcde1a226020b0a143ac-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-174.ec2.internal\" (UID: \"900907d9d63ddcde1a226020b0a143ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-174.ec2.internal" Jul 07 16:18:11.491876 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:11.413313 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-174.ec2.internal\" not found" Jul 07 16:18:11.491876 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.486849 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-174.ec2.internal" Jul 07 16:18:11.687096 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:11.511761 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-174.ec2.internal" Jul 07 16:18:11.687096 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:11.513887 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-174.ec2.internal\" not found" Jul 07 16:18:11.687096 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:11.614755 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-174.ec2.internal\" not found" Jul 07 16:18:13.169574 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:11.715247 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-174.ec2.internal\" not found" Jul 07 16:18:13.169574 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:13.107334 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-174.ec2.internal\" not found" Jul 07 16:18:13.169574 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:13.162787 2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 16:18:13.169574 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:13.162979 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 16:18:13.381765 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:13.207509 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-174.ec2.internal\" not found" Jul 07 16:18:13.381765 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:13.267975 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 16:13:12 +0000 UTC" deadline="2028-01-07 23:44:13.291486208 +0000 UTC" Jul 07 16:18:13.381765 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:13.268023 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13183h26m0.023466032s" Jul 07 16:18:13.381765 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:13.281794 2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 16:18:13.381765 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:13.297483 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 16:18:13.381765 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:13.327608 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-174.ec2.internal" Jul 07 16:18:13.381765 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:13.355120 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 16:18:13.381765 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:13.356767 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-174.ec2.internal" Jul 07 16:18:13.381765 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:13.368529 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 16:18:13.381765 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:13.376697 2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 16:18:13.547313 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:13.524894 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod900907d9d63ddcde1a226020b0a143ac.slice/crio-7585fa12afc64c9e1e9242b57614d59f35721eb40bc8a80a0de98b4eb6d65eec WatchSource:0}: Error finding container 7585fa12afc64c9e1e9242b57614d59f35721eb40bc8a80a0de98b4eb6d65eec: Status 404 returned error can't find the container with id 7585fa12afc64c9e1e9242b57614d59f35721eb40bc8a80a0de98b4eb6d65eec Jul 07 16:18:13.547313 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:13.525256 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0639c46a9cdb652f922e26862f5abff.slice/crio-d97f006a3a51c228f0c47013a917acf9c15857896a44f2a3441f588ae5bac89b WatchSource:0}: Error finding container d97f006a3a51c228f0c47013a917acf9c15857896a44f2a3441f588ae5bac89b: Status 404 returned error can't find the container with id d97f006a3a51c228f0c47013a917acf9c15857896a44f2a3441f588ae5bac89b Jul 07 16:18:13.547313 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:13.534832 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 16:18:14.205356 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.205322 2564 apiserver.go:52] "Watching apiserver" Jul 07 16:18:14.218092 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.217567 2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 16:18:14.218558 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.218526 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-hd8f6","openshift-multus/network-metrics-daemon-xsj8b","openshift-network-diagnostics/network-check-target-g9pff","openshift-network-operator/iptables-alerter-wp8sh","openshift-ovn-kubernetes/ovnkube-node-q5hvx","kube-system/konnectivity-agent-mdnxp","kube-system/kube-apiserver-proxy-ip-10-0-129-174.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c","openshift-cluster-node-tuning-operator/tuned-k4sjf","openshift-dns/node-resolver-lxccs","openshift-image-registry/node-ca-vjgrf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-174.ec2.internal","openshift-multus/multus-additional-cni-plugins-8dn9p","kube-system/global-pull-secret-syncer-868j2"] Jul 07 16:18:14.221137 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.221112 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lxccs" Jul 07 16:18:14.223395 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.223373 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.224030 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.223691 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-w5dpt\"" Jul 07 16:18:14.224030 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.223844 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 16:18:14.224030 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.223946 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 16:18:14.225830 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.225585 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 16:18:14.225830 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.225754 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:14.225830 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:14.225813 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9pff" podUID="83eacc13-cd9b-436a-882b-18c36f62e74c" Jul 07 16:18:14.225830 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.225832 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-87226\"" Jul 07 16:18:14.226118 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.226100 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 16:18:14.226175 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.226155 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 16:18:14.226225 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.226175 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 16:18:14.226323 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.226308 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 16:18:14.228277 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.227688 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 16:18:14.228277 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.228151 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wp8sh" Jul 07 16:18:14.230265 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.230248 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.231087 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.230583 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 16:18:14.231158 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.231120 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-r76vg\"" Jul 07 16:18:14.231215 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.231203 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 16:18:14.231375 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.231357 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 16:18:14.232246 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.232226 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 16:18:14.232888 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.232728 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 16:18:14.232888 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.232791 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 16:18:14.232888 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.232799 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x7mpw\"" Jul 07 16:18:14.232888 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.232847 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 16:18:14.233554 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.233423 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mdnxp" Jul 07 16:18:14.234003 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.233826 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 16:18:14.234003 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.233832 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 16:18:14.235999 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.235982 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zc949\"" Jul 07 16:18:14.236274 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.236015 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 16:18:14.236274 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.236149 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.236274 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.236032 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 16:18:14.238442 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.238424 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:14.238545 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:14.238493 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-868j2" podUID="093da19e-76ae-42f9-b23a-5ac90eb50fb1" Jul 07 16:18:14.238752 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.238734 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 16:18:14.238752 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.238748 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-r2xvc\"" Jul 07 16:18:14.238874 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.238835 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 16:18:14.239634 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.239492 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 16:18:14.240235 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240217 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdlhp\" (UniqueName: \"kubernetes.io/projected/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-kube-api-access-kdlhp\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.240336 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240241 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-host-cni-bin\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.240336 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240280 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ks4qc\" (UniqueName: \"kubernetes.io/projected/83dce208-f3dc-4e5c-ba27-e2d6c9472a68-kube-api-access-ks4qc\") pod \"node-resolver-lxccs\" (UID: \"83dce208-f3dc-4e5c-ba27-e2d6c9472a68\") " pod="openshift-dns/node-resolver-lxccs" Jul 07 16:18:14.240336 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240312 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-system-cni-dir\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.240478 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240336 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.240478 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240359 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/683ef4c1-d767-42cb-bcce-a5219be09147-env-overrides\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.240478 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240385 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/83dce208-f3dc-4e5c-ba27-e2d6c9472a68-hosts-file\") pod \"node-resolver-lxccs\" (UID: \"83dce208-f3dc-4e5c-ba27-e2d6c9472a68\") " pod="openshift-dns/node-resolver-lxccs" Jul 07 16:18:14.240478 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240411 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-host-run-ovn-kubernetes\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.240478 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240438 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c41bae1-d8e5-425b-b17c-4efd2fedbb65-host-slash\") pod \"iptables-alerter-wp8sh\" (UID: \"6c41bae1-d8e5-425b-b17c-4efd2fedbb65\") " pod="openshift-network-operator/iptables-alerter-wp8sh" Jul 07 16:18:14.240478 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240463 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/683ef4c1-d767-42cb-bcce-a5219be09147-ovnkube-config\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.240757 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240485 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-var-lib-openvswitch\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.240757 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240510 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-host-kubelet\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.240757 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240540 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-host-run-netns\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.240757 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240577 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-run-systemd\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.240757 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240607 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/683ef4c1-d767-42cb-bcce-a5219be09147-ovn-node-metrics-cert\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.240757 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240631 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83dce208-f3dc-4e5c-ba27-e2d6c9472a68-tmp-dir\") pod \"node-resolver-lxccs\" (UID: \"83dce208-f3dc-4e5c-ba27-e2d6c9472a68\") " pod="openshift-dns/node-resolver-lxccs" Jul 07 16:18:14.240757 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240662 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-cnibin\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.240757 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240702 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-systemd-units\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.240757 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240739 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-etc-openvswitch\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.241165 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240771 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-node-log\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.241165 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240802 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.241165 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240851 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.241165 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240877 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgr9c\" (UniqueName: \"kubernetes.io/projected/683ef4c1-d767-42cb-bcce-a5219be09147-kube-api-access-mgr9c\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.241165 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240924 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggx76\" (UniqueName: \"kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76\") pod \"network-check-target-g9pff\" (UID: \"83eacc13-cd9b-436a-882b-18c36f62e74c\") " pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:14.241165 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240951 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6c41bae1-d8e5-425b-b17c-4efd2fedbb65-iptables-alerter-script\") pod \"iptables-alerter-wp8sh\" (UID: \"6c41bae1-d8e5-425b-b17c-4efd2fedbb65\") " pod="openshift-network-operator/iptables-alerter-wp8sh" Jul 07 16:18:14.241165 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.240982 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-host-slash\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.241165 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.241011 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-run-openvswitch\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.241165 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.241058 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-run-ovn\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.241165 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.241078 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-log-socket\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.241165 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.241117 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-cni-binary-copy\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.241165 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.241142 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gt4q2\" (UniqueName: \"kubernetes.io/projected/6c41bae1-d8e5-425b-b17c-4efd2fedbb65-kube-api-access-gt4q2\") pod \"iptables-alerter-wp8sh\" (UID: \"6c41bae1-d8e5-425b-b17c-4efd2fedbb65\") " pod="openshift-network-operator/iptables-alerter-wp8sh" Jul 07 16:18:14.241165 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.241164 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-host-cni-netd\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.241759 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.241187 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/683ef4c1-d767-42cb-bcce-a5219be09147-ovnkube-script-lib\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.241759 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.241223 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-os-release\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.241759 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.241249 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.241759 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.241405 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:14.241759 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:14.241482 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xsj8b" podUID="97f47a3d-062a-4159-8ebd-96d0afd8372b" Jul 07 16:18:14.243886 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.243867 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.246799 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.246354 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vjgrf" Jul 07 16:18:14.246799 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.246384 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zsrvb\"" Jul 07 16:18:14.247095 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.247075 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 16:18:14.249498 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.249478 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 16:18:14.249996 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.249978 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 16:18:14.250371 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.250351 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 16:18:14.251963 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.251098 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" Jul 07 16:18:14.255024 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.255005 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-g5h8q\"" Jul 07 16:18:14.255157 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.255119 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 16:18:14.255245 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.255166 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 16:18:14.255311 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.255296 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 16:18:14.256576 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.256559 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bt5c7\"" Jul 07 16:18:14.267231 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.267211 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-949kr" Jul 07 16:18:14.275843 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.275821 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-949kr" Jul 07 16:18:14.328921 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.328834 2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 16:18:14.341994 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.341960 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-etc-kubernetes\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.342163 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342004 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-etc-sysctl-d\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.342163 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342029 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-lib-modules\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.342163 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342052 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-cnibin\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.342163 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342082 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c41bae1-d8e5-425b-b17c-4efd2fedbb65-host-slash\") pod \"iptables-alerter-wp8sh\" (UID: \"6c41bae1-d8e5-425b-b17c-4efd2fedbb65\") " pod="openshift-network-operator/iptables-alerter-wp8sh" Jul 07 16:18:14.342163 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342106 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/683ef4c1-d767-42cb-bcce-a5219be09147-ovnkube-config\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.342163 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342129 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa-sys-fs\") pod \"aws-ebs-csi-driver-node-6kr6c\" (UID: \"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" Jul 07 16:18:14.342163 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342155 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret\") pod \"global-pull-secret-syncer-868j2\" (UID: \"093da19e-76ae-42f9-b23a-5ac90eb50fb1\") " pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:14.342500 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342182 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-var-lib-openvswitch\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.342500 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342212 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-run\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.342500 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342234 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/24c71ffc-37a5-48f2-b602-62dff8cf504d-etc-tuned\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.342500 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342270 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8716191e-7e57-4a6a-a3b9-6699f18c4914-multus-daemon-config\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.342500 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342295 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-host-kubelet\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.342500 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342319 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/683ef4c1-d767-42cb-bcce-a5219be09147-ovn-node-metrics-cert\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.342500 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342344 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-cnibin\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.342500 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342387 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n95w\" (UniqueName: \"kubernetes.io/projected/24c71ffc-37a5-48f2-b602-62dff8cf504d-kube-api-access-4n95w\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.342500 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342413 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc7fk\" (UniqueName: \"kubernetes.io/projected/97f47a3d-062a-4159-8ebd-96d0afd8372b-kube-api-access-nc7fk\") pod \"network-metrics-daemon-xsj8b\" (UID: \"97f47a3d-062a-4159-8ebd-96d0afd8372b\") " pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:14.342500 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342433 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6kr6c\" (UID: \"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" Jul 07 16:18:14.342500 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342448 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa-registration-dir\") pod \"aws-ebs-csi-driver-node-6kr6c\" (UID: \"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" Jul 07 16:18:14.342500 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342464 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/093da19e-76ae-42f9-b23a-5ac90eb50fb1-kubelet-config\") pod \"global-pull-secret-syncer-868j2\" (UID: \"093da19e-76ae-42f9-b23a-5ac90eb50fb1\") " pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:14.342500 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342486 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-systemd-units\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.342500 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342501 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-etc-openvswitch\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.343677 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342515 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-node-log\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.343677 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342530 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa-etc-selinux\") pod \"aws-ebs-csi-driver-node-6kr6c\" (UID: \"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" Jul 07 16:18:14.343677 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342544 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-multus-cni-dir\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.343677 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342558 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-os-release\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.343677 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342584 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-run-openvswitch\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.343677 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342605 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-run-ovn\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.343677 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342620 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-log-socket\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.343677 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342640 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdlhp\" (UniqueName: \"kubernetes.io/projected/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-kube-api-access-kdlhp\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.343677 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342666 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-etc-modprobe-d\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.343677 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342684 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-host-var-lib-kubelet\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.343677 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342698 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-host-run-multus-certs\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.343677 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342714 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-host-cni-netd\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.343677 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342746 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/683ef4c1-d767-42cb-bcce-a5219be09147-ovnkube-script-lib\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.343677 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342763 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-os-release\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.344445 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342778 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-etc-systemd\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.344445 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342794 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd8wv\" (UniqueName: \"kubernetes.io/projected/709a00b4-4252-4273-ad18-72ac558f394c-kube-api-access-hd8wv\") pod \"node-ca-vjgrf\" (UID: \"709a00b4-4252-4273-ad18-72ac558f394c\") " pod="openshift-image-registry/node-ca-vjgrf" Jul 07 16:18:14.344445 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342808 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa-device-dir\") pod \"aws-ebs-csi-driver-node-6kr6c\" (UID: \"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" Jul 07 16:18:14.344445 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342822 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-hostroot\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.344445 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342838 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-multus-conf-dir\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.344445 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342856 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-host-cni-bin\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.344445 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342880 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ks4qc\" (UniqueName: \"kubernetes.io/projected/83dce208-f3dc-4e5c-ba27-e2d6c9472a68-kube-api-access-ks4qc\") pod \"node-resolver-lxccs\" (UID: \"83dce208-f3dc-4e5c-ba27-e2d6c9472a68\") " pod="openshift-dns/node-resolver-lxccs" Jul 07 16:18:14.344445 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342916 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/709a00b4-4252-4273-ad18-72ac558f394c-serviceca\") pod \"node-ca-vjgrf\" (UID: \"709a00b4-4252-4273-ad18-72ac558f394c\") " pod="openshift-image-registry/node-ca-vjgrf" Jul 07 16:18:14.344445 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342942 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-etc-kubernetes\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.344445 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342968 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/683ef4c1-d767-42cb-bcce-a5219be09147-env-overrides\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.344445 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342992 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/709a00b4-4252-4273-ad18-72ac558f394c-host\") pod \"node-ca-vjgrf\" (UID: \"709a00b4-4252-4273-ad18-72ac558f394c\") " pod="openshift-image-registry/node-ca-vjgrf" Jul 07 16:18:14.344445 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.342992 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-etc-openvswitch\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.344445 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.343015 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7kbr\" (UniqueName: \"kubernetes.io/projected/df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa-kube-api-access-d7kbr\") pod \"aws-ebs-csi-driver-node-6kr6c\" (UID: \"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" Jul 07 16:18:14.344445 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.343042 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6chh\" (UniqueName: \"kubernetes.io/projected/8716191e-7e57-4a6a-a3b9-6699f18c4914-kube-api-access-z6chh\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.345124 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.343079 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-etc-sysconfig\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.345124 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.343252 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c41bae1-d8e5-425b-b17c-4efd2fedbb65-host-slash\") pod \"iptables-alerter-wp8sh\" (UID: \"6c41bae1-d8e5-425b-b17c-4efd2fedbb65\") " pod="openshift-network-operator/iptables-alerter-wp8sh" Jul 07 16:18:14.345124 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.343682 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/683ef4c1-d767-42cb-bcce-a5219be09147-ovnkube-config\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.345124 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.343757 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-os-release\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.345124 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.343758 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-host-cni-netd\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.345124 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.343782 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-systemd-units\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.345124 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.343803 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-host-cni-bin\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.345124 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.343804 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-host-run-netns\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.345124 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.343832 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-etc-sysctl-conf\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.345124 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.343856 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-sys\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.345124 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.343881 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs\") pod \"network-metrics-daemon-xsj8b\" (UID: \"97f47a3d-062a-4159-8ebd-96d0afd8372b\") " pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:14.345124 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.343933 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-host-run-netns\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.345124 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.343963 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-run-systemd\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.345124 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.343990 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83dce208-f3dc-4e5c-ba27-e2d6c9472a68-tmp-dir\") pod \"node-resolver-lxccs\" (UID: \"83dce208-f3dc-4e5c-ba27-e2d6c9472a68\") " pod="openshift-dns/node-resolver-lxccs" Jul 07 16:18:14.345674 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.344016 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-var-lib-kubelet\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.345674 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.344039 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24c71ffc-37a5-48f2-b602-62dff8cf504d-tmp\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.345674 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.344063 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa-socket-dir\") pod \"aws-ebs-csi-driver-node-6kr6c\" (UID: \"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" Jul 07 16:18:14.345674 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.344089 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8716191e-7e57-4a6a-a3b9-6699f18c4914-cni-binary-copy\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.345674 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.344105 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/683ef4c1-d767-42cb-bcce-a5219be09147-env-overrides\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.345674 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.344117 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.345674 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.344148 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.345674 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.344156 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-cnibin\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.345674 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.344175 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-multus-socket-dir-parent\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.345674 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.344277 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-var-lib-openvswitch\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.345674 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.344321 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-node-log\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.345674 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.344354 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-host-kubelet\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.345674 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.344393 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-run-openvswitch\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.345674 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.344596 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/683ef4c1-d767-42cb-bcce-a5219be09147-ovnkube-script-lib\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.346267 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.344662 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-host-run-netns\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.346267 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.344719 2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 16:18:14.346267 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.344890 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/83dce208-f3dc-4e5c-ba27-e2d6c9472a68-tmp-dir\") pod \"node-resolver-lxccs\" (UID: \"83dce208-f3dc-4e5c-ba27-e2d6c9472a68\") " pod="openshift-dns/node-resolver-lxccs" Jul 07 16:18:14.346267 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.344982 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-run-ovn\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.346267 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.345011 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-log-socket\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.346267 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.345038 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-run-systemd\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.346267 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.345068 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgr9c\" (UniqueName: \"kubernetes.io/projected/683ef4c1-d767-42cb-bcce-a5219be09147-kube-api-access-mgr9c\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.346267 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.345092 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggx76\" (UniqueName: \"kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76\") pod \"network-check-target-g9pff\" (UID: \"83eacc13-cd9b-436a-882b-18c36f62e74c\") " pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:14.346267 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.345115 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6c41bae1-d8e5-425b-b17c-4efd2fedbb65-iptables-alerter-script\") pod \"iptables-alerter-wp8sh\" (UID: \"6c41bae1-d8e5-425b-b17c-4efd2fedbb65\") " pod="openshift-network-operator/iptables-alerter-wp8sh" Jul 07 16:18:14.346267 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.345166 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-host-slash\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.346267 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.345191 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-cni-binary-copy\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.346267 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.345215 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-host-var-lib-cni-bin\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.346267 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.345236 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-host-var-lib-cni-multus\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.346267 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.345259 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gt4q2\" (UniqueName: \"kubernetes.io/projected/6c41bae1-d8e5-425b-b17c-4efd2fedbb65-kube-api-access-gt4q2\") pod \"iptables-alerter-wp8sh\" (UID: \"6c41bae1-d8e5-425b-b17c-4efd2fedbb65\") " pod="openshift-network-operator/iptables-alerter-wp8sh" Jul 07 16:18:14.347005 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.345287 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.347005 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.345310 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/459203a6-cfbe-4739-892a-f6802b684809-agent-certs\") pod \"konnectivity-agent-mdnxp\" (UID: \"459203a6-cfbe-4739-892a-f6802b684809\") " pod="kube-system/konnectivity-agent-mdnxp" Jul 07 16:18:14.347005 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.345330 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-host\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.347005 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.345351 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/093da19e-76ae-42f9-b23a-5ac90eb50fb1-dbus\") pod \"global-pull-secret-syncer-868j2\" (UID: \"093da19e-76ae-42f9-b23a-5ac90eb50fb1\") " pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:14.347005 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.345375 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-system-cni-dir\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.347005 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.345401 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-host-run-k8s-cni-cncf-io\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.347005 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.345518 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-system-cni-dir\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.347005 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.345546 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.347005 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.345573 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/83dce208-f3dc-4e5c-ba27-e2d6c9472a68-hosts-file\") pod \"node-resolver-lxccs\" (UID: \"83dce208-f3dc-4e5c-ba27-e2d6c9472a68\") " pod="openshift-dns/node-resolver-lxccs" Jul 07 16:18:14.347005 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.345604 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-host-run-ovn-kubernetes\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.347005 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.345631 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/459203a6-cfbe-4739-892a-f6802b684809-konnectivity-ca\") pod \"konnectivity-agent-mdnxp\" (UID: \"459203a6-cfbe-4739-892a-f6802b684809\") " pod="kube-system/konnectivity-agent-mdnxp" Jul 07 16:18:14.347005 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.345878 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-host-slash\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.347005 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.345977 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.347005 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.346176 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6c41bae1-d8e5-425b-b17c-4efd2fedbb65-iptables-alerter-script\") pod \"iptables-alerter-wp8sh\" (UID: \"6c41bae1-d8e5-425b-b17c-4efd2fedbb65\") " pod="openshift-network-operator/iptables-alerter-wp8sh" Jul 07 16:18:14.347677 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.346242 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/83dce208-f3dc-4e5c-ba27-e2d6c9472a68-hosts-file\") pod \"node-resolver-lxccs\" (UID: \"83dce208-f3dc-4e5c-ba27-e2d6c9472a68\") " pod="openshift-dns/node-resolver-lxccs" Jul 07 16:18:14.347677 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.346283 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/683ef4c1-d767-42cb-bcce-a5219be09147-host-run-ovn-kubernetes\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.347677 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.346365 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.347677 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.346365 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.347677 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.346425 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-system-cni-dir\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.347677 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.346544 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.347677 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.346699 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-cni-binary-copy\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.350848 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.350715 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-174.ec2.internal" event={"ID":"900907d9d63ddcde1a226020b0a143ac","Type":"ContainerStarted","Data":"7585fa12afc64c9e1e9242b57614d59f35721eb40bc8a80a0de98b4eb6d65eec"} Jul 07 16:18:14.352390 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.352369 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/683ef4c1-d767-42cb-bcce-a5219be09147-ovn-node-metrics-cert\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.354408 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.354339 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-174.ec2.internal" event={"ID":"e0639c46a9cdb652f922e26862f5abff","Type":"ContainerStarted","Data":"d97f006a3a51c228f0c47013a917acf9c15857896a44f2a3441f588ae5bac89b"} Jul 07 16:18:14.356515 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:14.356345 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 16:18:14.356515 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:14.356371 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 16:18:14.356515 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:14.356386 2564 projected.go:196] Error preparing data for projected volume kube-api-access-ggx76 for pod openshift-network-diagnostics/network-check-target-g9pff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 16:18:14.356515 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:14.356464 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76 podName:83eacc13-cd9b-436a-882b-18c36f62e74c nodeName:}" failed. No retries permitted until 2026-07-07 16:18:14.856430679 +0000 UTC m=+3.018243617 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ggx76" (UniqueName: "kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76") pod "network-check-target-g9pff" (UID: "83eacc13-cd9b-436a-882b-18c36f62e74c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 16:18:14.357218 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.356579 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ks4qc\" (UniqueName: \"kubernetes.io/projected/83dce208-f3dc-4e5c-ba27-e2d6c9472a68-kube-api-access-ks4qc\") pod \"node-resolver-lxccs\" (UID: \"83dce208-f3dc-4e5c-ba27-e2d6c9472a68\") " pod="openshift-dns/node-resolver-lxccs" Jul 07 16:18:14.359601 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.359541 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdlhp\" (UniqueName: \"kubernetes.io/projected/7b78b3da-77fb-4414-8154-6e6b3d5f06a1-kube-api-access-kdlhp\") pod \"multus-additional-cni-plugins-8dn9p\" (UID: \"7b78b3da-77fb-4414-8154-6e6b3d5f06a1\") " pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.361002 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.360957 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gt4q2\" (UniqueName: \"kubernetes.io/projected/6c41bae1-d8e5-425b-b17c-4efd2fedbb65-kube-api-access-gt4q2\") pod \"iptables-alerter-wp8sh\" (UID: \"6c41bae1-d8e5-425b-b17c-4efd2fedbb65\") " pod="openshift-network-operator/iptables-alerter-wp8sh" Jul 07 16:18:14.362605 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.362562 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgr9c\" (UniqueName: \"kubernetes.io/projected/683ef4c1-d767-42cb-bcce-a5219be09147-kube-api-access-mgr9c\") pod \"ovnkube-node-q5hvx\" (UID: \"683ef4c1-d767-42cb-bcce-a5219be09147\") " pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.446109 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446072 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-host\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.446293 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446120 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/093da19e-76ae-42f9-b23a-5ac90eb50fb1-dbus\") pod \"global-pull-secret-syncer-868j2\" (UID: \"093da19e-76ae-42f9-b23a-5ac90eb50fb1\") " pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:14.446293 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446144 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-system-cni-dir\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.446293 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446168 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-host-run-k8s-cni-cncf-io\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.446293 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446247 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-host\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.446293 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446251 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-host-run-k8s-cni-cncf-io\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.446540 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446349 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/459203a6-cfbe-4739-892a-f6802b684809-konnectivity-ca\") pod \"konnectivity-agent-mdnxp\" (UID: \"459203a6-cfbe-4739-892a-f6802b684809\") " pod="kube-system/konnectivity-agent-mdnxp" Jul 07 16:18:14.446540 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446383 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-system-cni-dir\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.446540 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446391 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-etc-kubernetes\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.446540 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446415 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-etc-sysctl-d\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.446540 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446438 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-lib-modules\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.446540 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446449 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/093da19e-76ae-42f9-b23a-5ac90eb50fb1-dbus\") pod \"global-pull-secret-syncer-868j2\" (UID: \"093da19e-76ae-42f9-b23a-5ac90eb50fb1\") " pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:14.446540 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446459 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-cnibin\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.446540 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446484 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa-sys-fs\") pod \"aws-ebs-csi-driver-node-6kr6c\" (UID: \"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" Jul 07 16:18:14.446540 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446518 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret\") pod \"global-pull-secret-syncer-868j2\" (UID: \"093da19e-76ae-42f9-b23a-5ac90eb50fb1\") " pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:14.446540 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446539 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-run\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.446540 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446541 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-etc-sysctl-d\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.447099 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446562 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/24c71ffc-37a5-48f2-b602-62dff8cf504d-etc-tuned\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.447099 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446586 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8716191e-7e57-4a6a-a3b9-6699f18c4914-multus-daemon-config\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.447099 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446589 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-etc-kubernetes\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.447099 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446612 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n95w\" (UniqueName: \"kubernetes.io/projected/24c71ffc-37a5-48f2-b602-62dff8cf504d-kube-api-access-4n95w\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.447099 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446635 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc7fk\" (UniqueName: \"kubernetes.io/projected/97f47a3d-062a-4159-8ebd-96d0afd8372b-kube-api-access-nc7fk\") pod \"network-metrics-daemon-xsj8b\" (UID: \"97f47a3d-062a-4159-8ebd-96d0afd8372b\") " pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:14.447099 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446658 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6kr6c\" (UID: \"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" Jul 07 16:18:14.447099 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446681 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa-registration-dir\") pod \"aws-ebs-csi-driver-node-6kr6c\" (UID: \"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" Jul 07 16:18:14.447099 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446703 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/093da19e-76ae-42f9-b23a-5ac90eb50fb1-kubelet-config\") pod \"global-pull-secret-syncer-868j2\" (UID: \"093da19e-76ae-42f9-b23a-5ac90eb50fb1\") " pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:14.447099 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446717 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-lib-modules\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.447099 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446732 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa-etc-selinux\") pod \"aws-ebs-csi-driver-node-6kr6c\" (UID: \"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" Jul 07 16:18:14.447099 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446755 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-multus-cni-dir\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.447099 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446779 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-cnibin\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.447099 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446798 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-os-release\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.447099 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446825 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-etc-modprobe-d\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.447767 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446849 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-host-var-lib-kubelet\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.447767 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446874 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-host-run-multus-certs\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.447767 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446900 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-etc-systemd\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.447767 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446943 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd8wv\" (UniqueName: \"kubernetes.io/projected/709a00b4-4252-4273-ad18-72ac558f394c-kube-api-access-hd8wv\") pod \"node-ca-vjgrf\" (UID: \"709a00b4-4252-4273-ad18-72ac558f394c\") " pod="openshift-image-registry/node-ca-vjgrf" Jul 07 16:18:14.447767 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446955 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/459203a6-cfbe-4739-892a-f6802b684809-konnectivity-ca\") pod \"konnectivity-agent-mdnxp\" (UID: \"459203a6-cfbe-4739-892a-f6802b684809\") " pod="kube-system/konnectivity-agent-mdnxp" Jul 07 16:18:14.447767 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446967 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa-device-dir\") pod \"aws-ebs-csi-driver-node-6kr6c\" (UID: \"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" Jul 07 16:18:14.447767 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446637 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa-sys-fs\") pod \"aws-ebs-csi-driver-node-6kr6c\" (UID: \"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" Jul 07 16:18:14.447767 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.446990 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-hostroot\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.447767 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447012 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-multus-conf-dir\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.447767 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447032 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6kr6c\" (UID: \"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" Jul 07 16:18:14.447767 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447096 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-os-release\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.447767 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447121 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa-etc-selinux\") pod \"aws-ebs-csi-driver-node-6kr6c\" (UID: \"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" Jul 07 16:18:14.447767 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447141 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-run\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.447767 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447176 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/709a00b4-4252-4273-ad18-72ac558f394c-serviceca\") pod \"node-ca-vjgrf\" (UID: \"709a00b4-4252-4273-ad18-72ac558f394c\") " pod="openshift-image-registry/node-ca-vjgrf" Jul 07 16:18:14.447767 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447204 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-etc-kubernetes\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.448484 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447229 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/709a00b4-4252-4273-ad18-72ac558f394c-host\") pod \"node-ca-vjgrf\" (UID: \"709a00b4-4252-4273-ad18-72ac558f394c\") " pod="openshift-image-registry/node-ca-vjgrf" Jul 07 16:18:14.448484 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447259 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7kbr\" (UniqueName: \"kubernetes.io/projected/df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa-kube-api-access-d7kbr\") pod \"aws-ebs-csi-driver-node-6kr6c\" (UID: \"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" Jul 07 16:18:14.448484 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447285 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6chh\" (UniqueName: \"kubernetes.io/projected/8716191e-7e57-4a6a-a3b9-6699f18c4914-kube-api-access-z6chh\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.448484 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447309 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-etc-sysconfig\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.448484 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447335 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-host-run-netns\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.448484 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447358 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-etc-sysctl-conf\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.448484 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447386 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-hostroot\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.448484 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447415 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-etc-systemd\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.448484 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447460 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-sys\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.448484 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447480 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-etc-sysctl-conf\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.448484 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447489 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs\") pod \"network-metrics-daemon-xsj8b\" (UID: \"97f47a3d-062a-4159-8ebd-96d0afd8372b\") " pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:14.448484 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447521 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-var-lib-kubelet\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.448484 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447527 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa-registration-dir\") pod \"aws-ebs-csi-driver-node-6kr6c\" (UID: \"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" Jul 07 16:18:14.448484 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447546 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24c71ffc-37a5-48f2-b602-62dff8cf504d-tmp\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.448484 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447574 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa-socket-dir\") pod \"aws-ebs-csi-driver-node-6kr6c\" (UID: \"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" Jul 07 16:18:14.449208 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447599 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8716191e-7e57-4a6a-a3b9-6699f18c4914-cni-binary-copy\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.449208 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447628 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-multus-socket-dir-parent\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.449208 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447632 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa-device-dir\") pod \"aws-ebs-csi-driver-node-6kr6c\" (UID: \"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" Jul 07 16:18:14.449208 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447673 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-multus-conf-dir\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.449208 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447728 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-multus-cni-dir\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.449208 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447684 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/093da19e-76ae-42f9-b23a-5ac90eb50fb1-kubelet-config\") pod \"global-pull-secret-syncer-868j2\" (UID: \"093da19e-76ae-42f9-b23a-5ac90eb50fb1\") " pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:14.449208 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447813 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-etc-modprobe-d\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.449208 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447850 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-host-var-lib-kubelet\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.449208 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.447887 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-host-run-multus-certs\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.449208 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:14.447980 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 16:18:14.449208 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:14.448044 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs podName:97f47a3d-062a-4159-8ebd-96d0afd8372b nodeName:}" failed. No retries permitted until 2026-07-07 16:18:14.948026046 +0000 UTC m=+3.109839004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs") pod "network-metrics-daemon-xsj8b" (UID: "97f47a3d-062a-4159-8ebd-96d0afd8372b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 16:18:14.449208 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.448129 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/709a00b4-4252-4273-ad18-72ac558f394c-serviceca\") pod \"node-ca-vjgrf\" (UID: \"709a00b4-4252-4273-ad18-72ac558f394c\") " pod="openshift-image-registry/node-ca-vjgrf" Jul 07 16:18:14.449208 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:14.448151 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 16:18:14.449208 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.448177 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-etc-kubernetes\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.449208 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.448216 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-var-lib-kubelet\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.449899 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:14.448228 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret podName:093da19e-76ae-42f9-b23a-5ac90eb50fb1 nodeName:}" failed. No retries permitted until 2026-07-07 16:18:14.948212339 +0000 UTC m=+3.110025293 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret") pod "global-pull-secret-syncer-868j2" (UID: "093da19e-76ae-42f9-b23a-5ac90eb50fb1") : object "kube-system"/"original-pull-secret" not registered Jul 07 16:18:14.449899 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.448218 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/709a00b4-4252-4273-ad18-72ac558f394c-host\") pod \"node-ca-vjgrf\" (UID: \"709a00b4-4252-4273-ad18-72ac558f394c\") " pod="openshift-image-registry/node-ca-vjgrf" Jul 07 16:18:14.449899 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.448251 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-sys\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.449899 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.448271 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-host-run-netns\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.449899 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.448319 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/24c71ffc-37a5-48f2-b602-62dff8cf504d-etc-sysconfig\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.449899 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.448346 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa-socket-dir\") pod \"aws-ebs-csi-driver-node-6kr6c\" (UID: \"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" Jul 07 16:18:14.449899 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.448398 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-host-var-lib-cni-bin\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.449899 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.448425 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-host-var-lib-cni-multus\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.449899 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.448445 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-multus-socket-dir-parent\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.449899 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.448454 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-host-var-lib-cni-bin\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.449899 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.448486 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8716191e-7e57-4a6a-a3b9-6699f18c4914-host-var-lib-cni-multus\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.449899 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.448501 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/459203a6-cfbe-4739-892a-f6802b684809-agent-certs\") pod \"konnectivity-agent-mdnxp\" (UID: \"459203a6-cfbe-4739-892a-f6802b684809\") " pod="kube-system/konnectivity-agent-mdnxp" Jul 07 16:18:14.449899 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.449144 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8716191e-7e57-4a6a-a3b9-6699f18c4914-multus-daemon-config\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.449899 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.449479 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8716191e-7e57-4a6a-a3b9-6699f18c4914-cni-binary-copy\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.450990 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.450967 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24c71ffc-37a5-48f2-b602-62dff8cf504d-tmp\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.451958 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.451897 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/24c71ffc-37a5-48f2-b602-62dff8cf504d-etc-tuned\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.452951 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.452884 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/459203a6-cfbe-4739-892a-f6802b684809-agent-certs\") pod \"konnectivity-agent-mdnxp\" (UID: \"459203a6-cfbe-4739-892a-f6802b684809\") " pod="kube-system/konnectivity-agent-mdnxp" Jul 07 16:18:14.463917 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.463807 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd8wv\" (UniqueName: \"kubernetes.io/projected/709a00b4-4252-4273-ad18-72ac558f394c-kube-api-access-hd8wv\") pod \"node-ca-vjgrf\" (UID: \"709a00b4-4252-4273-ad18-72ac558f394c\") " pod="openshift-image-registry/node-ca-vjgrf" Jul 07 16:18:14.463917 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.463863 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n95w\" (UniqueName: \"kubernetes.io/projected/24c71ffc-37a5-48f2-b602-62dff8cf504d-kube-api-access-4n95w\") pod \"tuned-k4sjf\" (UID: \"24c71ffc-37a5-48f2-b602-62dff8cf504d\") " pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.464368 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.464340 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6chh\" (UniqueName: \"kubernetes.io/projected/8716191e-7e57-4a6a-a3b9-6699f18c4914-kube-api-access-z6chh\") pod \"multus-hd8f6\" (UID: \"8716191e-7e57-4a6a-a3b9-6699f18c4914\") " pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.465395 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.465371 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7kbr\" (UniqueName: \"kubernetes.io/projected/df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa-kube-api-access-d7kbr\") pod \"aws-ebs-csi-driver-node-6kr6c\" (UID: \"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" Jul 07 16:18:14.466632 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.466576 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc7fk\" (UniqueName: \"kubernetes.io/projected/97f47a3d-062a-4159-8ebd-96d0afd8372b-kube-api-access-nc7fk\") pod \"network-metrics-daemon-xsj8b\" (UID: \"97f47a3d-062a-4159-8ebd-96d0afd8372b\") " pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:14.535207 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.535111 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-lxccs" Jul 07 16:18:14.543978 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.543943 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8dn9p" Jul 07 16:18:14.546347 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:14.546317 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83dce208_f3dc_4e5c_ba27_e2d6c9472a68.slice/crio-b66002b0547ece6714bb7d15ae27f2335420ece797e256193246e3707d446048 WatchSource:0}: Error finding container b66002b0547ece6714bb7d15ae27f2335420ece797e256193246e3707d446048: Status 404 returned error can't find the container with id b66002b0547ece6714bb7d15ae27f2335420ece797e256193246e3707d446048 Jul 07 16:18:14.553917 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:14.553877 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b78b3da_77fb_4414_8154_6e6b3d5f06a1.slice/crio-1d007325106dbb5da59940fd1c4edc520e8294169bed9e1aafcc169c3e7f44c6 WatchSource:0}: Error finding container 1d007325106dbb5da59940fd1c4edc520e8294169bed9e1aafcc169c3e7f44c6: Status 404 returned error can't find the container with id 1d007325106dbb5da59940fd1c4edc520e8294169bed9e1aafcc169c3e7f44c6 Jul 07 16:18:14.557177 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.556849 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wp8sh" Jul 07 16:18:14.565577 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.565556 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hd8f6" Jul 07 16:18:14.567276 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:14.567252 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c41bae1_d8e5_425b_b17c_4efd2fedbb65.slice/crio-3c8110e376ca3b7b1cc8897a5ba485d7eff71c8600d6b04fefa3f751ec940aea WatchSource:0}: Error finding container 3c8110e376ca3b7b1cc8897a5ba485d7eff71c8600d6b04fefa3f751ec940aea: Status 404 returned error can't find the container with id 3c8110e376ca3b7b1cc8897a5ba485d7eff71c8600d6b04fefa3f751ec940aea Jul 07 16:18:14.572918 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.572882 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:14.577047 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:14.577019 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8716191e_7e57_4a6a_a3b9_6699f18c4914.slice/crio-77bf55ec9b224dfeaa13d3dc7e55234f51096657175feb61fd65a89dd68fe226 WatchSource:0}: Error finding container 77bf55ec9b224dfeaa13d3dc7e55234f51096657175feb61fd65a89dd68fe226: Status 404 returned error can't find the container with id 77bf55ec9b224dfeaa13d3dc7e55234f51096657175feb61fd65a89dd68fe226 Jul 07 16:18:14.581141 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.581120 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mdnxp" Jul 07 16:18:14.584496 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:14.584462 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod683ef4c1_d767_42cb_bcce_a5219be09147.slice/crio-98dc824c74f1a44f719c773a44cf67e3ec4a4f70412f1d13041eb4e5b8d3a05b WatchSource:0}: Error finding container 98dc824c74f1a44f719c773a44cf67e3ec4a4f70412f1d13041eb4e5b8d3a05b: Status 404 returned error can't find the container with id 98dc824c74f1a44f719c773a44cf67e3ec4a4f70412f1d13041eb4e5b8d3a05b Jul 07 16:18:14.588258 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.588240 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" Jul 07 16:18:14.596814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.596787 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vjgrf" Jul 07 16:18:14.599484 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:14.599453 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod459203a6_cfbe_4739_892a_f6802b684809.slice/crio-193659e5a5ae84ab57a31e2332c4e0c0e5ccd932c328968c62bbeb4a3a688ad6 WatchSource:0}: Error finding container 193659e5a5ae84ab57a31e2332c4e0c0e5ccd932c328968c62bbeb4a3a688ad6: Status 404 returned error can't find the container with id 193659e5a5ae84ab57a31e2332c4e0c0e5ccd932c328968c62bbeb4a3a688ad6 Jul 07 16:18:14.600979 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.600953 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" Jul 07 16:18:14.612866 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:14.612819 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod709a00b4_4252_4273_ad18_72ac558f394c.slice/crio-375d4be5753becb4d1b4f2da6ccfe9761b0c87c1b53bd838c656b269da2e59ae WatchSource:0}: Error finding container 375d4be5753becb4d1b4f2da6ccfe9761b0c87c1b53bd838c656b269da2e59ae: Status 404 returned error can't find the container with id 375d4be5753becb4d1b4f2da6ccfe9761b0c87c1b53bd838c656b269da2e59ae Jul 07 16:18:14.615051 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:14.615015 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf472d8b_ea1c_4a0b_a0cf_b69713e0b9fa.slice/crio-165e74e404f173a354f4e32bfd74f416051037c62100d55a9883500f6cfaf3c1 WatchSource:0}: Error finding container 165e74e404f173a354f4e32bfd74f416051037c62100d55a9883500f6cfaf3c1: Status 404 returned error can't find the container with id 165e74e404f173a354f4e32bfd74f416051037c62100d55a9883500f6cfaf3c1 Jul 07 16:18:14.634634 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.634607 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 16:18:14.951638 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.951518 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs\") pod \"network-metrics-daemon-xsj8b\" (UID: \"97f47a3d-062a-4159-8ebd-96d0afd8372b\") " pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:14.951813 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:14.951683 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 16:18:14.951813 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.951710 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggx76\" (UniqueName: \"kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76\") pod \"network-check-target-g9pff\" (UID: \"83eacc13-cd9b-436a-882b-18c36f62e74c\") " pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:14.951813 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:14.951745 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs podName:97f47a3d-062a-4159-8ebd-96d0afd8372b nodeName:}" failed. No retries permitted until 2026-07-07 16:18:15.951728992 +0000 UTC m=+4.113541925 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs") pod "network-metrics-daemon-xsj8b" (UID: "97f47a3d-062a-4159-8ebd-96d0afd8372b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 16:18:14.951813 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:14.951788 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret\") pod \"global-pull-secret-syncer-868j2\" (UID: \"093da19e-76ae-42f9-b23a-5ac90eb50fb1\") " pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:14.952041 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:14.951832 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 16:18:14.952041 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:14.951849 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 16:18:14.952041 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:14.951859 2564 projected.go:196] Error preparing data for projected volume kube-api-access-ggx76 for pod openshift-network-diagnostics/network-check-target-g9pff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 16:18:14.952041 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:14.951861 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 16:18:14.952041 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:14.951901 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret podName:093da19e-76ae-42f9-b23a-5ac90eb50fb1 nodeName:}" failed. No retries permitted until 2026-07-07 16:18:15.951889666 +0000 UTC m=+4.113702601 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret") pod "global-pull-secret-syncer-868j2" (UID: "093da19e-76ae-42f9-b23a-5ac90eb50fb1") : object "kube-system"/"original-pull-secret" not registered Jul 07 16:18:14.952041 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:14.951937 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76 podName:83eacc13-cd9b-436a-882b-18c36f62e74c nodeName:}" failed. No retries permitted until 2026-07-07 16:18:15.951924868 +0000 UTC m=+4.113737801 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggx76" (UniqueName: "kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76") pod "network-check-target-g9pff" (UID: "83eacc13-cd9b-436a-882b-18c36f62e74c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 16:18:15.279175 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:15.279066 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 16:13:14 +0000 UTC" deadline="2028-02-12 10:45:24.687029812 +0000 UTC" Jul 07 16:18:15.279175 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:15.279124 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14034h27m9.407911208s" Jul 07 16:18:15.357051 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:15.357015 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vjgrf" event={"ID":"709a00b4-4252-4273-ad18-72ac558f394c","Type":"ContainerStarted","Data":"375d4be5753becb4d1b4f2da6ccfe9761b0c87c1b53bd838c656b269da2e59ae"} Jul 07 16:18:15.358341 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:15.358300 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" event={"ID":"24c71ffc-37a5-48f2-b602-62dff8cf504d","Type":"ContainerStarted","Data":"6a8b42a2a6409c84c6173926966b055b1df518e1636d6903e68ceeb9d01413de"} Jul 07 16:18:15.359503 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:15.359452 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mdnxp" event={"ID":"459203a6-cfbe-4739-892a-f6802b684809","Type":"ContainerStarted","Data":"193659e5a5ae84ab57a31e2332c4e0c0e5ccd932c328968c62bbeb4a3a688ad6"} Jul 07 16:18:15.360865 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:15.360779 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" event={"ID":"683ef4c1-d767-42cb-bcce-a5219be09147","Type":"ContainerStarted","Data":"98dc824c74f1a44f719c773a44cf67e3ec4a4f70412f1d13041eb4e5b8d3a05b"} Jul 07 16:18:15.362196 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:15.362171 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hd8f6" event={"ID":"8716191e-7e57-4a6a-a3b9-6699f18c4914","Type":"ContainerStarted","Data":"77bf55ec9b224dfeaa13d3dc7e55234f51096657175feb61fd65a89dd68fe226"} Jul 07 16:18:15.364382 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:15.364337 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lxccs" event={"ID":"83dce208-f3dc-4e5c-ba27-e2d6c9472a68","Type":"ContainerStarted","Data":"b66002b0547ece6714bb7d15ae27f2335420ece797e256193246e3707d446048"} Jul 07 16:18:15.366357 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:15.366324 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" event={"ID":"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa","Type":"ContainerStarted","Data":"165e74e404f173a354f4e32bfd74f416051037c62100d55a9883500f6cfaf3c1"} Jul 07 16:18:15.368681 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:15.368614 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wp8sh" event={"ID":"6c41bae1-d8e5-425b-b17c-4efd2fedbb65","Type":"ContainerStarted","Data":"3c8110e376ca3b7b1cc8897a5ba485d7eff71c8600d6b04fefa3f751ec940aea"} Jul 07 16:18:15.369962 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:15.369935 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8dn9p" event={"ID":"7b78b3da-77fb-4414-8154-6e6b3d5f06a1","Type":"ContainerStarted","Data":"1d007325106dbb5da59940fd1c4edc520e8294169bed9e1aafcc169c3e7f44c6"} Jul 07 16:18:15.961855 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:15.960810 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs\") pod \"network-metrics-daemon-xsj8b\" (UID: \"97f47a3d-062a-4159-8ebd-96d0afd8372b\") " pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:15.961855 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:15.960936 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggx76\" (UniqueName: \"kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76\") pod \"network-check-target-g9pff\" (UID: \"83eacc13-cd9b-436a-882b-18c36f62e74c\") " pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:15.961855 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:15.960985 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret\") pod \"global-pull-secret-syncer-868j2\" (UID: \"093da19e-76ae-42f9-b23a-5ac90eb50fb1\") " pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:15.961855 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:15.961137 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 16:18:15.961855 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:15.961200 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret podName:093da19e-76ae-42f9-b23a-5ac90eb50fb1 nodeName:}" failed. No retries permitted until 2026-07-07 16:18:17.961183503 +0000 UTC m=+6.122996440 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret") pod "global-pull-secret-syncer-868j2" (UID: "093da19e-76ae-42f9-b23a-5ac90eb50fb1") : object "kube-system"/"original-pull-secret" not registered Jul 07 16:18:15.961855 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:15.961595 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 16:18:15.961855 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:15.961645 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs podName:97f47a3d-062a-4159-8ebd-96d0afd8372b nodeName:}" failed. No retries permitted until 2026-07-07 16:18:17.961629594 +0000 UTC m=+6.123442530 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs") pod "network-metrics-daemon-xsj8b" (UID: "97f47a3d-062a-4159-8ebd-96d0afd8372b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 16:18:15.961855 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:15.961733 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 16:18:15.961855 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:15.961772 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 16:18:15.961855 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:15.961784 2564 projected.go:196] Error preparing data for projected volume kube-api-access-ggx76 for pod openshift-network-diagnostics/network-check-target-g9pff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 16:18:15.961855 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:15.961819 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76 podName:83eacc13-cd9b-436a-882b-18c36f62e74c nodeName:}" failed. No retries permitted until 2026-07-07 16:18:17.961808058 +0000 UTC m=+6.123620994 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggx76" (UniqueName: "kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76") pod "network-check-target-g9pff" (UID: "83eacc13-cd9b-436a-882b-18c36f62e74c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 16:18:16.279711 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:16.279647 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 16:13:14 +0000 UTC" deadline="2027-12-22 08:11:49.333399033 +0000 UTC" Jul 07 16:18:16.279711 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:16.279710 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12783h53m33.053693389s" Jul 07 16:18:16.347169 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:16.346242 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:16.347169 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:16.346396 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-868j2" podUID="093da19e-76ae-42f9-b23a-5ac90eb50fb1" Jul 07 16:18:16.347169 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:16.346773 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:16.347169 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:16.346865 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9pff" podUID="83eacc13-cd9b-436a-882b-18c36f62e74c" Jul 07 16:18:16.347169 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:16.346947 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:16.347169 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:16.347023 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xsj8b" podUID="97f47a3d-062a-4159-8ebd-96d0afd8372b" Jul 07 16:18:16.379203 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:16.378332 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-174.ec2.internal" event={"ID":"e0639c46a9cdb652f922e26862f5abff","Type":"ContainerStarted","Data":"524d1491ee51892355cbc05dff092d20d8298f4c1830ebbf93e17fd1c7387825"} Jul 07 16:18:17.397026 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:17.396983 2564 generic.go:356] "Generic (PLEG): container finished" podID="900907d9d63ddcde1a226020b0a143ac" containerID="280570ea706aaab3e92265099ef6e385bdc464bab31e401cdba034cf6a1e59e3" exitCode=0 Jul 07 16:18:17.397546 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:17.397520 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-174.ec2.internal" event={"ID":"900907d9d63ddcde1a226020b0a143ac","Type":"ContainerDied","Data":"280570ea706aaab3e92265099ef6e385bdc464bab31e401cdba034cf6a1e59e3"} Jul 07 16:18:17.415279 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:17.415164 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-174.ec2.internal" podStartSLOduration=4.415141797 podStartE2EDuration="4.415141797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 16:18:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:18:16.39493286 +0000 UTC m=+4.556745816" watchObservedRunningTime="2026-07-07 16:18:17.415141797 +0000 UTC m=+5.576954753" Jul 07 16:18:17.981356 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:17.980451 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret\") pod \"global-pull-secret-syncer-868j2\" (UID: \"093da19e-76ae-42f9-b23a-5ac90eb50fb1\") " pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:17.981356 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:17.980530 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs\") pod \"network-metrics-daemon-xsj8b\" (UID: \"97f47a3d-062a-4159-8ebd-96d0afd8372b\") " pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:17.981356 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:17.980567 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggx76\" (UniqueName: \"kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76\") pod \"network-check-target-g9pff\" (UID: \"83eacc13-cd9b-436a-882b-18c36f62e74c\") " pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:17.981356 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:17.980714 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 16:18:17.981356 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:17.980732 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 16:18:17.981356 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:17.980744 2564 projected.go:196] Error preparing data for projected volume kube-api-access-ggx76 for pod openshift-network-diagnostics/network-check-target-g9pff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 16:18:17.981356 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:17.980802 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76 podName:83eacc13-cd9b-436a-882b-18c36f62e74c nodeName:}" failed. No retries permitted until 2026-07-07 16:18:21.980783806 +0000 UTC m=+10.142596743 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggx76" (UniqueName: "kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76") pod "network-check-target-g9pff" (UID: "83eacc13-cd9b-436a-882b-18c36f62e74c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 16:18:17.981356 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:17.981197 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 16:18:17.981356 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:17.981241 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret podName:093da19e-76ae-42f9-b23a-5ac90eb50fb1 nodeName:}" failed. No retries permitted until 2026-07-07 16:18:21.981227408 +0000 UTC m=+10.143040345 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret") pod "global-pull-secret-syncer-868j2" (UID: "093da19e-76ae-42f9-b23a-5ac90eb50fb1") : object "kube-system"/"original-pull-secret" not registered Jul 07 16:18:17.981356 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:17.981299 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 16:18:17.981356 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:17.981331 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs podName:97f47a3d-062a-4159-8ebd-96d0afd8372b nodeName:}" failed. No retries permitted until 2026-07-07 16:18:21.981319894 +0000 UTC m=+10.143132832 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs") pod "network-metrics-daemon-xsj8b" (UID: "97f47a3d-062a-4159-8ebd-96d0afd8372b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 16:18:18.344675 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:18.344141 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:18.344675 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:18.344272 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-868j2" podUID="093da19e-76ae-42f9-b23a-5ac90eb50fb1" Jul 07 16:18:18.344675 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:18.344348 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:18.344675 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:18.344458 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9pff" podUID="83eacc13-cd9b-436a-882b-18c36f62e74c" Jul 07 16:18:18.344675 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:18.344506 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:18.344675 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:18.344586 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xsj8b" podUID="97f47a3d-062a-4159-8ebd-96d0afd8372b" Jul 07 16:18:20.343955 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:20.343898 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:20.344393 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:20.343897 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:20.344393 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:20.344059 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:20.344393 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:20.344048 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9pff" podUID="83eacc13-cd9b-436a-882b-18c36f62e74c" Jul 07 16:18:20.344393 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:20.344215 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-868j2" podUID="093da19e-76ae-42f9-b23a-5ac90eb50fb1" Jul 07 16:18:20.344393 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:20.344309 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xsj8b" podUID="97f47a3d-062a-4159-8ebd-96d0afd8372b" Jul 07 16:18:22.012366 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:22.012326 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs\") pod \"network-metrics-daemon-xsj8b\" (UID: \"97f47a3d-062a-4159-8ebd-96d0afd8372b\") " pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:22.012851 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:22.012386 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggx76\" (UniqueName: \"kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76\") pod \"network-check-target-g9pff\" (UID: \"83eacc13-cd9b-436a-882b-18c36f62e74c\") " pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:22.012851 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:22.012422 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret\") pod \"global-pull-secret-syncer-868j2\" (UID: \"093da19e-76ae-42f9-b23a-5ac90eb50fb1\") " pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:22.012851 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:22.012503 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 16:18:22.012851 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:22.012539 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 16:18:22.012851 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:22.012562 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 16:18:22.012851 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:22.012582 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 16:18:22.012851 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:22.012587 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs podName:97f47a3d-062a-4159-8ebd-96d0afd8372b nodeName:}" failed. No retries permitted until 2026-07-07 16:18:30.012566122 +0000 UTC m=+18.174379057 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs") pod "network-metrics-daemon-xsj8b" (UID: "97f47a3d-062a-4159-8ebd-96d0afd8372b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 16:18:22.012851 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:22.012595 2564 projected.go:196] Error preparing data for projected volume kube-api-access-ggx76 for pod openshift-network-diagnostics/network-check-target-g9pff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 16:18:22.012851 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:22.012609 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret podName:093da19e-76ae-42f9-b23a-5ac90eb50fb1 nodeName:}" failed. No retries permitted until 2026-07-07 16:18:30.012598571 +0000 UTC m=+18.174411511 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret") pod "global-pull-secret-syncer-868j2" (UID: "093da19e-76ae-42f9-b23a-5ac90eb50fb1") : object "kube-system"/"original-pull-secret" not registered Jul 07 16:18:22.012851 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:22.012638 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76 podName:83eacc13-cd9b-436a-882b-18c36f62e74c nodeName:}" failed. No retries permitted until 2026-07-07 16:18:30.012623589 +0000 UTC m=+18.174436549 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggx76" (UniqueName: "kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76") pod "network-check-target-g9pff" (UID: "83eacc13-cd9b-436a-882b-18c36f62e74c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 16:18:22.344552 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:22.344463 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:22.344704 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:22.344583 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xsj8b" podUID="97f47a3d-062a-4159-8ebd-96d0afd8372b" Jul 07 16:18:22.344828 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:22.344804 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:22.344992 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:22.344897 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:22.344992 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:22.344945 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-868j2" podUID="093da19e-76ae-42f9-b23a-5ac90eb50fb1" Jul 07 16:18:22.345116 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:22.344974 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9pff" podUID="83eacc13-cd9b-436a-882b-18c36f62e74c" Jul 07 16:18:24.343303 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:24.343275 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:24.343651 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:24.343275 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:24.343651 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:24.343408 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-868j2" podUID="093da19e-76ae-42f9-b23a-5ac90eb50fb1" Jul 07 16:18:24.343651 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:24.343493 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9pff" podUID="83eacc13-cd9b-436a-882b-18c36f62e74c" Jul 07 16:18:24.343651 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:24.343288 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:24.343651 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:24.343599 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xsj8b" podUID="97f47a3d-062a-4159-8ebd-96d0afd8372b" Jul 07 16:18:25.416083 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:25.416036 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vjgrf" event={"ID":"709a00b4-4252-4273-ad18-72ac558f394c","Type":"ContainerStarted","Data":"bc51b9e1af35652f00fc90e3eed3e15a8039a9d8c20c50890ff7a7ea082a1d15"} Jul 07 16:18:25.418075 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:25.418047 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" event={"ID":"24c71ffc-37a5-48f2-b602-62dff8cf504d","Type":"ContainerStarted","Data":"da777ccb924b7dee40b3c391deb391ea8f620aff2c0016003cb9f08345832744"} Jul 07 16:18:25.419633 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:25.419603 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mdnxp" event={"ID":"459203a6-cfbe-4739-892a-f6802b684809","Type":"ContainerStarted","Data":"a543958432aa323969fd6c2187c5197e591c4aca8a76352de38d8161c10841b9"} Jul 07 16:18:25.421139 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:25.421117 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-lxccs" event={"ID":"83dce208-f3dc-4e5c-ba27-e2d6c9472a68","Type":"ContainerStarted","Data":"b594c227f67cb046b5685fe0886bad71779189882e985bef6b3c434485b18216"} Jul 07 16:18:25.422957 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:25.422929 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" event={"ID":"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa","Type":"ContainerStarted","Data":"60351310c73a113822b0022ad664ed6adb8ddcd1f7b357f8081530bc858f2e58"} Jul 07 16:18:25.426531 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:25.426498 2564 generic.go:356] "Generic (PLEG): container finished" podID="7b78b3da-77fb-4414-8154-6e6b3d5f06a1" containerID="efd32692feac9507ba742fc64f3941bf04014a520ff01fdedf6047eb0e1be6f9" exitCode=0 Jul 07 16:18:25.426689 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:25.426614 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8dn9p" event={"ID":"7b78b3da-77fb-4414-8154-6e6b3d5f06a1","Type":"ContainerDied","Data":"efd32692feac9507ba742fc64f3941bf04014a520ff01fdedf6047eb0e1be6f9"} Jul 07 16:18:25.429827 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:25.429800 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-174.ec2.internal" event={"ID":"900907d9d63ddcde1a226020b0a143ac","Type":"ContainerStarted","Data":"7a39b759ed6781d0e312e22d841d9ccb8a76087ab08082fa9c85ae20324d9757"} Jul 07 16:18:25.435338 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:25.435293 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vjgrf" podStartSLOduration=3.742803784 podStartE2EDuration="13.435280419s" totalImagesPullingTime="9.692476635s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 16:18:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:18:25.434758757 +0000 UTC m=+13.596571713" watchObservedRunningTime="2026-07-07 16:18:25.435280419 +0000 UTC m=+13.597093373" Jul 07 16:18:25.449523 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:25.449473 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mdnxp" podStartSLOduration=3.741575427 podStartE2EDuration="13.449446736s" totalImagesPullingTime="9.707871309s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:18:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:18:25.449226137 +0000 UTC m=+13.611039094" watchObservedRunningTime="2026-07-07 16:18:25.449446736 +0000 UTC m=+13.611259692" Jul 07 16:18:25.465549 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:25.465372 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-lxccs" podStartSLOduration=3.711084269 podStartE2EDuration="13.465351021s" totalImagesPullingTime="9.754266752s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 16:18:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:18:25.464868326 +0000 UTC m=+13.626681284" watchObservedRunningTime="2026-07-07 16:18:25.465351021 +0000 UTC m=+13.627163977" Jul 07 16:18:25.509682 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:25.509629 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-k4sjf" podStartSLOduration=3.685875676 podStartE2EDuration="13.509614489s" totalImagesPullingTime="9.823738813s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 16:18:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:18:25.509119989 +0000 UTC m=+13.670932944" watchObservedRunningTime="2026-07-07 16:18:25.509614489 +0000 UTC m=+13.671427443" Jul 07 16:18:25.529068 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:25.528999 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-174.ec2.internal" podStartSLOduration=12.528975903 podStartE2EDuration="12.528975903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 16:18:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:18:25.528131265 +0000 UTC m=+13.689944223" watchObservedRunningTime="2026-07-07 16:18:25.528975903 +0000 UTC m=+13.690788861" Jul 07 16:18:26.344047 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:26.343938 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:26.344047 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:26.343983 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:26.344367 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:26.344075 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9pff" podUID="83eacc13-cd9b-436a-882b-18c36f62e74c" Jul 07 16:18:26.344434 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:26.343938 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:26.344644 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:26.344537 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xsj8b" podUID="97f47a3d-062a-4159-8ebd-96d0afd8372b" Jul 07 16:18:26.344644 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:26.344616 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-868j2" podUID="093da19e-76ae-42f9-b23a-5ac90eb50fb1" Jul 07 16:18:26.433965 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:26.433884 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wp8sh" event={"ID":"6c41bae1-d8e5-425b-b17c-4efd2fedbb65","Type":"ContainerStarted","Data":"b9ea679eb924ba65e087f7ca1ca99babc4c3b2a5703d1f938220c6c00cce0592"} Jul 07 16:18:26.449649 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:26.449589 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wp8sh" podStartSLOduration=4.717889328 podStartE2EDuration="14.449571229s" totalImagesPullingTime="9.731681901s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 16:18:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:18:26.44944808 +0000 UTC m=+14.611261036" watchObservedRunningTime="2026-07-07 16:18:26.449571229 +0000 UTC m=+14.611384184" Jul 07 16:18:27.799562 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:27.799520 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mdnxp" Jul 07 16:18:27.800453 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:27.800430 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mdnxp" Jul 07 16:18:28.343777 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:28.343724 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:28.343777 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:28.343762 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:28.344065 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:28.343762 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:28.344065 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:28.343870 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xsj8b" podUID="97f47a3d-062a-4159-8ebd-96d0afd8372b" Jul 07 16:18:28.344065 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:28.343961 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-868j2" podUID="093da19e-76ae-42f9-b23a-5ac90eb50fb1" Jul 07 16:18:28.344065 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:28.344050 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9pff" podUID="83eacc13-cd9b-436a-882b-18c36f62e74c" Jul 07 16:18:30.076304 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:30.076161 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs\") pod \"network-metrics-daemon-xsj8b\" (UID: \"97f47a3d-062a-4159-8ebd-96d0afd8372b\") " pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:30.076304 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:30.076244 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggx76\" (UniqueName: \"kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76\") pod \"network-check-target-g9pff\" (UID: \"83eacc13-cd9b-436a-882b-18c36f62e74c\") " pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:30.077210 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:30.076356 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret\") pod \"global-pull-secret-syncer-868j2\" (UID: \"093da19e-76ae-42f9-b23a-5ac90eb50fb1\") " pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:30.077210 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:30.076514 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 16:18:30.077210 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:30.076578 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret podName:093da19e-76ae-42f9-b23a-5ac90eb50fb1 nodeName:}" failed. No retries permitted until 2026-07-07 16:18:46.076560051 +0000 UTC m=+34.238372984 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret") pod "global-pull-secret-syncer-868j2" (UID: "093da19e-76ae-42f9-b23a-5ac90eb50fb1") : object "kube-system"/"original-pull-secret" not registered Jul 07 16:18:30.077210 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:30.076901 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 16:18:30.077210 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:30.076939 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 16:18:30.077210 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:30.076953 2564 projected.go:196] Error preparing data for projected volume kube-api-access-ggx76 for pod openshift-network-diagnostics/network-check-target-g9pff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 16:18:30.077210 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:30.076988 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 16:18:30.077210 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:30.077005 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76 podName:83eacc13-cd9b-436a-882b-18c36f62e74c nodeName:}" failed. No retries permitted until 2026-07-07 16:18:46.07698832 +0000 UTC m=+34.238801268 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggx76" (UniqueName: "kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76") pod "network-check-target-g9pff" (UID: "83eacc13-cd9b-436a-882b-18c36f62e74c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 16:18:30.077210 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:30.077041 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs podName:97f47a3d-062a-4159-8ebd-96d0afd8372b nodeName:}" failed. No retries permitted until 2026-07-07 16:18:46.07702534 +0000 UTC m=+34.238838295 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs") pod "network-metrics-daemon-xsj8b" (UID: "97f47a3d-062a-4159-8ebd-96d0afd8372b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 16:18:30.344732 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:30.344009 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:30.344732 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:30.344146 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xsj8b" podUID="97f47a3d-062a-4159-8ebd-96d0afd8372b" Jul 07 16:18:30.344732 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:30.344528 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:30.344732 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:30.344621 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-868j2" podUID="093da19e-76ae-42f9-b23a-5ac90eb50fb1" Jul 07 16:18:30.345140 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:30.345116 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:30.345286 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:30.345207 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9pff" podUID="83eacc13-cd9b-436a-882b-18c36f62e74c" Jul 07 16:18:32.345037 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:32.344797 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:32.345515 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:32.344880 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:32.345515 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:32.345115 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9pff" podUID="83eacc13-cd9b-436a-882b-18c36f62e74c" Jul 07 16:18:32.345515 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:32.344922 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:32.345515 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:32.345217 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xsj8b" podUID="97f47a3d-062a-4159-8ebd-96d0afd8372b" Jul 07 16:18:32.345515 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:32.345323 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-868j2" podUID="093da19e-76ae-42f9-b23a-5ac90eb50fb1" Jul 07 16:18:33.654485 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:33.654431 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mdnxp" Jul 07 16:18:33.654949 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:33.654601 2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 07 16:18:33.655439 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:33.655409 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mdnxp" Jul 07 16:18:34.346640 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:34.346611 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:34.346812 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:34.346611 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:34.346812 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:34.346717 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-868j2" podUID="093da19e-76ae-42f9-b23a-5ac90eb50fb1" Jul 07 16:18:34.346929 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:34.346814 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xsj8b" podUID="97f47a3d-062a-4159-8ebd-96d0afd8372b" Jul 07 16:18:34.346929 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:34.346611 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:34.346929 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:34.346919 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9pff" podUID="83eacc13-cd9b-436a-882b-18c36f62e74c" Jul 07 16:18:36.343302 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:36.343268 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:36.343831 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:36.343276 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:36.343831 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:36.343389 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9pff" podUID="83eacc13-cd9b-436a-882b-18c36f62e74c" Jul 07 16:18:36.343831 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:36.343468 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xsj8b" podUID="97f47a3d-062a-4159-8ebd-96d0afd8372b" Jul 07 16:18:36.343831 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:36.343498 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:36.343831 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:36.343568 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-868j2" podUID="093da19e-76ae-42f9-b23a-5ac90eb50fb1" Jul 07 16:18:36.998590 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:36.998564 2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 16:18:37.366049 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:37.365804 2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T16:18:36.998584093Z","UUID":"5653eac9-8c98-4f2a-879d-e33e977890ac","Handler":null,"Name":"","Endpoint":""} Jul 07 16:18:37.367174 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:37.367156 2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 16:18:37.367269 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:37.367190 2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 16:18:37.460121 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:37.460084 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" event={"ID":"683ef4c1-d767-42cb-bcce-a5219be09147","Type":"ContainerStarted","Data":"937f8972feb451aaa245d7ed5bc1866bc508cb5e21e4fc969e015a224d5208d9"} Jul 07 16:18:37.460264 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:37.460184 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" event={"ID":"683ef4c1-d767-42cb-bcce-a5219be09147","Type":"ContainerStarted","Data":"e8d376ee734b049f4aa9d3494f467d104240f33de5ae0f36e37a70c5b3d7bd49"} Jul 07 16:18:37.460264 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:37.460220 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" event={"ID":"683ef4c1-d767-42cb-bcce-a5219be09147","Type":"ContainerStarted","Data":"c710211e18c4316153e007f09fbbae10abcebd7b9c0adaf9bd614e340664f299"} Jul 07 16:18:37.460264 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:37.460249 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" event={"ID":"683ef4c1-d767-42cb-bcce-a5219be09147","Type":"ContainerStarted","Data":"01317cf3cc52d2e883e3e52c21de5e610a4feab5fdc809e252d56a2830c846b3"} Jul 07 16:18:37.460437 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:37.460284 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" event={"ID":"683ef4c1-d767-42cb-bcce-a5219be09147","Type":"ContainerStarted","Data":"ccfb2ea6a8421855faad9a434a1f363cda50b7d8773c98b5ebed0ab8f5a660e6"} Jul 07 16:18:37.461360 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:37.461334 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hd8f6" event={"ID":"8716191e-7e57-4a6a-a3b9-6699f18c4914","Type":"ContainerStarted","Data":"ff4171890b5bdb456dfc29a17e628fc1d4739feb1b659a4f9e18a8bc7d720b32"} Jul 07 16:18:37.463012 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:37.462992 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" event={"ID":"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa","Type":"ContainerStarted","Data":"99e0a12aa15532900b16f253a92b2aeb78722ee944478c8b518118fa7f6f5f18"} Jul 07 16:18:37.464646 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:37.464626 2564 generic.go:356] "Generic (PLEG): container finished" podID="7b78b3da-77fb-4414-8154-6e6b3d5f06a1" containerID="b92bb733a10d3777728947c0e0dd41bfc373d7fe41d36b12c8cce5ac8a5a278b" exitCode=0 Jul 07 16:18:37.464748 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:37.464651 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8dn9p" event={"ID":"7b78b3da-77fb-4414-8154-6e6b3d5f06a1","Type":"ContainerDied","Data":"b92bb733a10d3777728947c0e0dd41bfc373d7fe41d36b12c8cce5ac8a5a278b"} Jul 07 16:18:37.480698 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:37.480652 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-hd8f6" podStartSLOduration=2.967128395 podStartE2EDuration="25.48063785s" totalImagesPullingTime="22.513509455s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 16:18:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:18:37.480104276 +0000 UTC m=+25.641917231" watchObservedRunningTime="2026-07-07 16:18:37.48063785 +0000 UTC m=+25.642450804" Jul 07 16:18:38.343614 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:38.343577 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:38.343927 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:38.343579 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:38.343927 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:38.343692 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xsj8b" podUID="97f47a3d-062a-4159-8ebd-96d0afd8372b" Jul 07 16:18:38.343927 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:38.343584 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:38.343927 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:38.343752 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-868j2" podUID="093da19e-76ae-42f9-b23a-5ac90eb50fb1" Jul 07 16:18:38.343927 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:38.343828 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9pff" podUID="83eacc13-cd9b-436a-882b-18c36f62e74c" Jul 07 16:18:38.469885 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:38.469849 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" event={"ID":"683ef4c1-d767-42cb-bcce-a5219be09147","Type":"ContainerStarted","Data":"b20a31a6d90d911c695d442d2fa2966e82a9c60d5754910da86290307a363924"} Jul 07 16:18:39.473657 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:39.473564 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" event={"ID":"df472d8b-ea1c-4a0b-a0cf-b69713e0b9fa","Type":"ContainerStarted","Data":"c868d49a350a157ae96928076513a2fb107ee9d049e1c08080f8936dbc56f302"} Jul 07 16:18:39.475530 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:39.475506 2564 generic.go:356] "Generic (PLEG): container finished" podID="7b78b3da-77fb-4414-8154-6e6b3d5f06a1" containerID="3ab128dac9a860b51a9b07cd1186c7a745609f52cb911843a368eb18336d8dc9" exitCode=0 Jul 07 16:18:39.475634 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:39.475538 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8dn9p" event={"ID":"7b78b3da-77fb-4414-8154-6e6b3d5f06a1","Type":"ContainerDied","Data":"3ab128dac9a860b51a9b07cd1186c7a745609f52cb911843a368eb18336d8dc9"} Jul 07 16:18:39.494396 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:39.494349 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6kr6c" podStartSLOduration=4.060648837 podStartE2EDuration="27.494333656s" totalImagesPullingTime="23.433684819s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 16:18:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:18:39.492924209 +0000 UTC m=+27.654737159" watchObservedRunningTime="2026-07-07 16:18:39.494333656 +0000 UTC m=+27.656146608" Jul 07 16:18:40.343314 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:40.343279 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:40.343487 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:40.343380 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9pff" podUID="83eacc13-cd9b-436a-882b-18c36f62e74c" Jul 07 16:18:40.343487 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:40.343385 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:40.343487 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:40.343408 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:40.343487 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:40.343455 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xsj8b" podUID="97f47a3d-062a-4159-8ebd-96d0afd8372b" Jul 07 16:18:40.343619 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:40.343500 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-868j2" podUID="093da19e-76ae-42f9-b23a-5ac90eb50fb1" Jul 07 16:18:40.480413 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:40.480371 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" event={"ID":"683ef4c1-d767-42cb-bcce-a5219be09147","Type":"ContainerStarted","Data":"ead5b57d42c6eef62b71912a3502e24f01afa882503f08b813db1d61a1af2acf"} Jul 07 16:18:41.485104 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:41.485073 2564 generic.go:356] "Generic (PLEG): container finished" podID="7b78b3da-77fb-4414-8154-6e6b3d5f06a1" containerID="435b3f18950722399a3ba73cd39310f8bedd31e765edd01dc47b3a26afa321ca" exitCode=0 Jul 07 16:18:41.485563 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:41.485120 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8dn9p" event={"ID":"7b78b3da-77fb-4414-8154-6e6b3d5f06a1","Type":"ContainerDied","Data":"435b3f18950722399a3ba73cd39310f8bedd31e765edd01dc47b3a26afa321ca"} Jul 07 16:18:42.345100 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:42.344854 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:42.345320 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:42.344933 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:42.345320 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:42.345194 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xsj8b" podUID="97f47a3d-062a-4159-8ebd-96d0afd8372b" Jul 07 16:18:42.345320 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:42.345245 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9pff" podUID="83eacc13-cd9b-436a-882b-18c36f62e74c" Jul 07 16:18:42.345320 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:42.344986 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:42.345320 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:42.345317 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-868j2" podUID="093da19e-76ae-42f9-b23a-5ac90eb50fb1" Jul 07 16:18:42.490348 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:42.490314 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" event={"ID":"683ef4c1-d767-42cb-bcce-a5219be09147","Type":"ContainerStarted","Data":"c5ef851e754ade61c03ad6801d7c7b37a64bb39469741d7587cc86f619e765f9"} Jul 07 16:18:42.490776 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:42.490602 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:42.490776 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:42.490666 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:42.507314 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:42.507288 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:42.520235 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:42.520191 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" podStartSLOduration=8.293874711 podStartE2EDuration="30.520177792s" totalImagesPullingTime="22.226303081s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 16:18:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:18:42.519774501 +0000 UTC m=+30.681587453" watchObservedRunningTime="2026-07-07 16:18:42.520177792 +0000 UTC m=+30.681990777" Jul 07 16:18:43.494502 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:43.494458 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:43.513154 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:43.513123 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:18:43.718789 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:43.718755 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-868j2"] Jul 07 16:18:43.718954 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:43.718864 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:43.718999 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:43.718963 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-868j2" podUID="093da19e-76ae-42f9-b23a-5ac90eb50fb1" Jul 07 16:18:43.720571 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:43.720544 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xsj8b"] Jul 07 16:18:43.720694 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:43.720658 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:43.720785 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:43.720767 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xsj8b" podUID="97f47a3d-062a-4159-8ebd-96d0afd8372b" Jul 07 16:18:43.722602 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:43.722579 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g9pff"] Jul 07 16:18:43.722703 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:43.722664 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:43.722767 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:43.722748 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9pff" podUID="83eacc13-cd9b-436a-882b-18c36f62e74c" Jul 07 16:18:45.343992 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:45.343957 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:45.343992 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:45.343985 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:45.344444 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:45.343958 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:45.344444 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:45.344089 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-868j2" podUID="093da19e-76ae-42f9-b23a-5ac90eb50fb1" Jul 07 16:18:45.344444 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:45.344136 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xsj8b" podUID="97f47a3d-062a-4159-8ebd-96d0afd8372b" Jul 07 16:18:45.344444 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:45.344221 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9pff" podUID="83eacc13-cd9b-436a-882b-18c36f62e74c" Jul 07 16:18:46.093837 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:46.093746 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret\") pod \"global-pull-secret-syncer-868j2\" (UID: \"093da19e-76ae-42f9-b23a-5ac90eb50fb1\") " pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:46.093837 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:46.093822 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs\") pod \"network-metrics-daemon-xsj8b\" (UID: \"97f47a3d-062a-4159-8ebd-96d0afd8372b\") " pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:46.094093 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:46.093853 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggx76\" (UniqueName: \"kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76\") pod \"network-check-target-g9pff\" (UID: \"83eacc13-cd9b-436a-882b-18c36f62e74c\") " pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:46.094093 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:46.093941 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 16:18:46.094093 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:46.093978 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 16:18:46.094093 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:46.093993 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 16:18:46.094093 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:46.094008 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 16:18:46.094093 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:46.094020 2564 projected.go:196] Error preparing data for projected volume kube-api-access-ggx76 for pod openshift-network-diagnostics/network-check-target-g9pff: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 16:18:46.094093 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:46.094023 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret podName:093da19e-76ae-42f9-b23a-5ac90eb50fb1 nodeName:}" failed. No retries permitted until 2026-07-07 16:19:18.093997909 +0000 UTC m=+66.255810864 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret") pod "global-pull-secret-syncer-868j2" (UID: "093da19e-76ae-42f9-b23a-5ac90eb50fb1") : object "kube-system"/"original-pull-secret" not registered Jul 07 16:18:46.094093 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:46.094045 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs podName:97f47a3d-062a-4159-8ebd-96d0afd8372b nodeName:}" failed. No retries permitted until 2026-07-07 16:19:18.094034862 +0000 UTC m=+66.255847803 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs") pod "network-metrics-daemon-xsj8b" (UID: "97f47a3d-062a-4159-8ebd-96d0afd8372b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 16:18:46.094093 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:46.094060 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76 podName:83eacc13-cd9b-436a-882b-18c36f62e74c nodeName:}" failed. No retries permitted until 2026-07-07 16:19:18.094051991 +0000 UTC m=+66.255864929 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ggx76" (UniqueName: "kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76") pod "network-check-target-g9pff" (UID: "83eacc13-cd9b-436a-882b-18c36f62e74c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 16:18:47.343781 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:47.343603 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:47.344306 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:47.343630 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:47.344306 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:47.343851 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9pff" podUID="83eacc13-cd9b-436a-882b-18c36f62e74c" Jul 07 16:18:47.344306 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:47.343663 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:47.344306 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:47.343975 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-868j2" podUID="093da19e-76ae-42f9-b23a-5ac90eb50fb1" Jul 07 16:18:47.344306 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:47.344049 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xsj8b" podUID="97f47a3d-062a-4159-8ebd-96d0afd8372b" Jul 07 16:18:47.509559 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:47.509520 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8dn9p" event={"ID":"7b78b3da-77fb-4414-8154-6e6b3d5f06a1","Type":"ContainerStarted","Data":"18dd56df53d158a1e71967cba86effe63ae61dac2a0df807d3dade153b5d80d1"} Jul 07 16:18:48.514666 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:48.514632 2564 generic.go:356] "Generic (PLEG): container finished" podID="7b78b3da-77fb-4414-8154-6e6b3d5f06a1" containerID="18dd56df53d158a1e71967cba86effe63ae61dac2a0df807d3dade153b5d80d1" exitCode=0 Jul 07 16:18:48.514666 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:48.514669 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8dn9p" event={"ID":"7b78b3da-77fb-4414-8154-6e6b3d5f06a1","Type":"ContainerDied","Data":"18dd56df53d158a1e71967cba86effe63ae61dac2a0df807d3dade153b5d80d1"} Jul 07 16:18:49.343274 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.343241 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:49.343456 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.343241 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:49.343456 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:49.343354 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xsj8b" podUID="97f47a3d-062a-4159-8ebd-96d0afd8372b" Jul 07 16:18:49.343456 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.343382 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:49.343617 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:49.343474 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-868j2" podUID="093da19e-76ae-42f9-b23a-5ac90eb50fb1" Jul 07 16:18:49.343617 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:49.343510 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-g9pff" podUID="83eacc13-cd9b-436a-882b-18c36f62e74c" Jul 07 16:18:49.520139 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.520108 2564 generic.go:356] "Generic (PLEG): container finished" podID="7b78b3da-77fb-4414-8154-6e6b3d5f06a1" containerID="48376bbf6682d3c04b2ccfb40a7d9f71c12882f3daf46b3a663a8cbbaf80e9a5" exitCode=0 Jul 07 16:18:49.520581 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.520166 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8dn9p" event={"ID":"7b78b3da-77fb-4414-8154-6e6b3d5f06a1","Type":"ContainerDied","Data":"48376bbf6682d3c04b2ccfb40a7d9f71c12882f3daf46b3a663a8cbbaf80e9a5"} Jul 07 16:18:49.652295 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.652270 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-174.ec2.internal" event="NodeReady" Jul 07 16:18:49.652565 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.652395 2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 16:18:49.696985 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.696953 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8c6f746f8-bb4kj"] Jul 07 16:18:49.716334 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.716304 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gpl7j"] Jul 07 16:18:49.716468 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.716449 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:49.719066 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.719039 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 16:18:49.719199 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.719090 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-p9zlp\"" Jul 07 16:18:49.719199 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.719190 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 16:18:49.719506 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.719488 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 16:18:49.725941 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.725925 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 16:18:49.732521 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.732504 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8c6f746f8-bb4kj"] Jul 07 16:18:49.732628 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.732609 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gpl7j" Jul 07 16:18:49.733871 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.733851 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gpl7j"] Jul 07 16:18:49.735478 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.735460 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2sv27\"" Jul 07 16:18:49.735747 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.735730 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 16:18:49.735862 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.735736 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 16:18:49.821170 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.821135 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e68eced5-8d40-432a-95f7-3f5ec13e0b25-tmp-dir\") pod \"dns-default-gpl7j\" (UID: \"e68eced5-8d40-432a-95f7-3f5ec13e0b25\") " pod="openshift-dns/dns-default-gpl7j" Jul 07 16:18:49.821170 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.821178 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e68eced5-8d40-432a-95f7-3f5ec13e0b25-config-volume\") pod \"dns-default-gpl7j\" (UID: \"e68eced5-8d40-432a-95f7-3f5ec13e0b25\") " pod="openshift-dns/dns-default-gpl7j" Jul 07 16:18:49.821401 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.821253 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4955c880-3f01-4f4e-b9e3-6025c974796f-image-registry-private-configuration\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:49.821401 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.821289 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:49.821401 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.821307 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4955c880-3f01-4f4e-b9e3-6025c974796f-ca-trust-extracted\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:49.821401 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.821323 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-bound-sa-token\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:49.821401 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.821347 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4955c880-3f01-4f4e-b9e3-6025c974796f-trusted-ca\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:49.821401 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.821365 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-certificates\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:49.821646 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.821421 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4955c880-3f01-4f4e-b9e3-6025c974796f-installation-pull-secrets\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:49.821646 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.821445 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvnqj\" (UniqueName: \"kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-kube-api-access-dvnqj\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:49.821646 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.821462 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e68eced5-8d40-432a-95f7-3f5ec13e0b25-metrics-tls\") pod \"dns-default-gpl7j\" (UID: \"e68eced5-8d40-432a-95f7-3f5ec13e0b25\") " pod="openshift-dns/dns-default-gpl7j" Jul 07 16:18:49.821646 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.821494 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qb9t\" (UniqueName: \"kubernetes.io/projected/e68eced5-8d40-432a-95f7-3f5ec13e0b25-kube-api-access-6qb9t\") pod \"dns-default-gpl7j\" (UID: \"e68eced5-8d40-432a-95f7-3f5ec13e0b25\") " pod="openshift-dns/dns-default-gpl7j" Jul 07 16:18:49.825036 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.825012 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-8hr9f"] Jul 07 16:18:49.839559 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.839541 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8hr9f" Jul 07 16:18:49.843200 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.843176 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8hr9f"] Jul 07 16:18:49.843643 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.843627 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5hz4f\"" Jul 07 16:18:49.843745 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.843645 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 16:18:49.844265 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.844201 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 16:18:49.844265 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.844219 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 16:18:49.922533 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.922498 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4955c880-3f01-4f4e-b9e3-6025c974796f-installation-pull-secrets\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:49.922533 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.922535 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvnqj\" (UniqueName: \"kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-kube-api-access-dvnqj\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:49.922809 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.922554 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e68eced5-8d40-432a-95f7-3f5ec13e0b25-metrics-tls\") pod \"dns-default-gpl7j\" (UID: \"e68eced5-8d40-432a-95f7-3f5ec13e0b25\") " pod="openshift-dns/dns-default-gpl7j" Jul 07 16:18:49.922809 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.922571 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qb9t\" (UniqueName: \"kubernetes.io/projected/e68eced5-8d40-432a-95f7-3f5ec13e0b25-kube-api-access-6qb9t\") pod \"dns-default-gpl7j\" (UID: \"e68eced5-8d40-432a-95f7-3f5ec13e0b25\") " pod="openshift-dns/dns-default-gpl7j" Jul 07 16:18:49.922809 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.922601 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e68eced5-8d40-432a-95f7-3f5ec13e0b25-tmp-dir\") pod \"dns-default-gpl7j\" (UID: \"e68eced5-8d40-432a-95f7-3f5ec13e0b25\") " pod="openshift-dns/dns-default-gpl7j" Jul 07 16:18:49.922809 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:49.922672 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 16:18:49.922809 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:49.922733 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e68eced5-8d40-432a-95f7-3f5ec13e0b25-metrics-tls podName:e68eced5-8d40-432a-95f7-3f5ec13e0b25 nodeName:}" failed. No retries permitted until 2026-07-07 16:18:50.422715316 +0000 UTC m=+38.584528249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e68eced5-8d40-432a-95f7-3f5ec13e0b25-metrics-tls") pod "dns-default-gpl7j" (UID: "e68eced5-8d40-432a-95f7-3f5ec13e0b25") : secret "dns-default-metrics-tls" not found Jul 07 16:18:49.922809 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.922781 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e68eced5-8d40-432a-95f7-3f5ec13e0b25-config-volume\") pod \"dns-default-gpl7j\" (UID: \"e68eced5-8d40-432a-95f7-3f5ec13e0b25\") " pod="openshift-dns/dns-default-gpl7j" Jul 07 16:18:49.923122 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.922828 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ebc29602-88aa-4ce8-aa11-341d20706083-cert\") pod \"ingress-canary-8hr9f\" (UID: \"ebc29602-88aa-4ce8-aa11-341d20706083\") " pod="openshift-ingress-canary/ingress-canary-8hr9f" Jul 07 16:18:49.923122 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.922873 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4955c880-3f01-4f4e-b9e3-6025c974796f-image-registry-private-configuration\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:49.923122 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.922916 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:49.923122 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.922967 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e68eced5-8d40-432a-95f7-3f5ec13e0b25-tmp-dir\") pod \"dns-default-gpl7j\" (UID: \"e68eced5-8d40-432a-95f7-3f5ec13e0b25\") " pod="openshift-dns/dns-default-gpl7j" Jul 07 16:18:49.923122 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.922994 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4955c880-3f01-4f4e-b9e3-6025c974796f-ca-trust-extracted\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:49.923122 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.923020 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-bound-sa-token\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:49.923122 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.923046 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4955c880-3f01-4f4e-b9e3-6025c974796f-trusted-ca\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:49.923122 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.923067 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-certificates\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:49.923122 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:49.923090 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 16:18:49.923122 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:49.923108 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8c6f746f8-bb4kj: secret "image-registry-tls" not found Jul 07 16:18:49.923492 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.923125 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dpwm\" (UniqueName: \"kubernetes.io/projected/ebc29602-88aa-4ce8-aa11-341d20706083-kube-api-access-9dpwm\") pod \"ingress-canary-8hr9f\" (UID: \"ebc29602-88aa-4ce8-aa11-341d20706083\") " pod="openshift-ingress-canary/ingress-canary-8hr9f" Jul 07 16:18:49.923492 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:49.923177 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls podName:4955c880-3f01-4f4e-b9e3-6025c974796f nodeName:}" failed. No retries permitted until 2026-07-07 16:18:50.423153794 +0000 UTC m=+38.584966745 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls") pod "image-registry-8c6f746f8-bb4kj" (UID: "4955c880-3f01-4f4e-b9e3-6025c974796f") : secret "image-registry-tls" not found Jul 07 16:18:49.923685 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.923663 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4955c880-3f01-4f4e-b9e3-6025c974796f-ca-trust-extracted\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:49.923771 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.923689 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e68eced5-8d40-432a-95f7-3f5ec13e0b25-config-volume\") pod \"dns-default-gpl7j\" (UID: \"e68eced5-8d40-432a-95f7-3f5ec13e0b25\") " pod="openshift-dns/dns-default-gpl7j" Jul 07 16:18:49.924655 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.924139 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-certificates\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:49.926797 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.925296 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4955c880-3f01-4f4e-b9e3-6025c974796f-trusted-ca\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:49.928176 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.928150 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4955c880-3f01-4f4e-b9e3-6025c974796f-installation-pull-secrets\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:49.928936 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.928891 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4955c880-3f01-4f4e-b9e3-6025c974796f-image-registry-private-configuration\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:49.932498 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.932474 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvnqj\" (UniqueName: \"kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-kube-api-access-dvnqj\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:49.932693 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.932671 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qb9t\" (UniqueName: \"kubernetes.io/projected/e68eced5-8d40-432a-95f7-3f5ec13e0b25-kube-api-access-6qb9t\") pod \"dns-default-gpl7j\" (UID: \"e68eced5-8d40-432a-95f7-3f5ec13e0b25\") " pod="openshift-dns/dns-default-gpl7j" Jul 07 16:18:49.933366 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:49.933337 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-bound-sa-token\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:50.024282 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:50.024250 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dpwm\" (UniqueName: \"kubernetes.io/projected/ebc29602-88aa-4ce8-aa11-341d20706083-kube-api-access-9dpwm\") pod \"ingress-canary-8hr9f\" (UID: \"ebc29602-88aa-4ce8-aa11-341d20706083\") " pod="openshift-ingress-canary/ingress-canary-8hr9f" Jul 07 16:18:50.024446 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:50.024318 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ebc29602-88aa-4ce8-aa11-341d20706083-cert\") pod \"ingress-canary-8hr9f\" (UID: \"ebc29602-88aa-4ce8-aa11-341d20706083\") " pod="openshift-ingress-canary/ingress-canary-8hr9f" Jul 07 16:18:50.026871 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:50.026849 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ebc29602-88aa-4ce8-aa11-341d20706083-cert\") pod \"ingress-canary-8hr9f\" (UID: \"ebc29602-88aa-4ce8-aa11-341d20706083\") " pod="openshift-ingress-canary/ingress-canary-8hr9f" Jul 07 16:18:50.033707 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:50.033679 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dpwm\" (UniqueName: \"kubernetes.io/projected/ebc29602-88aa-4ce8-aa11-341d20706083-kube-api-access-9dpwm\") pod \"ingress-canary-8hr9f\" (UID: \"ebc29602-88aa-4ce8-aa11-341d20706083\") " pod="openshift-ingress-canary/ingress-canary-8hr9f" Jul 07 16:18:50.148744 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:50.148559 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-8hr9f" Jul 07 16:18:50.299450 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:50.299418 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-8hr9f"] Jul 07 16:18:50.302974 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:18:50.302950 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebc29602_88aa_4ce8_aa11_341d20706083.slice/crio-70957674001a1c3cd7003a1ed2969464a05f192c513e35b8be2eed90886cdef0 WatchSource:0}: Error finding container 70957674001a1c3cd7003a1ed2969464a05f192c513e35b8be2eed90886cdef0: Status 404 returned error can't find the container with id 70957674001a1c3cd7003a1ed2969464a05f192c513e35b8be2eed90886cdef0 Jul 07 16:18:50.427848 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:50.427757 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:50.427848 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:50.427834 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e68eced5-8d40-432a-95f7-3f5ec13e0b25-metrics-tls\") pod \"dns-default-gpl7j\" (UID: \"e68eced5-8d40-432a-95f7-3f5ec13e0b25\") " pod="openshift-dns/dns-default-gpl7j" Jul 07 16:18:50.428058 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:50.427945 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 16:18:50.428058 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:50.427958 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 16:18:50.428058 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:50.427965 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8c6f746f8-bb4kj: secret "image-registry-tls" not found Jul 07 16:18:50.428058 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:50.428014 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e68eced5-8d40-432a-95f7-3f5ec13e0b25-metrics-tls podName:e68eced5-8d40-432a-95f7-3f5ec13e0b25 nodeName:}" failed. No retries permitted until 2026-07-07 16:18:51.428001368 +0000 UTC m=+39.589814300 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e68eced5-8d40-432a-95f7-3f5ec13e0b25-metrics-tls") pod "dns-default-gpl7j" (UID: "e68eced5-8d40-432a-95f7-3f5ec13e0b25") : secret "dns-default-metrics-tls" not found Jul 07 16:18:50.428058 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:50.428030 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls podName:4955c880-3f01-4f4e-b9e3-6025c974796f nodeName:}" failed. No retries permitted until 2026-07-07 16:18:51.428022659 +0000 UTC m=+39.589835592 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls") pod "image-registry-8c6f746f8-bb4kj" (UID: "4955c880-3f01-4f4e-b9e3-6025c974796f") : secret "image-registry-tls" not found Jul 07 16:18:50.525845 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:50.525807 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8dn9p" event={"ID":"7b78b3da-77fb-4414-8154-6e6b3d5f06a1","Type":"ContainerStarted","Data":"38d080a1117786a7247696f834b7dc81cdee4d2c89b5be59c7f551dfc26961ac"} Jul 07 16:18:50.527110 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:50.527076 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8hr9f" event={"ID":"ebc29602-88aa-4ce8-aa11-341d20706083","Type":"ContainerStarted","Data":"70957674001a1c3cd7003a1ed2969464a05f192c513e35b8be2eed90886cdef0"} Jul 07 16:18:50.552571 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:50.552516 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8dn9p" podStartSLOduration=8.219006945 podStartE2EDuration="38.552499426s" totalImagesPullingTime="30.333492481s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 16:18:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:18:50.55130472 +0000 UTC m=+38.713117678" watchObservedRunningTime="2026-07-07 16:18:50.552499426 +0000 UTC m=+38.714312381" Jul 07 16:18:51.344100 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:51.344065 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:18:51.344270 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:51.344163 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:18:51.344270 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:51.344192 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:18:51.347974 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:51.347950 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 16:18:51.348127 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:51.347951 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2nbnd\"" Jul 07 16:18:51.348127 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:51.347986 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 16:18:51.348127 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:51.348093 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-68hwt\"" Jul 07 16:18:51.348127 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:51.348112 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 16:18:51.348319 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:51.348224 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 16:18:51.434616 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:51.434572 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e68eced5-8d40-432a-95f7-3f5ec13e0b25-metrics-tls\") pod \"dns-default-gpl7j\" (UID: \"e68eced5-8d40-432a-95f7-3f5ec13e0b25\") " pod="openshift-dns/dns-default-gpl7j" Jul 07 16:18:51.434772 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:51.434653 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:51.434772 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:51.434725 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 16:18:51.434872 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:51.434786 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e68eced5-8d40-432a-95f7-3f5ec13e0b25-metrics-tls podName:e68eced5-8d40-432a-95f7-3f5ec13e0b25 nodeName:}" failed. No retries permitted until 2026-07-07 16:18:53.434771015 +0000 UTC m=+41.596583947 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e68eced5-8d40-432a-95f7-3f5ec13e0b25-metrics-tls") pod "dns-default-gpl7j" (UID: "e68eced5-8d40-432a-95f7-3f5ec13e0b25") : secret "dns-default-metrics-tls" not found Jul 07 16:18:51.434872 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:51.434790 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 16:18:51.434872 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:51.434803 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8c6f746f8-bb4kj: secret "image-registry-tls" not found Jul 07 16:18:51.435062 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:51.434887 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls podName:4955c880-3f01-4f4e-b9e3-6025c974796f nodeName:}" failed. No retries permitted until 2026-07-07 16:18:53.434874203 +0000 UTC m=+41.596687137 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls") pod "image-registry-8c6f746f8-bb4kj" (UID: "4955c880-3f01-4f4e-b9e3-6025c974796f") : secret "image-registry-tls" not found Jul 07 16:18:53.451891 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:53.451849 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e68eced5-8d40-432a-95f7-3f5ec13e0b25-metrics-tls\") pod \"dns-default-gpl7j\" (UID: \"e68eced5-8d40-432a-95f7-3f5ec13e0b25\") " pod="openshift-dns/dns-default-gpl7j" Jul 07 16:18:53.452261 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:53.451951 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:53.452261 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:53.452021 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 16:18:53.452261 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:53.452045 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 16:18:53.452261 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:53.452059 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8c6f746f8-bb4kj: secret "image-registry-tls" not found Jul 07 16:18:53.452261 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:53.452091 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e68eced5-8d40-432a-95f7-3f5ec13e0b25-metrics-tls podName:e68eced5-8d40-432a-95f7-3f5ec13e0b25 nodeName:}" failed. No retries permitted until 2026-07-07 16:18:57.452073016 +0000 UTC m=+45.613885968 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e68eced5-8d40-432a-95f7-3f5ec13e0b25-metrics-tls") pod "dns-default-gpl7j" (UID: "e68eced5-8d40-432a-95f7-3f5ec13e0b25") : secret "dns-default-metrics-tls" not found Jul 07 16:18:53.452261 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:53.452107 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls podName:4955c880-3f01-4f4e-b9e3-6025c974796f nodeName:}" failed. No retries permitted until 2026-07-07 16:18:57.452100536 +0000 UTC m=+45.613913473 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls") pod "image-registry-8c6f746f8-bb4kj" (UID: "4955c880-3f01-4f4e-b9e3-6025c974796f") : secret "image-registry-tls" not found Jul 07 16:18:55.541750 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:55.541475 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-8hr9f" event={"ID":"ebc29602-88aa-4ce8-aa11-341d20706083","Type":"ContainerStarted","Data":"1a204d9cf6a46e46137d695a3aee52d18c13b001330678331c1f924f1676a565"} Jul 07 16:18:55.559836 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:55.559787 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-8hr9f" podStartSLOduration=2.2833746440000002 podStartE2EDuration="6.559772532s" totalImagesPullingTime="4.276397888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:18:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:18:55.559452167 +0000 UTC m=+43.721265122" watchObservedRunningTime="2026-07-07 16:18:55.559772532 +0000 UTC m=+43.721585486" Jul 07 16:18:57.480948 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:57.480891 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e68eced5-8d40-432a-95f7-3f5ec13e0b25-metrics-tls\") pod \"dns-default-gpl7j\" (UID: \"e68eced5-8d40-432a-95f7-3f5ec13e0b25\") " pod="openshift-dns/dns-default-gpl7j" Jul 07 16:18:57.481586 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:18:57.480992 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:18:57.481586 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:57.481048 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 16:18:57.481586 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:57.481102 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 16:18:57.481586 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:57.481114 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8c6f746f8-bb4kj: secret "image-registry-tls" not found Jul 07 16:18:57.481586 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:57.481113 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e68eced5-8d40-432a-95f7-3f5ec13e0b25-metrics-tls podName:e68eced5-8d40-432a-95f7-3f5ec13e0b25 nodeName:}" failed. No retries permitted until 2026-07-07 16:19:05.48109795 +0000 UTC m=+53.642910887 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e68eced5-8d40-432a-95f7-3f5ec13e0b25-metrics-tls") pod "dns-default-gpl7j" (UID: "e68eced5-8d40-432a-95f7-3f5ec13e0b25") : secret "dns-default-metrics-tls" not found Jul 07 16:18:57.481586 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:18:57.481169 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls podName:4955c880-3f01-4f4e-b9e3-6025c974796f nodeName:}" failed. No retries permitted until 2026-07-07 16:19:05.481154924 +0000 UTC m=+53.642967857 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls") pod "image-registry-8c6f746f8-bb4kj" (UID: "4955c880-3f01-4f4e-b9e3-6025c974796f") : secret "image-registry-tls" not found Jul 07 16:19:01.042580 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.042549 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-7r79m"] Jul 07 16:19:01.079428 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.079399 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-mmrgp"] Jul 07 16:19:01.079611 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.079530 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-7r79m" Jul 07 16:19:01.082678 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.082654 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 07 16:19:01.083983 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.083963 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xl7xk\"" Jul 07 16:19:01.084087 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.083966 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 16:19:01.084087 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.084011 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 07 16:19:01.085146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.085128 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 16:19:01.094558 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.094536 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-7r79m"] Jul 07 16:19:01.094686 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.094567 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-mmrgp"] Jul 07 16:19:01.094686 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.094582 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-rhsmp"] Jul 07 16:19:01.094686 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.094670 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" Jul 07 16:19:01.097194 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.097174 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 07 16:19:01.097487 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.097468 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 16:19:01.097487 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.097493 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 07 16:19:01.097683 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.097502 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 16:19:01.097683 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.097576 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-hgtbz\"" Jul 07 16:19:01.102896 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.102876 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 07 16:19:01.112518 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.112491 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-rhsmp"] Jul 07 16:19:01.112626 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.112583 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-rhsmp" Jul 07 16:19:01.115569 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.115547 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 07 16:19:01.115702 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.115591 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-zh8ts\"" Jul 07 16:19:01.115767 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.115745 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 07 16:19:01.149179 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.149147 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xct9g"] Jul 07 16:19:01.164415 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.164373 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bsqxn"] Jul 07 16:19:01.164593 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.164524 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xct9g" Jul 07 16:19:01.167305 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.167283 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 07 16:19:01.167464 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.167306 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-vd6w5\"" Jul 07 16:19:01.167464 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.167323 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 07 16:19:01.167464 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.167414 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 07 16:19:01.167464 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.167439 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 07 16:19:01.188122 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.188096 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xct9g"] Jul 07 16:19:01.188122 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.188128 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bsqxn"] Jul 07 16:19:01.188334 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.188229 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bsqxn" Jul 07 16:19:01.190716 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.190683 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 07 16:19:01.190833 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.190767 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 07 16:19:01.190833 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.190770 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 07 16:19:01.190833 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.190814 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-2k2z6\"" Jul 07 16:19:01.209248 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.209215 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/698c0fe4-2b45-4ffd-b498-54dc14b4df02-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-7r79m\" (UID: \"698c0fe4-2b45-4ffd-b498-54dc14b4df02\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-7r79m" Jul 07 16:19:01.209412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.209287 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trbc5\" (UniqueName: \"kubernetes.io/projected/40c9cc16-aae3-4bb5-b11a-7723051bb9c0-kube-api-access-trbc5\") pod \"insights-operator-5c847dcfc4-mmrgp\" (UID: \"40c9cc16-aae3-4bb5-b11a-7723051bb9c0\") " pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" Jul 07 16:19:01.209412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.209307 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40c9cc16-aae3-4bb5-b11a-7723051bb9c0-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-mmrgp\" (UID: \"40c9cc16-aae3-4bb5-b11a-7723051bb9c0\") " pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" Jul 07 16:19:01.209412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.209328 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzdw8\" (UniqueName: \"kubernetes.io/projected/6284964a-74fe-4d50-9a06-b13ba29d2d98-kube-api-access-bzdw8\") pod \"volume-data-source-validator-5d456fdbcd-rhsmp\" (UID: \"6284964a-74fe-4d50-9a06-b13ba29d2d98\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-rhsmp" Jul 07 16:19:01.209412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.209346 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/40c9cc16-aae3-4bb5-b11a-7723051bb9c0-snapshots\") pod \"insights-operator-5c847dcfc4-mmrgp\" (UID: \"40c9cc16-aae3-4bb5-b11a-7723051bb9c0\") " pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" Jul 07 16:19:01.209412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.209388 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkhpk\" (UniqueName: \"kubernetes.io/projected/698c0fe4-2b45-4ffd-b498-54dc14b4df02-kube-api-access-fkhpk\") pod \"cluster-monitoring-operator-64786d9cbb-7r79m\" (UID: \"698c0fe4-2b45-4ffd-b498-54dc14b4df02\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-7r79m" Jul 07 16:19:01.209643 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.209410 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/698c0fe4-2b45-4ffd-b498-54dc14b4df02-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-7r79m\" (UID: \"698c0fe4-2b45-4ffd-b498-54dc14b4df02\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-7r79m" Jul 07 16:19:01.209643 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.209442 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40c9cc16-aae3-4bb5-b11a-7723051bb9c0-tmp\") pod \"insights-operator-5c847dcfc4-mmrgp\" (UID: \"40c9cc16-aae3-4bb5-b11a-7723051bb9c0\") " pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" Jul 07 16:19:01.209643 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.209457 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40c9cc16-aae3-4bb5-b11a-7723051bb9c0-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-mmrgp\" (UID: \"40c9cc16-aae3-4bb5-b11a-7723051bb9c0\") " pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" Jul 07 16:19:01.209643 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.209528 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/40c9cc16-aae3-4bb5-b11a-7723051bb9c0-serving-cert\") pod \"insights-operator-5c847dcfc4-mmrgp\" (UID: \"40c9cc16-aae3-4bb5-b11a-7723051bb9c0\") " pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" Jul 07 16:19:01.310700 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.310599 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt57q\" (UniqueName: \"kubernetes.io/projected/ad867472-6624-4250-af20-f3402dca4af1-kube-api-access-pt57q\") pod \"cluster-samples-operator-56fcf86bf-bsqxn\" (UID: \"ad867472-6624-4250-af20-f3402dca4af1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bsqxn" Jul 07 16:19:01.310700 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.310645 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40c9cc16-aae3-4bb5-b11a-7723051bb9c0-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-mmrgp\" (UID: \"40c9cc16-aae3-4bb5-b11a-7723051bb9c0\") " pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" Jul 07 16:19:01.310700 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.310666 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ad867472-6624-4250-af20-f3402dca4af1-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bsqxn\" (UID: \"ad867472-6624-4250-af20-f3402dca4af1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bsqxn" Jul 07 16:19:01.311014 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.310737 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40c9cc16-aae3-4bb5-b11a-7723051bb9c0-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-mmrgp\" (UID: \"40c9cc16-aae3-4bb5-b11a-7723051bb9c0\") " pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" Jul 07 16:19:01.311014 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.310772 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzdw8\" (UniqueName: \"kubernetes.io/projected/6284964a-74fe-4d50-9a06-b13ba29d2d98-kube-api-access-bzdw8\") pod \"volume-data-source-validator-5d456fdbcd-rhsmp\" (UID: \"6284964a-74fe-4d50-9a06-b13ba29d2d98\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-rhsmp" Jul 07 16:19:01.311014 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.310803 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/40c9cc16-aae3-4bb5-b11a-7723051bb9c0-snapshots\") pod \"insights-operator-5c847dcfc4-mmrgp\" (UID: \"40c9cc16-aae3-4bb5-b11a-7723051bb9c0\") " pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" Jul 07 16:19:01.311014 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.310830 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trbc5\" (UniqueName: \"kubernetes.io/projected/40c9cc16-aae3-4bb5-b11a-7723051bb9c0-kube-api-access-trbc5\") pod \"insights-operator-5c847dcfc4-mmrgp\" (UID: \"40c9cc16-aae3-4bb5-b11a-7723051bb9c0\") " pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" Jul 07 16:19:01.311014 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.310879 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c13e43af-02c1-486e-8ffd-e334bedac760-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-xct9g\" (UID: \"c13e43af-02c1-486e-8ffd-e334bedac760\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xct9g" Jul 07 16:19:01.311014 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.310961 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkhpk\" (UniqueName: \"kubernetes.io/projected/698c0fe4-2b45-4ffd-b498-54dc14b4df02-kube-api-access-fkhpk\") pod \"cluster-monitoring-operator-64786d9cbb-7r79m\" (UID: \"698c0fe4-2b45-4ffd-b498-54dc14b4df02\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-7r79m" Jul 07 16:19:01.311014 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.310985 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/40c9cc16-aae3-4bb5-b11a-7723051bb9c0-serving-cert\") pod \"insights-operator-5c847dcfc4-mmrgp\" (UID: \"40c9cc16-aae3-4bb5-b11a-7723051bb9c0\") " pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" Jul 07 16:19:01.311326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.311044 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c13e43af-02c1-486e-8ffd-e334bedac760-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-xct9g\" (UID: \"c13e43af-02c1-486e-8ffd-e334bedac760\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xct9g" Jul 07 16:19:01.311326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.311183 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwp4p\" (UniqueName: \"kubernetes.io/projected/c13e43af-02c1-486e-8ffd-e334bedac760-kube-api-access-gwp4p\") pod \"kube-storage-version-migrator-operator-846bf668f7-xct9g\" (UID: \"c13e43af-02c1-486e-8ffd-e334bedac760\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xct9g" Jul 07 16:19:01.311326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.311299 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40c9cc16-aae3-4bb5-b11a-7723051bb9c0-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-mmrgp\" (UID: \"40c9cc16-aae3-4bb5-b11a-7723051bb9c0\") " pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" Jul 07 16:19:01.311326 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.311319 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/698c0fe4-2b45-4ffd-b498-54dc14b4df02-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-7r79m\" (UID: \"698c0fe4-2b45-4ffd-b498-54dc14b4df02\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-7r79m" Jul 07 16:19:01.311509 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.311346 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40c9cc16-aae3-4bb5-b11a-7723051bb9c0-tmp\") pod \"insights-operator-5c847dcfc4-mmrgp\" (UID: \"40c9cc16-aae3-4bb5-b11a-7723051bb9c0\") " pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" Jul 07 16:19:01.311509 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.311382 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/698c0fe4-2b45-4ffd-b498-54dc14b4df02-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-7r79m\" (UID: \"698c0fe4-2b45-4ffd-b498-54dc14b4df02\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-7r79m" Jul 07 16:19:01.311509 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:01.311452 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 16:19:01.311509 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.311472 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/40c9cc16-aae3-4bb5-b11a-7723051bb9c0-snapshots\") pod \"insights-operator-5c847dcfc4-mmrgp\" (UID: \"40c9cc16-aae3-4bb5-b11a-7723051bb9c0\") " pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" Jul 07 16:19:01.311695 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:01.311516 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/698c0fe4-2b45-4ffd-b498-54dc14b4df02-cluster-monitoring-operator-tls podName:698c0fe4-2b45-4ffd-b498-54dc14b4df02 nodeName:}" failed. No retries permitted until 2026-07-07 16:19:01.811495983 +0000 UTC m=+49.973308920 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/698c0fe4-2b45-4ffd-b498-54dc14b4df02-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-7r79m" (UID: "698c0fe4-2b45-4ffd-b498-54dc14b4df02") : secret "cluster-monitoring-operator-tls" not found Jul 07 16:19:01.311695 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.311575 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40c9cc16-aae3-4bb5-b11a-7723051bb9c0-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-mmrgp\" (UID: \"40c9cc16-aae3-4bb5-b11a-7723051bb9c0\") " pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" Jul 07 16:19:01.311695 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.311648 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/40c9cc16-aae3-4bb5-b11a-7723051bb9c0-tmp\") pod \"insights-operator-5c847dcfc4-mmrgp\" (UID: \"40c9cc16-aae3-4bb5-b11a-7723051bb9c0\") " pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" Jul 07 16:19:01.312032 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.312013 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/698c0fe4-2b45-4ffd-b498-54dc14b4df02-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-7r79m\" (UID: \"698c0fe4-2b45-4ffd-b498-54dc14b4df02\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-7r79m" Jul 07 16:19:01.314939 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.314920 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/40c9cc16-aae3-4bb5-b11a-7723051bb9c0-serving-cert\") pod \"insights-operator-5c847dcfc4-mmrgp\" (UID: \"40c9cc16-aae3-4bb5-b11a-7723051bb9c0\") " pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" Jul 07 16:19:01.322134 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.322111 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzdw8\" (UniqueName: \"kubernetes.io/projected/6284964a-74fe-4d50-9a06-b13ba29d2d98-kube-api-access-bzdw8\") pod \"volume-data-source-validator-5d456fdbcd-rhsmp\" (UID: \"6284964a-74fe-4d50-9a06-b13ba29d2d98\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-rhsmp" Jul 07 16:19:01.322251 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.322156 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkhpk\" (UniqueName: \"kubernetes.io/projected/698c0fe4-2b45-4ffd-b498-54dc14b4df02-kube-api-access-fkhpk\") pod \"cluster-monitoring-operator-64786d9cbb-7r79m\" (UID: \"698c0fe4-2b45-4ffd-b498-54dc14b4df02\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-7r79m" Jul 07 16:19:01.322251 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.322229 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trbc5\" (UniqueName: \"kubernetes.io/projected/40c9cc16-aae3-4bb5-b11a-7723051bb9c0-kube-api-access-trbc5\") pod \"insights-operator-5c847dcfc4-mmrgp\" (UID: \"40c9cc16-aae3-4bb5-b11a-7723051bb9c0\") " pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" Jul 07 16:19:01.403923 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.403875 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" Jul 07 16:19:01.412237 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.412207 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c13e43af-02c1-486e-8ffd-e334bedac760-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-xct9g\" (UID: \"c13e43af-02c1-486e-8ffd-e334bedac760\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xct9g" Jul 07 16:19:01.412342 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.412242 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwp4p\" (UniqueName: \"kubernetes.io/projected/c13e43af-02c1-486e-8ffd-e334bedac760-kube-api-access-gwp4p\") pod \"kube-storage-version-migrator-operator-846bf668f7-xct9g\" (UID: \"c13e43af-02c1-486e-8ffd-e334bedac760\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xct9g" Jul 07 16:19:01.412446 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.412421 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt57q\" (UniqueName: \"kubernetes.io/projected/ad867472-6624-4250-af20-f3402dca4af1-kube-api-access-pt57q\") pod \"cluster-samples-operator-56fcf86bf-bsqxn\" (UID: \"ad867472-6624-4250-af20-f3402dca4af1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bsqxn" Jul 07 16:19:01.412493 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.412466 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ad867472-6624-4250-af20-f3402dca4af1-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bsqxn\" (UID: \"ad867472-6624-4250-af20-f3402dca4af1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bsqxn" Jul 07 16:19:01.412551 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.412524 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c13e43af-02c1-486e-8ffd-e334bedac760-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-xct9g\" (UID: \"c13e43af-02c1-486e-8ffd-e334bedac760\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xct9g" Jul 07 16:19:01.412611 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:01.412592 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 16:19:01.412662 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:01.412649 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad867472-6624-4250-af20-f3402dca4af1-samples-operator-tls podName:ad867472-6624-4250-af20-f3402dca4af1 nodeName:}" failed. No retries permitted until 2026-07-07 16:19:01.912634465 +0000 UTC m=+50.074447398 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ad867472-6624-4250-af20-f3402dca4af1-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-bsqxn" (UID: "ad867472-6624-4250-af20-f3402dca4af1") : secret "samples-operator-tls" not found Jul 07 16:19:01.412782 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.412762 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c13e43af-02c1-486e-8ffd-e334bedac760-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-xct9g\" (UID: \"c13e43af-02c1-486e-8ffd-e334bedac760\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xct9g" Jul 07 16:19:01.414666 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.414646 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c13e43af-02c1-486e-8ffd-e334bedac760-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-xct9g\" (UID: \"c13e43af-02c1-486e-8ffd-e334bedac760\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xct9g" Jul 07 16:19:01.421273 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.421254 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-rhsmp" Jul 07 16:19:01.424194 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.424167 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwp4p\" (UniqueName: \"kubernetes.io/projected/c13e43af-02c1-486e-8ffd-e334bedac760-kube-api-access-gwp4p\") pod \"kube-storage-version-migrator-operator-846bf668f7-xct9g\" (UID: \"c13e43af-02c1-486e-8ffd-e334bedac760\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xct9g" Jul 07 16:19:01.424323 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.424309 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt57q\" (UniqueName: \"kubernetes.io/projected/ad867472-6624-4250-af20-f3402dca4af1-kube-api-access-pt57q\") pod \"cluster-samples-operator-56fcf86bf-bsqxn\" (UID: \"ad867472-6624-4250-af20-f3402dca4af1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bsqxn" Jul 07 16:19:01.474067 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.474030 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xct9g" Jul 07 16:19:01.586776 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.586698 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-mmrgp"] Jul 07 16:19:01.591716 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:19:01.591681 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c9cc16_aae3_4bb5_b11a_7723051bb9c0.slice/crio-b57064aa751caa83aa72831b83c844fa5f80ed36e216f2952bfc4b2cc08845d0 WatchSource:0}: Error finding container b57064aa751caa83aa72831b83c844fa5f80ed36e216f2952bfc4b2cc08845d0: Status 404 returned error can't find the container with id b57064aa751caa83aa72831b83c844fa5f80ed36e216f2952bfc4b2cc08845d0 Jul 07 16:19:01.636615 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.636556 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-rhsmp"] Jul 07 16:19:01.639119 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:19:01.639095 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6284964a_74fe_4d50_9a06_b13ba29d2d98.slice/crio-f5ebeab067c73520894ccd43dc6ce357c6de06fcfd9bc28236c92d3da0cf55e8 WatchSource:0}: Error finding container f5ebeab067c73520894ccd43dc6ce357c6de06fcfd9bc28236c92d3da0cf55e8: Status 404 returned error can't find the container with id f5ebeab067c73520894ccd43dc6ce357c6de06fcfd9bc28236c92d3da0cf55e8 Jul 07 16:19:01.653758 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.653730 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xct9g"] Jul 07 16:19:01.657095 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:19:01.657068 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc13e43af_02c1_486e_8ffd_e334bedac760.slice/crio-82340c63e088fc4d521a5d0696671c98f0ad54feb505f11f7e690d4170ef3562 WatchSource:0}: Error finding container 82340c63e088fc4d521a5d0696671c98f0ad54feb505f11f7e690d4170ef3562: Status 404 returned error can't find the container with id 82340c63e088fc4d521a5d0696671c98f0ad54feb505f11f7e690d4170ef3562 Jul 07 16:19:01.816962 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.816900 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/698c0fe4-2b45-4ffd-b498-54dc14b4df02-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-7r79m\" (UID: \"698c0fe4-2b45-4ffd-b498-54dc14b4df02\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-7r79m" Jul 07 16:19:01.817161 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:01.817020 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 16:19:01.817161 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:01.817091 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/698c0fe4-2b45-4ffd-b498-54dc14b4df02-cluster-monitoring-operator-tls podName:698c0fe4-2b45-4ffd-b498-54dc14b4df02 nodeName:}" failed. No retries permitted until 2026-07-07 16:19:02.817074006 +0000 UTC m=+50.978886940 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/698c0fe4-2b45-4ffd-b498-54dc14b4df02-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-7r79m" (UID: "698c0fe4-2b45-4ffd-b498-54dc14b4df02") : secret "cluster-monitoring-operator-tls" not found Jul 07 16:19:01.917884 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:01.917841 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ad867472-6624-4250-af20-f3402dca4af1-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bsqxn\" (UID: \"ad867472-6624-4250-af20-f3402dca4af1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bsqxn" Jul 07 16:19:01.918083 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:01.917994 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 16:19:01.918083 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:01.918048 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad867472-6624-4250-af20-f3402dca4af1-samples-operator-tls podName:ad867472-6624-4250-af20-f3402dca4af1 nodeName:}" failed. No retries permitted until 2026-07-07 16:19:02.918033926 +0000 UTC m=+51.079846858 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ad867472-6624-4250-af20-f3402dca4af1-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-bsqxn" (UID: "ad867472-6624-4250-af20-f3402dca4af1") : secret "samples-operator-tls" not found Jul 07 16:19:02.566682 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:02.566573 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-rhsmp" event={"ID":"6284964a-74fe-4d50-9a06-b13ba29d2d98","Type":"ContainerStarted","Data":"f5ebeab067c73520894ccd43dc6ce357c6de06fcfd9bc28236c92d3da0cf55e8"} Jul 07 16:19:02.568693 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:02.568584 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" event={"ID":"40c9cc16-aae3-4bb5-b11a-7723051bb9c0","Type":"ContainerStarted","Data":"b57064aa751caa83aa72831b83c844fa5f80ed36e216f2952bfc4b2cc08845d0"} Jul 07 16:19:02.569924 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:02.569864 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xct9g" event={"ID":"c13e43af-02c1-486e-8ffd-e334bedac760","Type":"ContainerStarted","Data":"82340c63e088fc4d521a5d0696671c98f0ad54feb505f11f7e690d4170ef3562"} Jul 07 16:19:02.826830 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:02.826736 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/698c0fe4-2b45-4ffd-b498-54dc14b4df02-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-7r79m\" (UID: \"698c0fe4-2b45-4ffd-b498-54dc14b4df02\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-7r79m" Jul 07 16:19:02.827017 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:02.826923 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 16:19:02.827017 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:02.826994 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/698c0fe4-2b45-4ffd-b498-54dc14b4df02-cluster-monitoring-operator-tls podName:698c0fe4-2b45-4ffd-b498-54dc14b4df02 nodeName:}" failed. No retries permitted until 2026-07-07 16:19:04.826976151 +0000 UTC m=+52.988789085 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/698c0fe4-2b45-4ffd-b498-54dc14b4df02-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-7r79m" (UID: "698c0fe4-2b45-4ffd-b498-54dc14b4df02") : secret "cluster-monitoring-operator-tls" not found Jul 07 16:19:02.928146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:02.928105 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ad867472-6624-4250-af20-f3402dca4af1-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bsqxn\" (UID: \"ad867472-6624-4250-af20-f3402dca4af1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bsqxn" Jul 07 16:19:02.928332 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:02.928265 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 16:19:02.928332 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:02.928328 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad867472-6624-4250-af20-f3402dca4af1-samples-operator-tls podName:ad867472-6624-4250-af20-f3402dca4af1 nodeName:}" failed. No retries permitted until 2026-07-07 16:19:04.928309129 +0000 UTC m=+53.090122075 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ad867472-6624-4250-af20-f3402dca4af1-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-bsqxn" (UID: "ad867472-6624-4250-af20-f3402dca4af1") : secret "samples-operator-tls" not found Jul 07 16:19:03.575194 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:03.575142 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-rhsmp" event={"ID":"6284964a-74fe-4d50-9a06-b13ba29d2d98","Type":"ContainerStarted","Data":"c7c4ddc14e8ed22331cf1d0c9f115b1ab55dcab5f274c5039dad328b75e96479"} Jul 07 16:19:03.599255 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:03.599200 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-rhsmp" podStartSLOduration=0.856847577 podStartE2EDuration="2.599184614s" totalImagesPullingTime="1.742337037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:19:03.598325312 +0000 UTC m=+51.760138261" watchObservedRunningTime="2026-07-07 16:19:03.599184614 +0000 UTC m=+51.760997568" Jul 07 16:19:04.843624 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:04.843576 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/698c0fe4-2b45-4ffd-b498-54dc14b4df02-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-7r79m\" (UID: \"698c0fe4-2b45-4ffd-b498-54dc14b4df02\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-7r79m" Jul 07 16:19:04.844053 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:04.843730 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 16:19:04.844053 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:04.843801 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/698c0fe4-2b45-4ffd-b498-54dc14b4df02-cluster-monitoring-operator-tls podName:698c0fe4-2b45-4ffd-b498-54dc14b4df02 nodeName:}" failed. No retries permitted until 2026-07-07 16:19:08.843781709 +0000 UTC m=+57.005594659 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/698c0fe4-2b45-4ffd-b498-54dc14b4df02-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-7r79m" (UID: "698c0fe4-2b45-4ffd-b498-54dc14b4df02") : secret "cluster-monitoring-operator-tls" not found Jul 07 16:19:04.944943 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:04.944914 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ad867472-6624-4250-af20-f3402dca4af1-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bsqxn\" (UID: \"ad867472-6624-4250-af20-f3402dca4af1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bsqxn" Jul 07 16:19:04.945039 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:04.945017 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 16:19:04.945094 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:04.945084 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad867472-6624-4250-af20-f3402dca4af1-samples-operator-tls podName:ad867472-6624-4250-af20-f3402dca4af1 nodeName:}" failed. No retries permitted until 2026-07-07 16:19:08.945065155 +0000 UTC m=+57.106878089 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ad867472-6624-4250-af20-f3402dca4af1-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-bsqxn" (UID: "ad867472-6624-4250-af20-f3402dca4af1") : secret "samples-operator-tls" not found Jul 07 16:19:05.549698 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:05.549650 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:19:05.549894 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:05.549746 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e68eced5-8d40-432a-95f7-3f5ec13e0b25-metrics-tls\") pod \"dns-default-gpl7j\" (UID: \"e68eced5-8d40-432a-95f7-3f5ec13e0b25\") " pod="openshift-dns/dns-default-gpl7j" Jul 07 16:19:05.549894 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:05.549824 2564 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 16:19:05.549894 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:05.549852 2564 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8c6f746f8-bb4kj: secret "image-registry-tls" not found Jul 07 16:19:05.549894 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:05.549880 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 16:19:05.550063 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:05.549939 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls podName:4955c880-3f01-4f4e-b9e3-6025c974796f nodeName:}" failed. No retries permitted until 2026-07-07 16:19:21.549900564 +0000 UTC m=+69.711713507 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls") pod "image-registry-8c6f746f8-bb4kj" (UID: "4955c880-3f01-4f4e-b9e3-6025c974796f") : secret "image-registry-tls" not found Jul 07 16:19:05.550063 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:05.549958 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e68eced5-8d40-432a-95f7-3f5ec13e0b25-metrics-tls podName:e68eced5-8d40-432a-95f7-3f5ec13e0b25 nodeName:}" failed. No retries permitted until 2026-07-07 16:19:21.549948397 +0000 UTC m=+69.711761333 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e68eced5-8d40-432a-95f7-3f5ec13e0b25-metrics-tls") pod "dns-default-gpl7j" (UID: "e68eced5-8d40-432a-95f7-3f5ec13e0b25") : secret "dns-default-metrics-tls" not found Jul 07 16:19:05.581605 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:05.581567 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xct9g" event={"ID":"c13e43af-02c1-486e-8ffd-e334bedac760","Type":"ContainerStarted","Data":"4dbf7933d3c0b87a08d1bf8eb4286c6309671daf9f708e9bb1d433ef3dfbfc52"} Jul 07 16:19:05.583088 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:05.583062 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" event={"ID":"40c9cc16-aae3-4bb5-b11a-7723051bb9c0","Type":"ContainerStarted","Data":"3e5b9d5d21e81917150203688c1db60c6443b8ea032f167b35704415d75e4cac"} Jul 07 16:19:05.601850 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:05.601791 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xct9g" podStartSLOduration=1.37543151 podStartE2EDuration="4.601772774s" totalImagesPullingTime="3.226341264s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:19:05.601631305 +0000 UTC m=+53.763444261" watchObservedRunningTime="2026-07-07 16:19:05.601772774 +0000 UTC m=+53.763585730" Jul 07 16:19:05.623686 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:05.623617 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" podStartSLOduration=1.3409260170000001 podStartE2EDuration="4.623595284s" totalImagesPullingTime="3.282669267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:19:05.622515113 +0000 UTC m=+53.784328069" watchObservedRunningTime="2026-07-07 16:19:05.623595284 +0000 UTC m=+53.785408246" Jul 07 16:19:05.998372 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:05.998337 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-n78cx"] Jul 07 16:19:06.000820 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:06.000799 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:06.012937 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:06.012892 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 16:19:06.013059 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:06.013005 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xmpnl\"" Jul 07 16:19:06.017223 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:06.017193 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n78cx"] Jul 07 16:19:06.018150 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:06.018132 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 16:19:06.053310 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:06.053275 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f029375b-5a3c-4990-9712-27f99db223ae-data-volume\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:06.053547 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:06.053348 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2dgm\" (UniqueName: \"kubernetes.io/projected/f029375b-5a3c-4990-9712-27f99db223ae-kube-api-access-x2dgm\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:06.053547 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:06.053399 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f029375b-5a3c-4990-9712-27f99db223ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:06.053547 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:06.053418 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f029375b-5a3c-4990-9712-27f99db223ae-crio-socket\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:06.053547 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:06.053469 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f029375b-5a3c-4990-9712-27f99db223ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:06.153892 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:06.153845 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2dgm\" (UniqueName: \"kubernetes.io/projected/f029375b-5a3c-4990-9712-27f99db223ae-kube-api-access-x2dgm\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:06.154079 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:06.154047 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f029375b-5a3c-4990-9712-27f99db223ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:06.154122 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:06.154079 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f029375b-5a3c-4990-9712-27f99db223ae-crio-socket\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:06.154122 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:06.154105 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f029375b-5a3c-4990-9712-27f99db223ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:06.154210 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:06.154183 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 16:19:06.154259 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:06.154221 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f029375b-5a3c-4990-9712-27f99db223ae-data-volume\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:06.154259 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:06.154249 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f029375b-5a3c-4990-9712-27f99db223ae-kube-rbac-proxy-cm podName:f029375b-5a3c-4990-9712-27f99db223ae nodeName:}" failed. No retries permitted until 2026-07-07 16:19:06.654230627 +0000 UTC m=+54.816043571 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f029375b-5a3c-4990-9712-27f99db223ae-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n78cx" (UID: "f029375b-5a3c-4990-9712-27f99db223ae") : configmap "kube-rbac-proxy" not found Jul 07 16:19:06.154351 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:06.154293 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 16:19:06.154351 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:06.154336 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f029375b-5a3c-4990-9712-27f99db223ae-crio-socket\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:06.154430 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:06.154363 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f029375b-5a3c-4990-9712-27f99db223ae-insights-runtime-extractor-tls podName:f029375b-5a3c-4990-9712-27f99db223ae nodeName:}" failed. No retries permitted until 2026-07-07 16:19:06.654345875 +0000 UTC m=+54.816158827 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f029375b-5a3c-4990-9712-27f99db223ae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n78cx" (UID: "f029375b-5a3c-4990-9712-27f99db223ae") : secret "insights-runtime-extractor-tls" not found Jul 07 16:19:06.154536 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:06.154519 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f029375b-5a3c-4990-9712-27f99db223ae-data-volume\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:06.162851 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:06.162827 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2dgm\" (UniqueName: \"kubernetes.io/projected/f029375b-5a3c-4990-9712-27f99db223ae-kube-api-access-x2dgm\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:06.658738 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:06.658698 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f029375b-5a3c-4990-9712-27f99db223ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:06.659030 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:06.658748 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f029375b-5a3c-4990-9712-27f99db223ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:06.659030 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:06.658828 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 16:19:06.659030 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:06.658832 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 16:19:06.659030 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:06.658885 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f029375b-5a3c-4990-9712-27f99db223ae-insights-runtime-extractor-tls podName:f029375b-5a3c-4990-9712-27f99db223ae nodeName:}" failed. No retries permitted until 2026-07-07 16:19:07.658871349 +0000 UTC m=+55.820684285 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f029375b-5a3c-4990-9712-27f99db223ae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n78cx" (UID: "f029375b-5a3c-4990-9712-27f99db223ae") : secret "insights-runtime-extractor-tls" not found Jul 07 16:19:06.659030 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:06.658961 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f029375b-5a3c-4990-9712-27f99db223ae-kube-rbac-proxy-cm podName:f029375b-5a3c-4990-9712-27f99db223ae nodeName:}" failed. No retries permitted until 2026-07-07 16:19:07.65894716 +0000 UTC m=+55.820760096 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f029375b-5a3c-4990-9712-27f99db223ae-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n78cx" (UID: "f029375b-5a3c-4990-9712-27f99db223ae") : configmap "kube-rbac-proxy" not found Jul 07 16:19:07.665784 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:07.665746 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f029375b-5a3c-4990-9712-27f99db223ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:07.665784 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:07.665786 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f029375b-5a3c-4990-9712-27f99db223ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:07.666239 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:07.665898 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 16:19:07.666239 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:07.665936 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 16:19:07.666239 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:07.666003 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f029375b-5a3c-4990-9712-27f99db223ae-insights-runtime-extractor-tls podName:f029375b-5a3c-4990-9712-27f99db223ae nodeName:}" failed. No retries permitted until 2026-07-07 16:19:09.665982644 +0000 UTC m=+57.827795577 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f029375b-5a3c-4990-9712-27f99db223ae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n78cx" (UID: "f029375b-5a3c-4990-9712-27f99db223ae") : secret "insights-runtime-extractor-tls" not found Jul 07 16:19:07.666239 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:07.666024 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f029375b-5a3c-4990-9712-27f99db223ae-kube-rbac-proxy-cm podName:f029375b-5a3c-4990-9712-27f99db223ae nodeName:}" failed. No retries permitted until 2026-07-07 16:19:09.66601481 +0000 UTC m=+57.827827744 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f029375b-5a3c-4990-9712-27f99db223ae-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n78cx" (UID: "f029375b-5a3c-4990-9712-27f99db223ae") : configmap "kube-rbac-proxy" not found Jul 07 16:19:08.000547 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:08.000469 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lxccs_83dce208-f3dc-4e5c-ba27-e2d6c9472a68/dns-node-resolver/0.log" Jul 07 16:19:08.875968 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:08.875893 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/698c0fe4-2b45-4ffd-b498-54dc14b4df02-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-7r79m\" (UID: \"698c0fe4-2b45-4ffd-b498-54dc14b4df02\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-7r79m" Jul 07 16:19:08.876378 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:08.876039 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 16:19:08.876378 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:08.876105 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/698c0fe4-2b45-4ffd-b498-54dc14b4df02-cluster-monitoring-operator-tls podName:698c0fe4-2b45-4ffd-b498-54dc14b4df02 nodeName:}" failed. No retries permitted until 2026-07-07 16:19:16.876089198 +0000 UTC m=+65.037902131 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/698c0fe4-2b45-4ffd-b498-54dc14b4df02-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-7r79m" (UID: "698c0fe4-2b45-4ffd-b498-54dc14b4df02") : secret "cluster-monitoring-operator-tls" not found Jul 07 16:19:08.976456 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:08.976425 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ad867472-6624-4250-af20-f3402dca4af1-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bsqxn\" (UID: \"ad867472-6624-4250-af20-f3402dca4af1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bsqxn" Jul 07 16:19:08.976589 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:08.976568 2564 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 16:19:08.976647 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:08.976635 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad867472-6624-4250-af20-f3402dca4af1-samples-operator-tls podName:ad867472-6624-4250-af20-f3402dca4af1 nodeName:}" failed. No retries permitted until 2026-07-07 16:19:16.976620311 +0000 UTC m=+65.138433245 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/ad867472-6624-4250-af20-f3402dca4af1-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-bsqxn" (UID: "ad867472-6624-4250-af20-f3402dca4af1") : secret "samples-operator-tls" not found Jul 07 16:19:09.200210 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:09.200142 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vjgrf_709a00b4-4252-4273-ad18-72ac558f394c/node-ca/0.log" Jul 07 16:19:09.601825 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:09.601751 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8hr9f_ebc29602-88aa-4ce8-aa11-341d20706083/serve-healthcheck-canary/0.log" Jul 07 16:19:09.680792 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:09.680758 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f029375b-5a3c-4990-9712-27f99db223ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:09.680965 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:09.680797 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f029375b-5a3c-4990-9712-27f99db223ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:09.680965 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:09.680926 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 16:19:09.680965 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:09.680933 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 16:19:09.681088 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:09.680990 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f029375b-5a3c-4990-9712-27f99db223ae-insights-runtime-extractor-tls podName:f029375b-5a3c-4990-9712-27f99db223ae nodeName:}" failed. No retries permitted until 2026-07-07 16:19:13.680974453 +0000 UTC m=+61.842787386 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f029375b-5a3c-4990-9712-27f99db223ae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n78cx" (UID: "f029375b-5a3c-4990-9712-27f99db223ae") : secret "insights-runtime-extractor-tls" not found Jul 07 16:19:09.681088 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:09.681004 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f029375b-5a3c-4990-9712-27f99db223ae-kube-rbac-proxy-cm podName:f029375b-5a3c-4990-9712-27f99db223ae nodeName:}" failed. No retries permitted until 2026-07-07 16:19:13.680998647 +0000 UTC m=+61.842811581 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f029375b-5a3c-4990-9712-27f99db223ae-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n78cx" (UID: "f029375b-5a3c-4990-9712-27f99db223ae") : configmap "kube-rbac-proxy" not found Jul 07 16:19:13.716390 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:13.716345 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f029375b-5a3c-4990-9712-27f99db223ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:13.716390 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:13.716390 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f029375b-5a3c-4990-9712-27f99db223ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:13.716951 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:13.716511 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 16:19:13.716951 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:13.716564 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f029375b-5a3c-4990-9712-27f99db223ae-kube-rbac-proxy-cm podName:f029375b-5a3c-4990-9712-27f99db223ae nodeName:}" failed. No retries permitted until 2026-07-07 16:19:21.716550947 +0000 UTC m=+69.878363879 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f029375b-5a3c-4990-9712-27f99db223ae-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n78cx" (UID: "f029375b-5a3c-4990-9712-27f99db223ae") : configmap "kube-rbac-proxy" not found Jul 07 16:19:13.716951 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:13.716510 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 16:19:13.716951 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:13.716654 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f029375b-5a3c-4990-9712-27f99db223ae-insights-runtime-extractor-tls podName:f029375b-5a3c-4990-9712-27f99db223ae nodeName:}" failed. No retries permitted until 2026-07-07 16:19:21.716639348 +0000 UTC m=+69.878452282 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f029375b-5a3c-4990-9712-27f99db223ae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n78cx" (UID: "f029375b-5a3c-4990-9712-27f99db223ae") : secret "insights-runtime-extractor-tls" not found Jul 07 16:19:15.509435 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:15.509408 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q5hvx" Jul 07 16:19:16.942268 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:16.942206 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/698c0fe4-2b45-4ffd-b498-54dc14b4df02-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-7r79m\" (UID: \"698c0fe4-2b45-4ffd-b498-54dc14b4df02\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-7r79m" Jul 07 16:19:16.942701 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:16.942368 2564 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 16:19:16.942701 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:16.942438 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/698c0fe4-2b45-4ffd-b498-54dc14b4df02-cluster-monitoring-operator-tls podName:698c0fe4-2b45-4ffd-b498-54dc14b4df02 nodeName:}" failed. No retries permitted until 2026-07-07 16:19:32.942420773 +0000 UTC m=+81.104233716 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/698c0fe4-2b45-4ffd-b498-54dc14b4df02-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-7r79m" (UID: "698c0fe4-2b45-4ffd-b498-54dc14b4df02") : secret "cluster-monitoring-operator-tls" not found Jul 07 16:19:17.043011 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:17.042970 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ad867472-6624-4250-af20-f3402dca4af1-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bsqxn\" (UID: \"ad867472-6624-4250-af20-f3402dca4af1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bsqxn" Jul 07 16:19:17.045340 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:17.045308 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/ad867472-6624-4250-af20-f3402dca4af1-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bsqxn\" (UID: \"ad867472-6624-4250-af20-f3402dca4af1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bsqxn" Jul 07 16:19:17.098793 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:17.098756 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-2k2z6\"" Jul 07 16:19:17.106957 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:17.106930 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bsqxn" Jul 07 16:19:17.232555 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:17.232474 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bsqxn"] Jul 07 16:19:17.613051 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:17.613017 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bsqxn" event={"ID":"ad867472-6624-4250-af20-f3402dca4af1","Type":"ContainerStarted","Data":"7a5b2fb028b4c08e5a89a26b9577aee836033b666a2ae14bd7c1e4216134791f"} Jul 07 16:19:18.153849 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:18.153817 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs\") pod \"network-metrics-daemon-xsj8b\" (UID: \"97f47a3d-062a-4159-8ebd-96d0afd8372b\") " pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:19:18.154240 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:18.153871 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggx76\" (UniqueName: \"kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76\") pod \"network-check-target-g9pff\" (UID: \"83eacc13-cd9b-436a-882b-18c36f62e74c\") " pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:19:18.154240 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:18.153894 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret\") pod \"global-pull-secret-syncer-868j2\" (UID: \"093da19e-76ae-42f9-b23a-5ac90eb50fb1\") " pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:19:18.156448 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:18.156428 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 16:19:18.156610 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:18.156590 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 16:19:18.156643 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:18.156603 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 16:19:18.164882 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:18.164854 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 07 16:19:18.165033 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:18.164941 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs podName:97f47a3d-062a-4159-8ebd-96d0afd8372b nodeName:}" failed. No retries permitted until 2026-07-07 16:20:22.164919583 +0000 UTC m=+130.326732530 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs") pod "network-metrics-daemon-xsj8b" (UID: "97f47a3d-062a-4159-8ebd-96d0afd8372b") : secret "metrics-daemon-secret" not found Jul 07 16:19:18.167168 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:18.167141 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 16:19:18.167442 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:18.167423 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/093da19e-76ae-42f9-b23a-5ac90eb50fb1-original-pull-secret\") pod \"global-pull-secret-syncer-868j2\" (UID: \"093da19e-76ae-42f9-b23a-5ac90eb50fb1\") " pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:19:18.177888 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:18.177866 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggx76\" (UniqueName: \"kubernetes.io/projected/83eacc13-cd9b-436a-882b-18c36f62e74c-kube-api-access-ggx76\") pod \"network-check-target-g9pff\" (UID: \"83eacc13-cd9b-436a-882b-18c36f62e74c\") " pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:19:18.355087 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:18.355054 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-868j2" Jul 07 16:19:18.362788 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:18.362762 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-68hwt\"" Jul 07 16:19:18.370836 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:18.370803 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:19:18.506627 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:18.506590 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-868j2"] Jul 07 16:19:18.509604 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:19:18.509572 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod093da19e_76ae_42f9_b23a_5ac90eb50fb1.slice/crio-6eac8c1a71672e6e88804fa61f93efc5547546a83b296706b50b1ace0013eabb WatchSource:0}: Error finding container 6eac8c1a71672e6e88804fa61f93efc5547546a83b296706b50b1ace0013eabb: Status 404 returned error can't find the container with id 6eac8c1a71672e6e88804fa61f93efc5547546a83b296706b50b1ace0013eabb Jul 07 16:19:18.522697 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:18.522670 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-g9pff"] Jul 07 16:19:18.525709 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:19:18.525680 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83eacc13_cd9b_436a_882b_18c36f62e74c.slice/crio-a5625d1705d6859a0b42b4bfe60f036a550914b94fdae3c9de7b5c98d986ccc5 WatchSource:0}: Error finding container a5625d1705d6859a0b42b4bfe60f036a550914b94fdae3c9de7b5c98d986ccc5: Status 404 returned error can't find the container with id a5625d1705d6859a0b42b4bfe60f036a550914b94fdae3c9de7b5c98d986ccc5 Jul 07 16:19:18.616923 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:18.616866 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-868j2" event={"ID":"093da19e-76ae-42f9-b23a-5ac90eb50fb1","Type":"ContainerStarted","Data":"6eac8c1a71672e6e88804fa61f93efc5547546a83b296706b50b1ace0013eabb"} Jul 07 16:19:18.617991 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:18.617965 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g9pff" event={"ID":"83eacc13-cd9b-436a-882b-18c36f62e74c","Type":"ContainerStarted","Data":"a5625d1705d6859a0b42b4bfe60f036a550914b94fdae3c9de7b5c98d986ccc5"} Jul 07 16:19:20.627313 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:20.627280 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bsqxn" event={"ID":"ad867472-6624-4250-af20-f3402dca4af1","Type":"ContainerStarted","Data":"6fe25120bb371db32138bda576325344d6689990cf4656a37e314b845a0e92f1"} Jul 07 16:19:20.627730 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:20.627338 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bsqxn" event={"ID":"ad867472-6624-4250-af20-f3402dca4af1","Type":"ContainerStarted","Data":"6de5269271484f9952c68f0da3443bd76d04e6dc68ecfc16bfa46ad8435a674c"} Jul 07 16:19:20.650580 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:20.650515 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bsqxn" podStartSLOduration=16.93615637 podStartE2EDuration="19.650496152s" totalImagesPullingTime="2.714339782s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:19:20.650239223 +0000 UTC m=+68.812052179" watchObservedRunningTime="2026-07-07 16:19:20.650496152 +0000 UTC m=+68.812309110" Jul 07 16:19:21.581658 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:21.581617 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:19:21.581860 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:21.581694 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e68eced5-8d40-432a-95f7-3f5ec13e0b25-metrics-tls\") pod \"dns-default-gpl7j\" (UID: \"e68eced5-8d40-432a-95f7-3f5ec13e0b25\") " pod="openshift-dns/dns-default-gpl7j" Jul 07 16:19:21.584737 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:21.584708 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls\") pod \"image-registry-8c6f746f8-bb4kj\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:19:21.584870 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:21.584786 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e68eced5-8d40-432a-95f7-3f5ec13e0b25-metrics-tls\") pod \"dns-default-gpl7j\" (UID: \"e68eced5-8d40-432a-95f7-3f5ec13e0b25\") " pod="openshift-dns/dns-default-gpl7j" Jul 07 16:19:21.783599 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:21.783555 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f029375b-5a3c-4990-9712-27f99db223ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:21.783599 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:21.783610 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f029375b-5a3c-4990-9712-27f99db223ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:21.784116 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:21.783726 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 16:19:21.784116 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:21.783814 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f029375b-5a3c-4990-9712-27f99db223ae-insights-runtime-extractor-tls podName:f029375b-5a3c-4990-9712-27f99db223ae nodeName:}" failed. No retries permitted until 2026-07-07 16:19:37.783796991 +0000 UTC m=+85.945609924 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f029375b-5a3c-4990-9712-27f99db223ae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n78cx" (UID: "f029375b-5a3c-4990-9712-27f99db223ae") : secret "insights-runtime-extractor-tls" not found Jul 07 16:19:21.784309 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:21.784290 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f029375b-5a3c-4990-9712-27f99db223ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:21.828960 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:21.828928 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-p9zlp\"" Jul 07 16:19:21.836509 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:21.836425 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:19:21.843849 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:21.843828 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2sv27\"" Jul 07 16:19:21.852193 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:21.852171 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gpl7j" Jul 07 16:19:21.995102 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:21.995070 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8c6f746f8-bb4kj"] Jul 07 16:19:21.998229 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:19:21.998196 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4955c880_3f01_4f4e_b9e3_6025c974796f.slice/crio-d3dde68266362799b5290624fb10d9efb1e72650445907069e04a097eac3ca42 WatchSource:0}: Error finding container d3dde68266362799b5290624fb10d9efb1e72650445907069e04a097eac3ca42: Status 404 returned error can't find the container with id d3dde68266362799b5290624fb10d9efb1e72650445907069e04a097eac3ca42 Jul 07 16:19:22.004342 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:22.004300 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gpl7j"] Jul 07 16:19:22.010272 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:19:22.010244 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode68eced5_8d40_432a_95f7_3f5ec13e0b25.slice/crio-2fb4456f87dbe600c9a34e25490a346c1d249ec2875346a2e3e7da0a5788de38 WatchSource:0}: Error finding container 2fb4456f87dbe600c9a34e25490a346c1d249ec2875346a2e3e7da0a5788de38: Status 404 returned error can't find the container with id 2fb4456f87dbe600c9a34e25490a346c1d249ec2875346a2e3e7da0a5788de38 Jul 07 16:19:22.636775 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:22.636734 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gpl7j" event={"ID":"e68eced5-8d40-432a-95f7-3f5ec13e0b25","Type":"ContainerStarted","Data":"2fb4456f87dbe600c9a34e25490a346c1d249ec2875346a2e3e7da0a5788de38"} Jul 07 16:19:22.638547 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:22.638518 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-g9pff" event={"ID":"83eacc13-cd9b-436a-882b-18c36f62e74c","Type":"ContainerStarted","Data":"5913bc316225fffd660e0620c058d069110142e77e50d4a3a0dea2946117dc50"} Jul 07 16:19:22.638695 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:22.638609 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:19:22.640403 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:22.640368 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" event={"ID":"4955c880-3f01-4f4e-b9e3-6025c974796f","Type":"ContainerStarted","Data":"98fe3bc5cd665209c1599dc6943ea5ee9e29c69a361dc58e9fffe28964be2d89"} Jul 07 16:19:22.640549 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:22.640423 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" event={"ID":"4955c880-3f01-4f4e-b9e3-6025c974796f","Type":"ContainerStarted","Data":"d3dde68266362799b5290624fb10d9efb1e72650445907069e04a097eac3ca42"} Jul 07 16:19:22.640549 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:22.640508 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:19:22.653690 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:22.653644 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-g9pff" podStartSLOduration=67.345792651 podStartE2EDuration="1m10.653630457s" totalImagesPullingTime="3.307837806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:18:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:19:22.652642816 +0000 UTC m=+70.814455773" watchObservedRunningTime="2026-07-07 16:19:22.653630457 +0000 UTC m=+70.815443412" Jul 07 16:19:22.674284 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:22.674220 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" podStartSLOduration=69.674202105 podStartE2EDuration="1m9.674202105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:18:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:19:22.673951335 +0000 UTC m=+70.835764289" watchObservedRunningTime="2026-07-07 16:19:22.674202105 +0000 UTC m=+70.836015093" Jul 07 16:19:26.658285 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:26.658251 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gpl7j" event={"ID":"e68eced5-8d40-432a-95f7-3f5ec13e0b25","Type":"ContainerStarted","Data":"d787e5ba35f533af1373c6efa591ea4e4987ebe49362b19b58dbfd95e517222c"} Jul 07 16:19:26.658792 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:26.658306 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gpl7j" event={"ID":"e68eced5-8d40-432a-95f7-3f5ec13e0b25","Type":"ContainerStarted","Data":"c88e29ef9e8f3aef09d268e84d5fc0e609572a5f8dabfcfd8826861d652719a4"} Jul 07 16:19:26.658792 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:26.658370 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gpl7j" Jul 07 16:19:26.659727 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:26.659708 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-868j2" event={"ID":"093da19e-76ae-42f9-b23a-5ac90eb50fb1","Type":"ContainerStarted","Data":"ab76fc48857aaf177229e4e5f3e33093e9d1233f5bc194f8770d045ade54faca"} Jul 07 16:19:26.678449 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:26.678404 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gpl7j" podStartSLOduration=33.508907584 podStartE2EDuration="37.678389069s" totalImagesPullingTime="4.169481485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:18:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:19:26.677166389 +0000 UTC m=+74.838979344" watchObservedRunningTime="2026-07-07 16:19:26.678389069 +0000 UTC m=+74.840202033" Jul 07 16:19:26.697615 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:26.697566 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-868j2" podStartSLOduration=67.016736275 podStartE2EDuration="1m14.697549539s" totalImagesPullingTime="7.680813264s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 16:18:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:19:26.696783813 +0000 UTC m=+74.858596769" watchObservedRunningTime="2026-07-07 16:19:26.697549539 +0000 UTC m=+74.859362515" Jul 07 16:19:29.751482 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:29.751449 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584676cd8-9xqp2"] Jul 07 16:19:29.755561 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:29.755537 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584676cd8-9xqp2" Jul 07 16:19:29.766802 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:29.766774 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 07 16:19:29.767839 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:29.767821 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 07 16:19:29.772168 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:29.772149 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 07 16:19:29.772255 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:29.772220 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 07 16:19:29.797215 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:29.797181 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584676cd8-9xqp2"] Jul 07 16:19:29.850319 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:29.850288 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/408bbc42-9a5b-4649-8d0e-c57db4de4b0a-klusterlet-config\") pod \"klusterlet-addon-workmgr-5584676cd8-9xqp2\" (UID: \"408bbc42-9a5b-4649-8d0e-c57db4de4b0a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584676cd8-9xqp2" Jul 07 16:19:29.850513 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:29.850343 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4xd4\" (UniqueName: \"kubernetes.io/projected/408bbc42-9a5b-4649-8d0e-c57db4de4b0a-kube-api-access-v4xd4\") pod \"klusterlet-addon-workmgr-5584676cd8-9xqp2\" (UID: \"408bbc42-9a5b-4649-8d0e-c57db4de4b0a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584676cd8-9xqp2" Jul 07 16:19:29.850513 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:29.850455 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/408bbc42-9a5b-4649-8d0e-c57db4de4b0a-tmp\") pod \"klusterlet-addon-workmgr-5584676cd8-9xqp2\" (UID: \"408bbc42-9a5b-4649-8d0e-c57db4de4b0a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584676cd8-9xqp2" Jul 07 16:19:29.951066 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:29.951031 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4xd4\" (UniqueName: \"kubernetes.io/projected/408bbc42-9a5b-4649-8d0e-c57db4de4b0a-kube-api-access-v4xd4\") pod \"klusterlet-addon-workmgr-5584676cd8-9xqp2\" (UID: \"408bbc42-9a5b-4649-8d0e-c57db4de4b0a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584676cd8-9xqp2" Jul 07 16:19:29.951269 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:29.951092 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/408bbc42-9a5b-4649-8d0e-c57db4de4b0a-tmp\") pod \"klusterlet-addon-workmgr-5584676cd8-9xqp2\" (UID: \"408bbc42-9a5b-4649-8d0e-c57db4de4b0a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584676cd8-9xqp2" Jul 07 16:19:29.951269 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:29.951141 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/408bbc42-9a5b-4649-8d0e-c57db4de4b0a-klusterlet-config\") pod \"klusterlet-addon-workmgr-5584676cd8-9xqp2\" (UID: \"408bbc42-9a5b-4649-8d0e-c57db4de4b0a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584676cd8-9xqp2" Jul 07 16:19:29.951480 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:29.951459 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/408bbc42-9a5b-4649-8d0e-c57db4de4b0a-tmp\") pod \"klusterlet-addon-workmgr-5584676cd8-9xqp2\" (UID: \"408bbc42-9a5b-4649-8d0e-c57db4de4b0a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584676cd8-9xqp2" Jul 07 16:19:29.953663 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:29.953641 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/408bbc42-9a5b-4649-8d0e-c57db4de4b0a-klusterlet-config\") pod \"klusterlet-addon-workmgr-5584676cd8-9xqp2\" (UID: \"408bbc42-9a5b-4649-8d0e-c57db4de4b0a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584676cd8-9xqp2" Jul 07 16:19:29.965292 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:29.965261 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4xd4\" (UniqueName: \"kubernetes.io/projected/408bbc42-9a5b-4649-8d0e-c57db4de4b0a-kube-api-access-v4xd4\") pod \"klusterlet-addon-workmgr-5584676cd8-9xqp2\" (UID: \"408bbc42-9a5b-4649-8d0e-c57db4de4b0a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584676cd8-9xqp2" Jul 07 16:19:30.065269 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:30.065172 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584676cd8-9xqp2" Jul 07 16:19:30.202133 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:30.202095 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584676cd8-9xqp2"] Jul 07 16:19:30.205137 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:19:30.205105 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod408bbc42_9a5b_4649_8d0e_c57db4de4b0a.slice/crio-ffd5028c2f35f30285163054d1b03930fd9d2c5d071f2de8cb7b3ea21cbbcb0d WatchSource:0}: Error finding container ffd5028c2f35f30285163054d1b03930fd9d2c5d071f2de8cb7b3ea21cbbcb0d: Status 404 returned error can't find the container with id ffd5028c2f35f30285163054d1b03930fd9d2c5d071f2de8cb7b3ea21cbbcb0d Jul 07 16:19:30.677267 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:30.677225 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584676cd8-9xqp2" event={"ID":"408bbc42-9a5b-4649-8d0e-c57db4de4b0a","Type":"ContainerStarted","Data":"ffd5028c2f35f30285163054d1b03930fd9d2c5d071f2de8cb7b3ea21cbbcb0d"} Jul 07 16:19:32.973778 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:32.973734 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/698c0fe4-2b45-4ffd-b498-54dc14b4df02-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-7r79m\" (UID: \"698c0fe4-2b45-4ffd-b498-54dc14b4df02\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-7r79m" Jul 07 16:19:32.976467 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:32.976433 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/698c0fe4-2b45-4ffd-b498-54dc14b4df02-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-7r79m\" (UID: \"698c0fe4-2b45-4ffd-b498-54dc14b4df02\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-7r79m" Jul 07 16:19:33.191468 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:33.191437 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-xl7xk\"" Jul 07 16:19:33.199611 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:33.199571 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-7r79m" Jul 07 16:19:33.830471 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:33.830437 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-7r79m"] Jul 07 16:19:33.833252 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:19:33.833221 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod698c0fe4_2b45_4ffd_b498_54dc14b4df02.slice/crio-6d1535125c7ed3fd3a52548d017f99727be7b26e3d27aa298b80e9e503e326df WatchSource:0}: Error finding container 6d1535125c7ed3fd3a52548d017f99727be7b26e3d27aa298b80e9e503e326df: Status 404 returned error can't find the container with id 6d1535125c7ed3fd3a52548d017f99727be7b26e3d27aa298b80e9e503e326df Jul 07 16:19:34.694791 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:34.694648 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584676cd8-9xqp2" event={"ID":"408bbc42-9a5b-4649-8d0e-c57db4de4b0a","Type":"ContainerStarted","Data":"8b67b770b2559ad164aabf3d1c640079c11b68ede40561c7f7def55fa0df611b"} Jul 07 16:19:34.695333 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:34.694846 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584676cd8-9xqp2" Jul 07 16:19:34.696047 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:34.696021 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-7r79m" event={"ID":"698c0fe4-2b45-4ffd-b498-54dc14b4df02","Type":"ContainerStarted","Data":"6d1535125c7ed3fd3a52548d017f99727be7b26e3d27aa298b80e9e503e326df"} Jul 07 16:19:34.696728 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:34.696706 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584676cd8-9xqp2" Jul 07 16:19:34.712334 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:34.712274 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5584676cd8-9xqp2" podStartSLOduration=2.146154213 podStartE2EDuration="5.712256108s" totalImagesPullingTime="3.566101895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:19:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:19:34.710411167 +0000 UTC m=+82.872224122" watchObservedRunningTime="2026-07-07 16:19:34.712256108 +0000 UTC m=+82.874069063" Jul 07 16:19:36.665870 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:36.665842 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gpl7j" Jul 07 16:19:36.704839 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:36.704802 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-7r79m" event={"ID":"698c0fe4-2b45-4ffd-b498-54dc14b4df02","Type":"ContainerStarted","Data":"a907e8ce8c29dad439adf3076b6c9fcce188308d404822fd689451a3e1b7534f"} Jul 07 16:19:36.730563 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:36.730502 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-7r79m" podStartSLOduration=33.49640064 podStartE2EDuration="35.730478699s" totalImagesPullingTime="2.234078059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:19:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:19:36.72960698 +0000 UTC m=+84.891419936" watchObservedRunningTime="2026-07-07 16:19:36.730478699 +0000 UTC m=+84.892291655" Jul 07 16:19:37.813555 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:37.813513 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f029375b-5a3c-4990-9712-27f99db223ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:37.815953 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:37.815898 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f029375b-5a3c-4990-9712-27f99db223ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n78cx\" (UID: \"f029375b-5a3c-4990-9712-27f99db223ae\") " pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:38.114546 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:38.114520 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-xmpnl\"" Jul 07 16:19:38.121785 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:38.121759 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n78cx" Jul 07 16:19:38.288868 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:38.288838 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n78cx"] Jul 07 16:19:38.291934 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:19:38.291889 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf029375b_5a3c_4990_9712_27f99db223ae.slice/crio-410a86f41d2a725146995cce76c0ad148d68013733332ff6e1c82848745e8a73 WatchSource:0}: Error finding container 410a86f41d2a725146995cce76c0ad148d68013733332ff6e1c82848745e8a73: Status 404 returned error can't find the container with id 410a86f41d2a725146995cce76c0ad148d68013733332ff6e1c82848745e8a73 Jul 07 16:19:38.712926 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:38.712874 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n78cx" event={"ID":"f029375b-5a3c-4990-9712-27f99db223ae","Type":"ContainerStarted","Data":"410a86f41d2a725146995cce76c0ad148d68013733332ff6e1c82848745e8a73"} Jul 07 16:19:41.842592 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:41.842549 2564 patch_prober.go:28] interesting pod/image-registry-8c6f746f8-bb4kj container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 07 16:19:41.843075 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:41.842616 2564 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" podUID="4955c880-3f01-4f4e-b9e3-6025c974796f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 16:19:42.728348 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:42.728314 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n78cx" event={"ID":"f029375b-5a3c-4990-9712-27f99db223ae","Type":"ContainerStarted","Data":"89eca562e60a9b909ce833477fb673aab0626dddcef3ee167fa980a77ef4da03"} Jul 07 16:19:43.651035 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:43.651004 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:19:43.733204 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:43.733170 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n78cx" event={"ID":"f029375b-5a3c-4990-9712-27f99db223ae","Type":"ContainerStarted","Data":"cabfb340531ee07f2c9a70d0584bce3761c377cd8e746f3bac91c8c81b617a8f"} Jul 07 16:19:45.742214 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:45.742129 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n78cx" event={"ID":"f029375b-5a3c-4990-9712-27f99db223ae","Type":"ContainerStarted","Data":"085b09084f259c6a5b923c6ec73aa5b929cc37287c9aebd5bc1fe0e7af2eb0ba"} Jul 07 16:19:45.763373 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:45.763317 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-n78cx" podStartSLOduration=33.765735384 podStartE2EDuration="40.763300042s" totalImagesPullingTime="6.997564658s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 16:19:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:19:45.761396735 +0000 UTC m=+93.923209692" watchObservedRunningTime="2026-07-07 16:19:45.763300042 +0000 UTC m=+93.925112997" Jul 07 16:19:46.695800 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:46.695762 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-t2wmq"] Jul 07 16:19:46.698822 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:46.698804 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-t2wmq" Jul 07 16:19:46.702442 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:46.702414 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 16:19:46.702602 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:46.702581 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-wg8dg\"" Jul 07 16:19:46.703260 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:46.703245 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 16:19:46.726820 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:46.726791 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-t2wmq"] Jul 07 16:19:46.785499 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:46.785466 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4vdc\" (UniqueName: \"kubernetes.io/projected/68959a9c-22fd-4804-9b45-0eb23f396983-kube-api-access-g4vdc\") pod \"downloads-685bfb5f64-t2wmq\" (UID: \"68959a9c-22fd-4804-9b45-0eb23f396983\") " pod="openshift-console/downloads-685bfb5f64-t2wmq" Jul 07 16:19:46.886646 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:46.886611 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4vdc\" (UniqueName: \"kubernetes.io/projected/68959a9c-22fd-4804-9b45-0eb23f396983-kube-api-access-g4vdc\") pod \"downloads-685bfb5f64-t2wmq\" (UID: \"68959a9c-22fd-4804-9b45-0eb23f396983\") " pod="openshift-console/downloads-685bfb5f64-t2wmq" Jul 07 16:19:46.902677 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:46.902647 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4vdc\" (UniqueName: \"kubernetes.io/projected/68959a9c-22fd-4804-9b45-0eb23f396983-kube-api-access-g4vdc\") pod \"downloads-685bfb5f64-t2wmq\" (UID: \"68959a9c-22fd-4804-9b45-0eb23f396983\") " pod="openshift-console/downloads-685bfb5f64-t2wmq" Jul 07 16:19:47.008379 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.008288 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-t2wmq" Jul 07 16:19:47.156660 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.156629 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-t2wmq"] Jul 07 16:19:47.160000 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:19:47.159972 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68959a9c_22fd_4804_9b45_0eb23f396983.slice/crio-40c40b69290a1b922789187c27913168b9ab027b02a70e3f0e450297cf93e6d4 WatchSource:0}: Error finding container 40c40b69290a1b922789187c27913168b9ab027b02a70e3f0e450297cf93e6d4: Status 404 returned error can't find the container with id 40c40b69290a1b922789187c27913168b9ab027b02a70e3f0e450297cf93e6d4 Jul 07 16:19:47.262844 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.262765 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-d89dt"] Jul 07 16:19:47.267175 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.267153 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-ptnpp"] Jul 07 16:19:47.267303 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.267283 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.270181 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.270160 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-ptnpp" Jul 07 16:19:47.276514 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.276490 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 07 16:19:47.277284 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.277267 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 16:19:47.277398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.277381 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 16:19:47.277447 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.277394 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-4wvwn\"" Jul 07 16:19:47.277618 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.277598 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-5jpbc\"" Jul 07 16:19:47.277701 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.277677 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 16:19:47.277771 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.277678 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 07 16:19:47.281845 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.281828 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 16:19:47.293964 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.293937 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-ptnpp"] Jul 07 16:19:47.391101 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.391068 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-sys\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.391101 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.391103 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cdb29d62-6eb7-406a-b6fb-249a2bd04e4f-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-ptnpp\" (UID: \"cdb29d62-6eb7-406a-b6fb-249a2bd04e4f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-ptnpp" Jul 07 16:19:47.391304 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.391125 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xj52\" (UniqueName: \"kubernetes.io/projected/cdb29d62-6eb7-406a-b6fb-249a2bd04e4f-kube-api-access-4xj52\") pod \"openshift-state-metrics-54cf96dfcf-ptnpp\" (UID: \"cdb29d62-6eb7-406a-b6fb-249a2bd04e4f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-ptnpp" Jul 07 16:19:47.391304 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.391149 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.391304 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.391209 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-root\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.391304 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.391250 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-node-exporter-wtmp\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.391304 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.391272 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-node-exporter-accelerators-collector-config\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.391304 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.391291 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cdb29d62-6eb7-406a-b6fb-249a2bd04e4f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-ptnpp\" (UID: \"cdb29d62-6eb7-406a-b6fb-249a2bd04e4f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-ptnpp" Jul 07 16:19:47.391489 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.391316 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cdb29d62-6eb7-406a-b6fb-249a2bd04e4f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-ptnpp\" (UID: \"cdb29d62-6eb7-406a-b6fb-249a2bd04e4f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-ptnpp" Jul 07 16:19:47.391489 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.391369 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-node-exporter-tls\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.391489 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.391403 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-metrics-client-ca\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.391489 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.391429 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf7jr\" (UniqueName: \"kubernetes.io/projected/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-kube-api-access-xf7jr\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.391489 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.391447 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-node-exporter-textfile\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.492146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.492113 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-root\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.492325 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.492157 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-node-exporter-wtmp\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.492325 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.492189 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-node-exporter-accelerators-collector-config\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.492325 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.492207 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-root\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.492325 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.492218 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cdb29d62-6eb7-406a-b6fb-249a2bd04e4f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-ptnpp\" (UID: \"cdb29d62-6eb7-406a-b6fb-249a2bd04e4f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-ptnpp" Jul 07 16:19:47.492325 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.492272 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cdb29d62-6eb7-406a-b6fb-249a2bd04e4f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-ptnpp\" (UID: \"cdb29d62-6eb7-406a-b6fb-249a2bd04e4f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-ptnpp" Jul 07 16:19:47.492325 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.492292 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-node-exporter-wtmp\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.492325 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.492306 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-node-exporter-tls\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.492683 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.492355 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-metrics-client-ca\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.492683 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.492375 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf7jr\" (UniqueName: \"kubernetes.io/projected/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-kube-api-access-xf7jr\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.492683 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:47.492377 2564 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 07 16:19:47.492683 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:47.492485 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-node-exporter-tls podName:3b83dad7-acd2-43fe-8b45-b549bb50b2c3 nodeName:}" failed. No retries permitted until 2026-07-07 16:19:47.99246049 +0000 UTC m=+96.154273433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-node-exporter-tls") pod "node-exporter-d89dt" (UID: "3b83dad7-acd2-43fe-8b45-b549bb50b2c3") : secret "node-exporter-tls" not found Jul 07 16:19:47.492683 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.492404 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-node-exporter-textfile\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.492683 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.492646 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-node-exporter-textfile\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.492683 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.492666 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-sys\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.493052 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.492703 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cdb29d62-6eb7-406a-b6fb-249a2bd04e4f-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-ptnpp\" (UID: \"cdb29d62-6eb7-406a-b6fb-249a2bd04e4f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-ptnpp" Jul 07 16:19:47.493052 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.492735 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xj52\" (UniqueName: \"kubernetes.io/projected/cdb29d62-6eb7-406a-b6fb-249a2bd04e4f-kube-api-access-4xj52\") pod \"openshift-state-metrics-54cf96dfcf-ptnpp\" (UID: \"cdb29d62-6eb7-406a-b6fb-249a2bd04e4f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-ptnpp" Jul 07 16:19:47.493052 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.492785 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-sys\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.493052 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.492785 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.493052 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.492895 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-node-exporter-accelerators-collector-config\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.493261 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.493237 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-metrics-client-ca\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.493729 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.493702 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cdb29d62-6eb7-406a-b6fb-249a2bd04e4f-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-ptnpp\" (UID: \"cdb29d62-6eb7-406a-b6fb-249a2bd04e4f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-ptnpp" Jul 07 16:19:47.495013 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.494988 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cdb29d62-6eb7-406a-b6fb-249a2bd04e4f-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-ptnpp\" (UID: \"cdb29d62-6eb7-406a-b6fb-249a2bd04e4f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-ptnpp" Jul 07 16:19:47.495013 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.495005 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/cdb29d62-6eb7-406a-b6fb-249a2bd04e4f-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-ptnpp\" (UID: \"cdb29d62-6eb7-406a-b6fb-249a2bd04e4f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-ptnpp" Jul 07 16:19:47.495503 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.495483 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.511104 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.511085 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xj52\" (UniqueName: \"kubernetes.io/projected/cdb29d62-6eb7-406a-b6fb-249a2bd04e4f-kube-api-access-4xj52\") pod \"openshift-state-metrics-54cf96dfcf-ptnpp\" (UID: \"cdb29d62-6eb7-406a-b6fb-249a2bd04e4f\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-ptnpp" Jul 07 16:19:47.512270 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.512254 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf7jr\" (UniqueName: \"kubernetes.io/projected/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-kube-api-access-xf7jr\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:47.583584 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.583500 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-ptnpp" Jul 07 16:19:47.726858 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.726771 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-ptnpp"] Jul 07 16:19:47.729448 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:19:47.729419 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdb29d62_6eb7_406a_b6fb_249a2bd04e4f.slice/crio-43358795ee681fb79dccd103048c6361f60f979106e47eb71dde52f1e90bba43 WatchSource:0}: Error finding container 43358795ee681fb79dccd103048c6361f60f979106e47eb71dde52f1e90bba43: Status 404 returned error can't find the container with id 43358795ee681fb79dccd103048c6361f60f979106e47eb71dde52f1e90bba43 Jul 07 16:19:47.750667 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.750632 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-ptnpp" event={"ID":"cdb29d62-6eb7-406a-b6fb-249a2bd04e4f","Type":"ContainerStarted","Data":"43358795ee681fb79dccd103048c6361f60f979106e47eb71dde52f1e90bba43"} Jul 07 16:19:47.752033 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.752009 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-t2wmq" event={"ID":"68959a9c-22fd-4804-9b45-0eb23f396983","Type":"ContainerStarted","Data":"40c40b69290a1b922789187c27913168b9ab027b02a70e3f0e450297cf93e6d4"} Jul 07 16:19:47.997522 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:47.997480 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-node-exporter-tls\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:48.000288 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:48.000263 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3b83dad7-acd2-43fe-8b45-b549bb50b2c3-node-exporter-tls\") pod \"node-exporter-d89dt\" (UID: \"3b83dad7-acd2-43fe-8b45-b549bb50b2c3\") " pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:48.178689 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:48.178649 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-d89dt" Jul 07 16:19:48.191192 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:19:48.191157 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b83dad7_acd2_43fe_8b45_b549bb50b2c3.slice/crio-1dd180e127946a49ae3a235fd6036c4a2581120acdf05bb9afe07178f10fa31c WatchSource:0}: Error finding container 1dd180e127946a49ae3a235fd6036c4a2581120acdf05bb9afe07178f10fa31c: Status 404 returned error can't find the container with id 1dd180e127946a49ae3a235fd6036c4a2581120acdf05bb9afe07178f10fa31c Jul 07 16:19:48.758366 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:48.758285 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d89dt" event={"ID":"3b83dad7-acd2-43fe-8b45-b549bb50b2c3","Type":"ContainerStarted","Data":"1dd180e127946a49ae3a235fd6036c4a2581120acdf05bb9afe07178f10fa31c"} Jul 07 16:19:48.760715 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:48.760683 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-ptnpp" event={"ID":"cdb29d62-6eb7-406a-b6fb-249a2bd04e4f","Type":"ContainerStarted","Data":"334cbf737922d8a448e3b5ca189777477b55b18e9662752c27b23c0172342060"} Jul 07 16:19:48.760838 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:48.760747 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-ptnpp" event={"ID":"cdb29d62-6eb7-406a-b6fb-249a2bd04e4f","Type":"ContainerStarted","Data":"827825b77442eb7708b31bac70c91cc5e2b199b337b10d9fe78703a3618ad276"} Jul 07 16:19:49.238932 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.238880 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6"] Jul 07 16:19:49.243099 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.243043 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.247688 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.246409 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 07 16:19:49.247688 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.246625 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 07 16:19:49.247688 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.246811 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-pkswl\"" Jul 07 16:19:49.247688 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.247032 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 07 16:19:49.247688 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.247251 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 07 16:19:49.247688 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.247508 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 07 16:19:49.247688 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.247532 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-sh9h6hha4nna\"" Jul 07 16:19:49.259696 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.259669 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6"] Jul 07 16:19:49.310051 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.310016 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ba804cd6-8e99-4485-a928-a8cd6e0ff350-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.310051 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.310054 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ba804cd6-8e99-4485-a928-a8cd6e0ff350-secret-grpc-tls\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.310321 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.310079 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbjmm\" (UniqueName: \"kubernetes.io/projected/ba804cd6-8e99-4485-a928-a8cd6e0ff350-kube-api-access-mbjmm\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.310321 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.310149 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ba804cd6-8e99-4485-a928-a8cd6e0ff350-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.310321 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.310195 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ba804cd6-8e99-4485-a928-a8cd6e0ff350-secret-thanos-querier-tls\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.310321 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.310247 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ba804cd6-8e99-4485-a928-a8cd6e0ff350-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.310321 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.310270 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba804cd6-8e99-4485-a928-a8cd6e0ff350-metrics-client-ca\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.310672 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.310346 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ba804cd6-8e99-4485-a928-a8cd6e0ff350-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.411017 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.410977 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ba804cd6-8e99-4485-a928-a8cd6e0ff350-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.411172 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.411053 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ba804cd6-8e99-4485-a928-a8cd6e0ff350-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.411172 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.411096 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ba804cd6-8e99-4485-a928-a8cd6e0ff350-secret-grpc-tls\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.411172 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.411154 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbjmm\" (UniqueName: \"kubernetes.io/projected/ba804cd6-8e99-4485-a928-a8cd6e0ff350-kube-api-access-mbjmm\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.411420 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.411263 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ba804cd6-8e99-4485-a928-a8cd6e0ff350-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.411420 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.411337 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ba804cd6-8e99-4485-a928-a8cd6e0ff350-secret-thanos-querier-tls\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.411420 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.411404 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ba804cd6-8e99-4485-a928-a8cd6e0ff350-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.411582 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.411432 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba804cd6-8e99-4485-a928-a8cd6e0ff350-metrics-client-ca\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.413262 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.413042 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba804cd6-8e99-4485-a928-a8cd6e0ff350-metrics-client-ca\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.414475 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.414445 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ba804cd6-8e99-4485-a928-a8cd6e0ff350-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.414722 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.414695 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ba804cd6-8e99-4485-a928-a8cd6e0ff350-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.414808 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.414793 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ba804cd6-8e99-4485-a928-a8cd6e0ff350-secret-grpc-tls\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.415397 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.415373 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ba804cd6-8e99-4485-a928-a8cd6e0ff350-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.415558 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.415521 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ba804cd6-8e99-4485-a928-a8cd6e0ff350-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.416170 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.416147 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ba804cd6-8e99-4485-a928-a8cd6e0ff350-secret-thanos-querier-tls\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.435680 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.435639 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbjmm\" (UniqueName: \"kubernetes.io/projected/ba804cd6-8e99-4485-a928-a8cd6e0ff350-kube-api-access-mbjmm\") pod \"thanos-querier-6c4c696cc7-kk4k6\" (UID: \"ba804cd6-8e99-4485-a928-a8cd6e0ff350\") " pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.558676 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.558638 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:49.716015 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.715976 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6"] Jul 07 16:19:49.719630 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:19:49.719602 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba804cd6_8e99_4485_a928_a8cd6e0ff350.slice/crio-2bcbc584b8b84113b19892349765292d025df90ae273261a8db28b3c17d38185 WatchSource:0}: Error finding container 2bcbc584b8b84113b19892349765292d025df90ae273261a8db28b3c17d38185: Status 404 returned error can't find the container with id 2bcbc584b8b84113b19892349765292d025df90ae273261a8db28b3c17d38185 Jul 07 16:19:49.765762 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.765726 2564 generic.go:356] "Generic (PLEG): container finished" podID="3b83dad7-acd2-43fe-8b45-b549bb50b2c3" containerID="5ad641ba304aa55969a1006994a8ca3da95adeb741d0de1723ec7729e698457d" exitCode=0 Jul 07 16:19:49.765969 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.765796 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d89dt" event={"ID":"3b83dad7-acd2-43fe-8b45-b549bb50b2c3","Type":"ContainerDied","Data":"5ad641ba304aa55969a1006994a8ca3da95adeb741d0de1723ec7729e698457d"} Jul 07 16:19:49.768102 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.768073 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-ptnpp" event={"ID":"cdb29d62-6eb7-406a-b6fb-249a2bd04e4f","Type":"ContainerStarted","Data":"3e7a7ed175791b3bedba61f6acd793bdc4b2180ed2bcd7ce9b52a54ecc7e5def"} Jul 07 16:19:49.769531 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.769500 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" event={"ID":"ba804cd6-8e99-4485-a928-a8cd6e0ff350","Type":"ContainerStarted","Data":"2bcbc584b8b84113b19892349765292d025df90ae273261a8db28b3c17d38185"} Jul 07 16:19:49.809635 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:49.809571 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-ptnpp" podStartSLOduration=1.59810075 podStartE2EDuration="2.809545862s" totalImagesPullingTime="1.211445112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:19:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:19:49.807691941 +0000 UTC m=+97.969504896" watchObservedRunningTime="2026-07-07 16:19:49.809545862 +0000 UTC m=+97.971358820" Jul 07 16:19:50.777102 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:50.777058 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d89dt" event={"ID":"3b83dad7-acd2-43fe-8b45-b549bb50b2c3","Type":"ContainerStarted","Data":"4c60d514d38361c1d9ca5437918336cca737aab62039f5437c9b21b3f0125836"} Jul 07 16:19:50.777543 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:50.777171 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-d89dt" event={"ID":"3b83dad7-acd2-43fe-8b45-b549bb50b2c3","Type":"ContainerStarted","Data":"58bfa20025f14a7e01b4ff6dcdbb19b63eb99d8c708aba1a09baaa5afb863cad"} Jul 07 16:19:51.891037 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:51.890977 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-d89dt" podStartSLOduration=3.630040706 podStartE2EDuration="4.890951176s" totalImagesPullingTime="1.194579904s" totalInitContainerRuntime="66.330566ms" isStatefulPod=true podCreationTimestamp="2026-07-07 16:19:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:19:50.804469174 +0000 UTC m=+98.966282129" watchObservedRunningTime="2026-07-07 16:19:51.890951176 +0000 UTC m=+100.052764175" Jul 07 16:19:51.892391 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:51.892368 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-cwp65"] Jul 07 16:19:51.896791 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:51.896769 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cwp65" Jul 07 16:19:51.903516 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:51.903452 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 07 16:19:51.903516 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:51.903459 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-g7gtm\"" Jul 07 16:19:51.909751 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:51.909730 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-cwp65"] Jul 07 16:19:52.038132 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.038049 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8kfk\" (UniqueName: \"kubernetes.io/projected/68f7ee89-78c7-43d1-af0d-d40573c762dd-kube-api-access-l8kfk\") pod \"monitoring-plugin-d5dcff7cf-cwp65\" (UID: \"68f7ee89-78c7-43d1-af0d-d40573c762dd\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cwp65" Jul 07 16:19:52.038132 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.038099 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/68f7ee89-78c7-43d1-af0d-d40573c762dd-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-cwp65\" (UID: \"68f7ee89-78c7-43d1-af0d-d40573c762dd\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cwp65" Jul 07 16:19:52.139341 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.139305 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8kfk\" (UniqueName: \"kubernetes.io/projected/68f7ee89-78c7-43d1-af0d-d40573c762dd-kube-api-access-l8kfk\") pod \"monitoring-plugin-d5dcff7cf-cwp65\" (UID: \"68f7ee89-78c7-43d1-af0d-d40573c762dd\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cwp65" Jul 07 16:19:52.139533 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.139363 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/68f7ee89-78c7-43d1-af0d-d40573c762dd-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-cwp65\" (UID: \"68f7ee89-78c7-43d1-af0d-d40573c762dd\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cwp65" Jul 07 16:19:52.139645 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:52.139550 2564 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 07 16:19:52.139645 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:19:52.139618 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68f7ee89-78c7-43d1-af0d-d40573c762dd-monitoring-plugin-cert podName:68f7ee89-78c7-43d1-af0d-d40573c762dd nodeName:}" failed. No retries permitted until 2026-07-07 16:19:52.639599673 +0000 UTC m=+100.801412608 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/68f7ee89-78c7-43d1-af0d-d40573c762dd-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-cwp65" (UID: "68f7ee89-78c7-43d1-af0d-d40573c762dd") : secret "monitoring-plugin-cert" not found Jul 07 16:19:52.151141 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.151110 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8kfk\" (UniqueName: \"kubernetes.io/projected/68f7ee89-78c7-43d1-af0d-d40573c762dd-kube-api-access-l8kfk\") pod \"monitoring-plugin-d5dcff7cf-cwp65\" (UID: \"68f7ee89-78c7-43d1-af0d-d40573c762dd\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cwp65" Jul 07 16:19:52.200123 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.200072 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57797d4c6d-8zrf4"] Jul 07 16:19:52.206010 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.205987 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:19:52.209502 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.209477 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 16:19:52.209639 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.209590 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 16:19:52.209639 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.209595 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 16:19:52.209755 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.209664 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 16:19:52.209944 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.209928 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-tvnhw\"" Jul 07 16:19:52.211469 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.211454 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 16:19:52.218835 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.218811 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57797d4c6d-8zrf4"] Jul 07 16:19:52.341360 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.341271 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-console-config\") pod \"console-57797d4c6d-8zrf4\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:19:52.341360 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.341348 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-oauth-serving-cert\") pod \"console-57797d4c6d-8zrf4\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:19:52.341613 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.341437 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-console-serving-cert\") pod \"console-57797d4c6d-8zrf4\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:19:52.341613 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.341484 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-console-oauth-config\") pod \"console-57797d4c6d-8zrf4\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:19:52.341613 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.341553 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txrh7\" (UniqueName: \"kubernetes.io/projected/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-kube-api-access-txrh7\") pod \"console-57797d4c6d-8zrf4\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:19:52.341613 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.341589 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-service-ca\") pod \"console-57797d4c6d-8zrf4\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:19:52.442630 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.442587 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-console-config\") pod \"console-57797d4c6d-8zrf4\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:19:52.442842 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.442643 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-oauth-serving-cert\") pod \"console-57797d4c6d-8zrf4\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:19:52.442842 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.442679 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-console-serving-cert\") pod \"console-57797d4c6d-8zrf4\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:19:52.442842 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.442707 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-console-oauth-config\") pod \"console-57797d4c6d-8zrf4\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:19:52.442842 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.442738 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txrh7\" (UniqueName: \"kubernetes.io/projected/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-kube-api-access-txrh7\") pod \"console-57797d4c6d-8zrf4\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:19:52.442842 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.442786 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-service-ca\") pod \"console-57797d4c6d-8zrf4\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:19:52.443532 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.443470 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-console-config\") pod \"console-57797d4c6d-8zrf4\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:19:52.443648 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.443576 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-oauth-serving-cert\") pod \"console-57797d4c6d-8zrf4\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:19:52.443648 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.443600 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-service-ca\") pod \"console-57797d4c6d-8zrf4\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:19:52.445230 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.445206 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-console-oauth-config\") pod \"console-57797d4c6d-8zrf4\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:19:52.445306 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.445264 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-console-serving-cert\") pod \"console-57797d4c6d-8zrf4\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:19:52.452506 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.452483 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-txrh7\" (UniqueName: \"kubernetes.io/projected/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-kube-api-access-txrh7\") pod \"console-57797d4c6d-8zrf4\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:19:52.515476 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.515433 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:19:52.647612 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.647140 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/68f7ee89-78c7-43d1-af0d-d40573c762dd-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-cwp65\" (UID: \"68f7ee89-78c7-43d1-af0d-d40573c762dd\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cwp65" Jul 07 16:19:52.650835 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.650776 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/68f7ee89-78c7-43d1-af0d-d40573c762dd-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-cwp65\" (UID: \"68f7ee89-78c7-43d1-af0d-d40573c762dd\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cwp65" Jul 07 16:19:52.704469 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.704435 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57797d4c6d-8zrf4"] Jul 07 16:19:52.707526 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:19:52.707490 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e8c5f49_e296_49e0_bf7b_af31b4d95bdc.slice/crio-c2aee58740d60c7f77e95415a7f61e05f06921727d3962d253f73441f294efb6 WatchSource:0}: Error finding container c2aee58740d60c7f77e95415a7f61e05f06921727d3962d253f73441f294efb6: Status 404 returned error can't find the container with id c2aee58740d60c7f77e95415a7f61e05f06921727d3962d253f73441f294efb6 Jul 07 16:19:52.787994 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.787951 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57797d4c6d-8zrf4" event={"ID":"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc","Type":"ContainerStarted","Data":"c2aee58740d60c7f77e95415a7f61e05f06921727d3962d253f73441f294efb6"} Jul 07 16:19:52.790331 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.790293 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" event={"ID":"ba804cd6-8e99-4485-a928-a8cd6e0ff350","Type":"ContainerStarted","Data":"3a862a52ed77898d6b20b67502fd827e536ccee70ac79a78563eae82ffabf583"} Jul 07 16:19:52.790478 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.790370 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" event={"ID":"ba804cd6-8e99-4485-a928-a8cd6e0ff350","Type":"ContainerStarted","Data":"126b47fb5efc7b1bcf887eacfa49538f0e3f4dcc3468b05e7fd5f27aef9325a1"} Jul 07 16:19:52.790478 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.790412 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" event={"ID":"ba804cd6-8e99-4485-a928-a8cd6e0ff350","Type":"ContainerStarted","Data":"d81458c66f38260d27545b1872534892ad393de685ba6d71c926ef39702de659"} Jul 07 16:19:52.807518 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:52.807481 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cwp65" Jul 07 16:19:53.149807 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:53.149772 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-cwp65"] Jul 07 16:19:53.152525 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:19:53.152499 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68f7ee89_78c7_43d1_af0d_d40573c762dd.slice/crio-90878042bad53e9c250ace12a911b6339bb1015b14ed96a13832f2f46f49a9dc WatchSource:0}: Error finding container 90878042bad53e9c250ace12a911b6339bb1015b14ed96a13832f2f46f49a9dc: Status 404 returned error can't find the container with id 90878042bad53e9c250ace12a911b6339bb1015b14ed96a13832f2f46f49a9dc Jul 07 16:19:53.647707 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:53.647677 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-g9pff" Jul 07 16:19:53.800797 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:53.800745 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" event={"ID":"ba804cd6-8e99-4485-a928-a8cd6e0ff350","Type":"ContainerStarted","Data":"6209f8da278b5b1002f546403d7b695040c02f991d86e4817a7e35d511f640c5"} Jul 07 16:19:53.800994 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:53.800825 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" event={"ID":"ba804cd6-8e99-4485-a928-a8cd6e0ff350","Type":"ContainerStarted","Data":"5571e391a3160fbdd003b6809063a8a74de11f78dd8151bacca26785354d1e3f"} Jul 07 16:19:53.800994 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:53.800864 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" event={"ID":"ba804cd6-8e99-4485-a928-a8cd6e0ff350","Type":"ContainerStarted","Data":"7df5f89a4fea44f32991344972543ce7c1db97be3cc1283c9b88ced7981c0a7c"} Jul 07 16:19:53.801106 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:53.801070 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:19:53.802864 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:53.802834 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cwp65" event={"ID":"68f7ee89-78c7-43d1-af0d-d40573c762dd","Type":"ContainerStarted","Data":"90878042bad53e9c250ace12a911b6339bb1015b14ed96a13832f2f46f49a9dc"} Jul 07 16:19:53.825055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:53.824525 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" podStartSLOduration=1.41597842 podStartE2EDuration="4.824493735s" totalImagesPullingTime="3.408515315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:19:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:19:53.823491835 +0000 UTC m=+101.985304827" watchObservedRunningTime="2026-07-07 16:19:53.824493735 +0000 UTC m=+101.986306691" Jul 07 16:19:55.044666 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:55.044628 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8c6f746f8-bb4kj"] Jul 07 16:19:55.812923 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:55.812878 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cwp65" event={"ID":"68f7ee89-78c7-43d1-af0d-d40573c762dd","Type":"ContainerStarted","Data":"e0e95943d318722a5ea0e4468db4e62ec140af7dfc080e174aebb003c7b112a6"} Jul 07 16:19:55.813130 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:55.813110 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cwp65" Jul 07 16:19:55.818509 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:55.818483 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cwp65" Jul 07 16:19:55.832078 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:55.832026 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-cwp65" podStartSLOduration=3.132239719 podStartE2EDuration="4.832011006s" totalImagesPullingTime="1.699771287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:19:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:19:55.831177559 +0000 UTC m=+103.992990515" watchObservedRunningTime="2026-07-07 16:19:55.832011006 +0000 UTC m=+103.993823960" Jul 07 16:19:57.825590 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:57.825548 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57797d4c6d-8zrf4" event={"ID":"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc","Type":"ContainerStarted","Data":"2bd398de34433d80b480d95c8f2fb9001dff68ba825994e3a7d0d5be3a1b79ee"} Jul 07 16:19:57.844279 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:57.844213 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57797d4c6d-8zrf4" podStartSLOduration=1.364174491 podStartE2EDuration="5.844191469s" totalImagesPullingTime="4.480016978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:19:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:19:57.843491626 +0000 UTC m=+106.005304581" watchObservedRunningTime="2026-07-07 16:19:57.844191469 +0000 UTC m=+106.006004426" Jul 07 16:19:59.813290 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:19:59.813264 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6c4c696cc7-kk4k6" Jul 07 16:20:01.770724 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.770688 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85d44b498d-lcts4"] Jul 07 16:20:01.806751 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.806709 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85d44b498d-lcts4"] Jul 07 16:20:01.806953 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.806833 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:01.815198 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.815175 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 16:20:01.842661 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.842625 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-oauth-serving-cert\") pod \"console-85d44b498d-lcts4\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:01.842661 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.842659 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41357fa4-338c-4a94-9cb2-1e058de74578-console-serving-cert\") pod \"console-85d44b498d-lcts4\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:01.842868 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.842682 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-trusted-ca-bundle\") pod \"console-85d44b498d-lcts4\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:01.842868 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.842747 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-service-ca\") pod \"console-85d44b498d-lcts4\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:01.842868 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.842785 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41357fa4-338c-4a94-9cb2-1e058de74578-console-oauth-config\") pod \"console-85d44b498d-lcts4\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:01.842868 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.842835 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb2vg\" (UniqueName: \"kubernetes.io/projected/41357fa4-338c-4a94-9cb2-1e058de74578-kube-api-access-gb2vg\") pod \"console-85d44b498d-lcts4\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:01.843068 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.842877 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-console-config\") pod \"console-85d44b498d-lcts4\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:01.943449 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.943410 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb2vg\" (UniqueName: \"kubernetes.io/projected/41357fa4-338c-4a94-9cb2-1e058de74578-kube-api-access-gb2vg\") pod \"console-85d44b498d-lcts4\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:01.943449 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.943454 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-console-config\") pod \"console-85d44b498d-lcts4\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:01.943705 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.943496 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-oauth-serving-cert\") pod \"console-85d44b498d-lcts4\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:01.943705 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.943513 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41357fa4-338c-4a94-9cb2-1e058de74578-console-serving-cert\") pod \"console-85d44b498d-lcts4\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:01.943705 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.943533 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-trusted-ca-bundle\") pod \"console-85d44b498d-lcts4\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:01.943705 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.943561 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-service-ca\") pod \"console-85d44b498d-lcts4\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:01.943705 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.943599 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41357fa4-338c-4a94-9cb2-1e058de74578-console-oauth-config\") pod \"console-85d44b498d-lcts4\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:01.944430 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.944399 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-oauth-serving-cert\") pod \"console-85d44b498d-lcts4\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:01.944551 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.944415 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-console-config\") pod \"console-85d44b498d-lcts4\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:01.944551 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.944463 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-service-ca\") pod \"console-85d44b498d-lcts4\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:01.944733 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.944704 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-trusted-ca-bundle\") pod \"console-85d44b498d-lcts4\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:01.946201 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.946176 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41357fa4-338c-4a94-9cb2-1e058de74578-console-oauth-config\") pod \"console-85d44b498d-lcts4\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:01.946296 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.946183 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41357fa4-338c-4a94-9cb2-1e058de74578-console-serving-cert\") pod \"console-85d44b498d-lcts4\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:01.952421 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:01.952396 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb2vg\" (UniqueName: \"kubernetes.io/projected/41357fa4-338c-4a94-9cb2-1e058de74578-kube-api-access-gb2vg\") pod \"console-85d44b498d-lcts4\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:02.116877 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:02.116837 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:02.278612 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:02.278575 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85d44b498d-lcts4"] Jul 07 16:20:02.281394 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:20:02.281358 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41357fa4_338c_4a94_9cb2_1e058de74578.slice/crio-f1ceb80c0e8ffbb20984008bde1a2d1e8cf9ff2cc457d08e569d87bebef2c6d7 WatchSource:0}: Error finding container f1ceb80c0e8ffbb20984008bde1a2d1e8cf9ff2cc457d08e569d87bebef2c6d7: Status 404 returned error can't find the container with id f1ceb80c0e8ffbb20984008bde1a2d1e8cf9ff2cc457d08e569d87bebef2c6d7 Jul 07 16:20:02.516008 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:02.515976 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:20:02.516203 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:02.516052 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:20:02.520662 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:02.520638 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:20:02.848330 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:02.848225 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d44b498d-lcts4" event={"ID":"41357fa4-338c-4a94-9cb2-1e058de74578","Type":"ContainerStarted","Data":"7b96dea6916b004c212f5db78ad87dc651c887886cc751dba7bfa73052f6198c"} Jul 07 16:20:02.848330 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:02.848297 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d44b498d-lcts4" event={"ID":"41357fa4-338c-4a94-9cb2-1e058de74578","Type":"ContainerStarted","Data":"f1ceb80c0e8ffbb20984008bde1a2d1e8cf9ff2cc457d08e569d87bebef2c6d7"} Jul 07 16:20:02.852473 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:02.852447 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:20:02.868398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:02.868339 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85d44b498d-lcts4" podStartSLOduration=1.868314692 podStartE2EDuration="1.868314692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:20:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:20:02.865889072 +0000 UTC m=+111.027702031" watchObservedRunningTime="2026-07-07 16:20:02.868314692 +0000 UTC m=+111.030127648" Jul 07 16:20:10.888847 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:10.888801 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-t2wmq" event={"ID":"68959a9c-22fd-4804-9b45-0eb23f396983","Type":"ContainerStarted","Data":"5d0831a87f264f3f1b8abd6d2d8ec210d13f79787d5dca5bfa68326ac51edf32"} Jul 07 16:20:10.889347 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:10.889025 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-t2wmq" Jul 07 16:20:10.903536 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:10.903503 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-t2wmq" Jul 07 16:20:10.907963 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:10.907890 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-t2wmq" podStartSLOduration=1.9685023080000001 podStartE2EDuration="24.907871809s" totalImagesPullingTime="22.939369501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:19:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:20:10.906580286 +0000 UTC m=+119.068393241" watchObservedRunningTime="2026-07-07 16:20:10.907871809 +0000 UTC m=+119.069684763" Jul 07 16:20:12.117753 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:12.117707 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:12.118286 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:12.117782 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:12.123700 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:12.123673 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:12.903321 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:12.903291 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:20:12.952163 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:12.952129 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57797d4c6d-8zrf4"] Jul 07 16:20:20.066765 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.066707 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" podUID="4955c880-3f01-4f4e-b9e3-6025c974796f" containerName="registry" containerID="cri-o://98fe3bc5cd665209c1599dc6943ea5ee9e29c69a361dc58e9fffe28964be2d89" gracePeriod=30 Jul 07 16:20:20.353466 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.353393 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:20:20.426064 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.426033 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4955c880-3f01-4f4e-b9e3-6025c974796f-image-registry-private-configuration\") pod \"4955c880-3f01-4f4e-b9e3-6025c974796f\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " Jul 07 16:20:20.426261 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.426076 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-certificates\") pod \"4955c880-3f01-4f4e-b9e3-6025c974796f\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " Jul 07 16:20:20.426261 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.426114 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4955c880-3f01-4f4e-b9e3-6025c974796f-installation-pull-secrets\") pod \"4955c880-3f01-4f4e-b9e3-6025c974796f\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " Jul 07 16:20:20.426261 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.426146 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4955c880-3f01-4f4e-b9e3-6025c974796f-ca-trust-extracted\") pod \"4955c880-3f01-4f4e-b9e3-6025c974796f\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " Jul 07 16:20:20.426261 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.426224 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls\") pod \"4955c880-3f01-4f4e-b9e3-6025c974796f\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " Jul 07 16:20:20.426492 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.426275 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-bound-sa-token\") pod \"4955c880-3f01-4f4e-b9e3-6025c974796f\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " Jul 07 16:20:20.426492 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.426297 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dvnqj\" (UniqueName: \"kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-kube-api-access-dvnqj\") pod \"4955c880-3f01-4f4e-b9e3-6025c974796f\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " Jul 07 16:20:20.426492 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.426331 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4955c880-3f01-4f4e-b9e3-6025c974796f-trusted-ca\") pod \"4955c880-3f01-4f4e-b9e3-6025c974796f\" (UID: \"4955c880-3f01-4f4e-b9e3-6025c974796f\") " Jul 07 16:20:20.426757 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.426733 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4955c880-3f01-4f4e-b9e3-6025c974796f" (UID: "4955c880-3f01-4f4e-b9e3-6025c974796f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:20:20.427033 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.426998 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4955c880-3f01-4f4e-b9e3-6025c974796f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4955c880-3f01-4f4e-b9e3-6025c974796f" (UID: "4955c880-3f01-4f4e-b9e3-6025c974796f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:20:20.429060 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.429031 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4955c880-3f01-4f4e-b9e3-6025c974796f" (UID: "4955c880-3f01-4f4e-b9e3-6025c974796f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:20:20.429171 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.429112 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4955c880-3f01-4f4e-b9e3-6025c974796f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4955c880-3f01-4f4e-b9e3-6025c974796f" (UID: "4955c880-3f01-4f4e-b9e3-6025c974796f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:20:20.429234 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.429163 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-kube-api-access-dvnqj" (OuterVolumeSpecName: "kube-api-access-dvnqj") pod "4955c880-3f01-4f4e-b9e3-6025c974796f" (UID: "4955c880-3f01-4f4e-b9e3-6025c974796f"). InnerVolumeSpecName "kube-api-access-dvnqj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:20:20.429234 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.429211 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4955c880-3f01-4f4e-b9e3-6025c974796f" (UID: "4955c880-3f01-4f4e-b9e3-6025c974796f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:20:20.429234 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.429224 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4955c880-3f01-4f4e-b9e3-6025c974796f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4955c880-3f01-4f4e-b9e3-6025c974796f" (UID: "4955c880-3f01-4f4e-b9e3-6025c974796f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:20:20.435779 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.435749 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4955c880-3f01-4f4e-b9e3-6025c974796f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4955c880-3f01-4f4e-b9e3-6025c974796f" (UID: "4955c880-3f01-4f4e-b9e3-6025c974796f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:20:20.527771 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.527733 2564 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-tls\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:20:20.527771 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.527763 2564 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-bound-sa-token\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:20:20.527771 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.527772 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dvnqj\" (UniqueName: \"kubernetes.io/projected/4955c880-3f01-4f4e-b9e3-6025c974796f-kube-api-access-dvnqj\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:20:20.527771 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.527781 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4955c880-3f01-4f4e-b9e3-6025c974796f-trusted-ca\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:20:20.528049 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.527791 2564 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4955c880-3f01-4f4e-b9e3-6025c974796f-image-registry-private-configuration\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:20:20.528049 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.527800 2564 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4955c880-3f01-4f4e-b9e3-6025c974796f-registry-certificates\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:20:20.528049 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.527810 2564 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4955c880-3f01-4f4e-b9e3-6025c974796f-installation-pull-secrets\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:20:20.528049 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.527818 2564 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4955c880-3f01-4f4e-b9e3-6025c974796f-ca-trust-extracted\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:20:20.931676 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.931637 2564 generic.go:356] "Generic (PLEG): container finished" podID="4955c880-3f01-4f4e-b9e3-6025c974796f" containerID="98fe3bc5cd665209c1599dc6943ea5ee9e29c69a361dc58e9fffe28964be2d89" exitCode=0 Jul 07 16:20:20.931843 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.931685 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" event={"ID":"4955c880-3f01-4f4e-b9e3-6025c974796f","Type":"ContainerDied","Data":"98fe3bc5cd665209c1599dc6943ea5ee9e29c69a361dc58e9fffe28964be2d89"} Jul 07 16:20:20.931843 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.931690 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" Jul 07 16:20:20.931843 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.931734 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8c6f746f8-bb4kj" event={"ID":"4955c880-3f01-4f4e-b9e3-6025c974796f","Type":"ContainerDied","Data":"d3dde68266362799b5290624fb10d9efb1e72650445907069e04a097eac3ca42"} Jul 07 16:20:20.931843 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.931758 2564 scope.go:117] "RemoveContainer" containerID="98fe3bc5cd665209c1599dc6943ea5ee9e29c69a361dc58e9fffe28964be2d89" Jul 07 16:20:20.940848 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.940830 2564 scope.go:117] "RemoveContainer" containerID="98fe3bc5cd665209c1599dc6943ea5ee9e29c69a361dc58e9fffe28964be2d89" Jul 07 16:20:20.941152 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:20:20.941131 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98fe3bc5cd665209c1599dc6943ea5ee9e29c69a361dc58e9fffe28964be2d89\": container with ID starting with 98fe3bc5cd665209c1599dc6943ea5ee9e29c69a361dc58e9fffe28964be2d89 not found: ID does not exist" containerID="98fe3bc5cd665209c1599dc6943ea5ee9e29c69a361dc58e9fffe28964be2d89" Jul 07 16:20:20.941201 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.941161 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98fe3bc5cd665209c1599dc6943ea5ee9e29c69a361dc58e9fffe28964be2d89"} err="failed to get container status \"98fe3bc5cd665209c1599dc6943ea5ee9e29c69a361dc58e9fffe28964be2d89\": rpc error: code = NotFound desc = could not find container \"98fe3bc5cd665209c1599dc6943ea5ee9e29c69a361dc58e9fffe28964be2d89\": container with ID starting with 98fe3bc5cd665209c1599dc6943ea5ee9e29c69a361dc58e9fffe28964be2d89 not found: ID does not exist" Jul 07 16:20:20.953822 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.953797 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8c6f746f8-bb4kj"] Jul 07 16:20:20.959862 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:20.959841 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-8c6f746f8-bb4kj"] Jul 07 16:20:22.243692 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:22.243646 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs\") pod \"network-metrics-daemon-xsj8b\" (UID: \"97f47a3d-062a-4159-8ebd-96d0afd8372b\") " pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:20:22.245981 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:22.245952 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/97f47a3d-062a-4159-8ebd-96d0afd8372b-metrics-certs\") pod \"network-metrics-daemon-xsj8b\" (UID: \"97f47a3d-062a-4159-8ebd-96d0afd8372b\") " pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:20:22.267535 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:22.267501 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2nbnd\"" Jul 07 16:20:22.275606 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:22.275582 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xsj8b" Jul 07 16:20:22.350308 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:22.350271 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4955c880-3f01-4f4e-b9e3-6025c974796f" path="/var/lib/kubelet/pods/4955c880-3f01-4f4e-b9e3-6025c974796f/volumes" Jul 07 16:20:22.476287 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:22.476215 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xsj8b"] Jul 07 16:20:22.478828 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:20:22.478790 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97f47a3d_062a_4159_8ebd_96d0afd8372b.slice/crio-44abba638f0dc899d276ec9a7c09136f3a61b2be542ee84a8f5102e257f883a0 WatchSource:0}: Error finding container 44abba638f0dc899d276ec9a7c09136f3a61b2be542ee84a8f5102e257f883a0: Status 404 returned error can't find the container with id 44abba638f0dc899d276ec9a7c09136f3a61b2be542ee84a8f5102e257f883a0 Jul 07 16:20:22.941219 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:22.941183 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xsj8b" event={"ID":"97f47a3d-062a-4159-8ebd-96d0afd8372b","Type":"ContainerStarted","Data":"44abba638f0dc899d276ec9a7c09136f3a61b2be542ee84a8f5102e257f883a0"} Jul 07 16:20:24.951888 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:24.951798 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xsj8b" event={"ID":"97f47a3d-062a-4159-8ebd-96d0afd8372b","Type":"ContainerStarted","Data":"5292612251e54c2ddb33be6720b10284a835116001f29befd7e18ffa186a03d3"} Jul 07 16:20:24.951888 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:24.951851 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xsj8b" event={"ID":"97f47a3d-062a-4159-8ebd-96d0afd8372b","Type":"ContainerStarted","Data":"6aa569a67e4402ee5289ce29ee3740e86ebbb7db60995c803c647e204469cbbc"} Jul 07 16:20:24.970279 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:24.970227 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xsj8b" podStartSLOduration=131.01622867 podStartE2EDuration="2m12.970209986s" totalImagesPullingTime="1.953981316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:18:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:20:24.969554928 +0000 UTC m=+133.131367884" watchObservedRunningTime="2026-07-07 16:20:24.970209986 +0000 UTC m=+133.132022939" Jul 07 16:20:30.976463 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:30.976383 2564 generic.go:356] "Generic (PLEG): container finished" podID="c13e43af-02c1-486e-8ffd-e334bedac760" containerID="4dbf7933d3c0b87a08d1bf8eb4286c6309671daf9f708e9bb1d433ef3dfbfc52" exitCode=0 Jul 07 16:20:30.976879 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:30.976451 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xct9g" event={"ID":"c13e43af-02c1-486e-8ffd-e334bedac760","Type":"ContainerDied","Data":"4dbf7933d3c0b87a08d1bf8eb4286c6309671daf9f708e9bb1d433ef3dfbfc52"} Jul 07 16:20:30.976879 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:30.976760 2564 scope.go:117] "RemoveContainer" containerID="4dbf7933d3c0b87a08d1bf8eb4286c6309671daf9f708e9bb1d433ef3dfbfc52" Jul 07 16:20:31.981953 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:31.981899 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-xct9g" event={"ID":"c13e43af-02c1-486e-8ffd-e334bedac760","Type":"ContainerStarted","Data":"3f74c0491bd0a68a72277567b955bc737462d1476ed19aed30a814309a0084b0"} Jul 07 16:20:36.000333 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:36.000244 2564 generic.go:356] "Generic (PLEG): container finished" podID="40c9cc16-aae3-4bb5-b11a-7723051bb9c0" containerID="3e5b9d5d21e81917150203688c1db60c6443b8ea032f167b35704415d75e4cac" exitCode=0 Jul 07 16:20:36.000333 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:36.000317 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" event={"ID":"40c9cc16-aae3-4bb5-b11a-7723051bb9c0","Type":"ContainerDied","Data":"3e5b9d5d21e81917150203688c1db60c6443b8ea032f167b35704415d75e4cac"} Jul 07 16:20:36.000717 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:36.000669 2564 scope.go:117] "RemoveContainer" containerID="3e5b9d5d21e81917150203688c1db60c6443b8ea032f167b35704415d75e4cac" Jul 07 16:20:37.005557 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:37.005522 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-mmrgp" event={"ID":"40c9cc16-aae3-4bb5-b11a-7723051bb9c0","Type":"ContainerStarted","Data":"0e0e0e2e41d78c7009d63d64e2e2d4912605e3ac1876226cc98c6039f5ac0fab"} Jul 07 16:20:37.979263 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:37.979220 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-57797d4c6d-8zrf4" podUID="1e8c5f49-e296-49e0-bf7b-af31b4d95bdc" containerName="console" containerID="cri-o://2bd398de34433d80b480d95c8f2fb9001dff68ba825994e3a7d0d5be3a1b79ee" gracePeriod=15 Jul 07 16:20:38.260284 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:38.260244 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:20:38.381155 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:38.381122 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-console-serving-cert\") pod \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " Jul 07 16:20:38.381366 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:38.381180 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-console-oauth-config\") pod \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " Jul 07 16:20:38.381366 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:38.381217 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-service-ca\") pod \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " Jul 07 16:20:38.381366 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:38.381262 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-txrh7\" (UniqueName: \"kubernetes.io/projected/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-kube-api-access-txrh7\") pod \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " Jul 07 16:20:38.381366 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:38.381284 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-console-config\") pod \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " Jul 07 16:20:38.381366 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:38.381310 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-oauth-serving-cert\") pod \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\" (UID: \"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc\") " Jul 07 16:20:38.381762 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:38.381721 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-service-ca" (OuterVolumeSpecName: "service-ca") pod "1e8c5f49-e296-49e0-bf7b-af31b4d95bdc" (UID: "1e8c5f49-e296-49e0-bf7b-af31b4d95bdc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:20:38.381890 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:38.381733 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-console-config" (OuterVolumeSpecName: "console-config") pod "1e8c5f49-e296-49e0-bf7b-af31b4d95bdc" (UID: "1e8c5f49-e296-49e0-bf7b-af31b4d95bdc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:20:38.381890 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:38.381806 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1e8c5f49-e296-49e0-bf7b-af31b4d95bdc" (UID: "1e8c5f49-e296-49e0-bf7b-af31b4d95bdc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:20:38.383791 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:38.383761 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-kube-api-access-txrh7" (OuterVolumeSpecName: "kube-api-access-txrh7") pod "1e8c5f49-e296-49e0-bf7b-af31b4d95bdc" (UID: "1e8c5f49-e296-49e0-bf7b-af31b4d95bdc"). InnerVolumeSpecName "kube-api-access-txrh7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:20:38.383956 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:38.383847 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1e8c5f49-e296-49e0-bf7b-af31b4d95bdc" (UID: "1e8c5f49-e296-49e0-bf7b-af31b4d95bdc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:20:38.383956 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:38.383863 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1e8c5f49-e296-49e0-bf7b-af31b4d95bdc" (UID: "1e8c5f49-e296-49e0-bf7b-af31b4d95bdc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:20:38.482803 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:38.482708 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-console-serving-cert\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:20:38.482803 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:38.482739 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-console-oauth-config\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:20:38.482803 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:38.482749 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-service-ca\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:20:38.482803 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:38.482759 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-txrh7\" (UniqueName: \"kubernetes.io/projected/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-kube-api-access-txrh7\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:20:38.482803 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:38.482770 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-console-config\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:20:38.482803 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:38.482778 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc-oauth-serving-cert\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:20:39.015626 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:39.015590 2564 generic.go:356] "Generic (PLEG): container finished" podID="1e8c5f49-e296-49e0-bf7b-af31b4d95bdc" containerID="2bd398de34433d80b480d95c8f2fb9001dff68ba825994e3a7d0d5be3a1b79ee" exitCode=0 Jul 07 16:20:39.015797 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:39.015638 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57797d4c6d-8zrf4" Jul 07 16:20:39.015797 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:39.015651 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57797d4c6d-8zrf4" event={"ID":"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc","Type":"ContainerDied","Data":"2bd398de34433d80b480d95c8f2fb9001dff68ba825994e3a7d0d5be3a1b79ee"} Jul 07 16:20:39.015797 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:39.015691 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57797d4c6d-8zrf4" event={"ID":"1e8c5f49-e296-49e0-bf7b-af31b4d95bdc","Type":"ContainerDied","Data":"c2aee58740d60c7f77e95415a7f61e05f06921727d3962d253f73441f294efb6"} Jul 07 16:20:39.015797 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:39.015708 2564 scope.go:117] "RemoveContainer" containerID="2bd398de34433d80b480d95c8f2fb9001dff68ba825994e3a7d0d5be3a1b79ee" Jul 07 16:20:39.027623 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:39.027597 2564 scope.go:117] "RemoveContainer" containerID="2bd398de34433d80b480d95c8f2fb9001dff68ba825994e3a7d0d5be3a1b79ee" Jul 07 16:20:39.027966 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:20:39.027943 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bd398de34433d80b480d95c8f2fb9001dff68ba825994e3a7d0d5be3a1b79ee\": container with ID starting with 2bd398de34433d80b480d95c8f2fb9001dff68ba825994e3a7d0d5be3a1b79ee not found: ID does not exist" containerID="2bd398de34433d80b480d95c8f2fb9001dff68ba825994e3a7d0d5be3a1b79ee" Jul 07 16:20:39.028050 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:39.027978 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bd398de34433d80b480d95c8f2fb9001dff68ba825994e3a7d0d5be3a1b79ee"} err="failed to get container status \"2bd398de34433d80b480d95c8f2fb9001dff68ba825994e3a7d0d5be3a1b79ee\": rpc error: code = NotFound desc = could not find container \"2bd398de34433d80b480d95c8f2fb9001dff68ba825994e3a7d0d5be3a1b79ee\": container with ID starting with 2bd398de34433d80b480d95c8f2fb9001dff68ba825994e3a7d0d5be3a1b79ee not found: ID does not exist" Jul 07 16:20:39.039111 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:39.039079 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57797d4c6d-8zrf4"] Jul 07 16:20:39.042316 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:39.042293 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57797d4c6d-8zrf4"] Jul 07 16:20:40.349298 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:20:40.349262 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e8c5f49-e296-49e0-bf7b-af31b4d95bdc" path="/var/lib/kubelet/pods/1e8c5f49-e296-49e0-bf7b-af31b4d95bdc/volumes" Jul 07 16:21:11.616082 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.616002 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-f65687bb8-pr5pw"] Jul 07 16:21:11.616537 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.616340 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4955c880-3f01-4f4e-b9e3-6025c974796f" containerName="registry" Jul 07 16:21:11.616537 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.616354 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4955c880-3f01-4f4e-b9e3-6025c974796f" containerName="registry" Jul 07 16:21:11.616537 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.616379 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e8c5f49-e296-49e0-bf7b-af31b4d95bdc" containerName="console" Jul 07 16:21:11.616537 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.616385 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e8c5f49-e296-49e0-bf7b-af31b4d95bdc" containerName="console" Jul 07 16:21:11.616537 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.616434 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4955c880-3f01-4f4e-b9e3-6025c974796f" containerName="registry" Jul 07 16:21:11.616537 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.616441 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e8c5f49-e296-49e0-bf7b-af31b4d95bdc" containerName="console" Jul 07 16:21:11.622789 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.622762 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.626133 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.626105 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 07 16:21:11.626250 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.626107 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 07 16:21:11.627468 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.627453 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 07 16:21:11.627716 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.627701 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 07 16:21:11.627772 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.627705 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-vf89z\"" Jul 07 16:21:11.628188 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.628172 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 07 16:21:11.635731 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.635702 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-f65687bb8-pr5pw"] Jul 07 16:21:11.635942 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.635925 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 07 16:21:11.753999 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.753965 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptlfb\" (UniqueName: \"kubernetes.io/projected/f3274426-8f98-4f7c-be9c-b3a57b88c414-kube-api-access-ptlfb\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.754194 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.754006 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f3274426-8f98-4f7c-be9c-b3a57b88c414-telemeter-client-tls\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.754194 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.754087 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f3274426-8f98-4f7c-be9c-b3a57b88c414-federate-client-tls\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.754194 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.754123 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3274426-8f98-4f7c-be9c-b3a57b88c414-telemeter-trusted-ca-bundle\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.754194 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.754147 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f3274426-8f98-4f7c-be9c-b3a57b88c414-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.754194 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.754169 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f3274426-8f98-4f7c-be9c-b3a57b88c414-secret-telemeter-client\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.754373 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.754256 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3274426-8f98-4f7c-be9c-b3a57b88c414-serving-certs-ca-bundle\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.754373 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.754286 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3274426-8f98-4f7c-be9c-b3a57b88c414-metrics-client-ca\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.855005 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.854971 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f3274426-8f98-4f7c-be9c-b3a57b88c414-federate-client-tls\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.855179 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.855015 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3274426-8f98-4f7c-be9c-b3a57b88c414-telemeter-trusted-ca-bundle\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.855179 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.855039 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f3274426-8f98-4f7c-be9c-b3a57b88c414-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.855179 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.855068 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f3274426-8f98-4f7c-be9c-b3a57b88c414-secret-telemeter-client\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.855179 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.855101 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3274426-8f98-4f7c-be9c-b3a57b88c414-serving-certs-ca-bundle\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.855179 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.855141 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3274426-8f98-4f7c-be9c-b3a57b88c414-metrics-client-ca\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.855440 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.855185 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptlfb\" (UniqueName: \"kubernetes.io/projected/f3274426-8f98-4f7c-be9c-b3a57b88c414-kube-api-access-ptlfb\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.855440 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.855208 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f3274426-8f98-4f7c-be9c-b3a57b88c414-telemeter-client-tls\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.856002 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.855977 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f3274426-8f98-4f7c-be9c-b3a57b88c414-metrics-client-ca\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.856194 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.855974 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3274426-8f98-4f7c-be9c-b3a57b88c414-serving-certs-ca-bundle\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.856330 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.856305 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f3274426-8f98-4f7c-be9c-b3a57b88c414-telemeter-trusted-ca-bundle\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.857613 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.857581 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f3274426-8f98-4f7c-be9c-b3a57b88c414-secret-telemeter-client\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.857756 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.857737 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f3274426-8f98-4f7c-be9c-b3a57b88c414-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.857812 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.857785 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f3274426-8f98-4f7c-be9c-b3a57b88c414-federate-client-tls\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.857849 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.857831 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f3274426-8f98-4f7c-be9c-b3a57b88c414-telemeter-client-tls\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.863749 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.863726 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptlfb\" (UniqueName: \"kubernetes.io/projected/f3274426-8f98-4f7c-be9c-b3a57b88c414-kube-api-access-ptlfb\") pod \"telemeter-client-f65687bb8-pr5pw\" (UID: \"f3274426-8f98-4f7c-be9c-b3a57b88c414\") " pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:11.932114 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:11.932029 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" Jul 07 16:21:12.076677 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:12.076653 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-f65687bb8-pr5pw"] Jul 07 16:21:12.078157 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:21:12.078128 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3274426_8f98_4f7c_be9c_b3a57b88c414.slice/crio-7f32d2d3e699fb3b6f0cbc8beffecc41545f4265b9cecfd8032c10fe3c96f372 WatchSource:0}: Error finding container 7f32d2d3e699fb3b6f0cbc8beffecc41545f4265b9cecfd8032c10fe3c96f372: Status 404 returned error can't find the container with id 7f32d2d3e699fb3b6f0cbc8beffecc41545f4265b9cecfd8032c10fe3c96f372 Jul 07 16:21:12.157436 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:12.157404 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" event={"ID":"f3274426-8f98-4f7c-be9c-b3a57b88c414","Type":"ContainerStarted","Data":"7f32d2d3e699fb3b6f0cbc8beffecc41545f4265b9cecfd8032c10fe3c96f372"} Jul 07 16:21:15.173334 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:15.173286 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" event={"ID":"f3274426-8f98-4f7c-be9c-b3a57b88c414","Type":"ContainerStarted","Data":"ce897714f3b39c07fbbf4904435fb4bf3bfb6fda2cfd10d231b2935c45d9294d"} Jul 07 16:21:16.179266 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:16.179228 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" event={"ID":"f3274426-8f98-4f7c-be9c-b3a57b88c414","Type":"ContainerStarted","Data":"293a41286d62c43dec558354cb73a587418e788a08f73b7f9c264f71070dc74c"} Jul 07 16:21:16.179768 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:16.179291 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" event={"ID":"f3274426-8f98-4f7c-be9c-b3a57b88c414","Type":"ContainerStarted","Data":"34bd0610d2dea191c3f24f680d818ca8ae47072afc8225a1c4a9a99378ea35da"} Jul 07 16:21:16.203971 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:16.203899 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-f65687bb8-pr5pw" podStartSLOduration=1.603230765 podStartE2EDuration="5.203879089s" totalImagesPullingTime="3.600648324s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:21:11 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:21:16.203058451 +0000 UTC m=+184.364871406" watchObservedRunningTime="2026-07-07 16:21:16.203879089 +0000 UTC m=+184.365692046" Jul 07 16:21:17.119833 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.119797 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d4857b964-bkzmj"] Jul 07 16:21:17.122092 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.122071 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:17.136245 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.136218 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d4857b964-bkzmj"] Jul 07 16:21:17.203110 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.203067 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-service-ca\") pod \"console-d4857b964-bkzmj\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:17.203110 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.203118 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-oauth-serving-cert\") pod \"console-d4857b964-bkzmj\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:17.203584 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.203186 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-trusted-ca-bundle\") pod \"console-d4857b964-bkzmj\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:17.203584 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.203217 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57ddj\" (UniqueName: \"kubernetes.io/projected/0db3adaa-65f9-4928-bc0c-6f6111b57b77-kube-api-access-57ddj\") pod \"console-d4857b964-bkzmj\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:17.203584 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.203248 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-console-config\") pod \"console-d4857b964-bkzmj\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:17.203584 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.203288 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0db3adaa-65f9-4928-bc0c-6f6111b57b77-console-oauth-config\") pod \"console-d4857b964-bkzmj\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:17.203584 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.203323 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0db3adaa-65f9-4928-bc0c-6f6111b57b77-console-serving-cert\") pod \"console-d4857b964-bkzmj\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:17.304372 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.304330 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0db3adaa-65f9-4928-bc0c-6f6111b57b77-console-oauth-config\") pod \"console-d4857b964-bkzmj\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:17.304574 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.304386 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0db3adaa-65f9-4928-bc0c-6f6111b57b77-console-serving-cert\") pod \"console-d4857b964-bkzmj\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:17.304653 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.304585 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-service-ca\") pod \"console-d4857b964-bkzmj\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:17.304712 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.304655 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-oauth-serving-cert\") pod \"console-d4857b964-bkzmj\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:17.304765 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.304749 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-trusted-ca-bundle\") pod \"console-d4857b964-bkzmj\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:17.304815 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.304780 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57ddj\" (UniqueName: \"kubernetes.io/projected/0db3adaa-65f9-4928-bc0c-6f6111b57b77-kube-api-access-57ddj\") pod \"console-d4857b964-bkzmj\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:17.304815 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.304810 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-console-config\") pod \"console-d4857b964-bkzmj\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:17.305380 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.305346 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-service-ca\") pod \"console-d4857b964-bkzmj\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:17.305526 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.305502 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-console-config\") pod \"console-d4857b964-bkzmj\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:17.305766 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.305746 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-oauth-serving-cert\") pod \"console-d4857b964-bkzmj\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:17.305837 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.305806 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-trusted-ca-bundle\") pod \"console-d4857b964-bkzmj\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:17.307000 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.306971 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0db3adaa-65f9-4928-bc0c-6f6111b57b77-console-serving-cert\") pod \"console-d4857b964-bkzmj\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:17.307000 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.306984 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0db3adaa-65f9-4928-bc0c-6f6111b57b77-console-oauth-config\") pod \"console-d4857b964-bkzmj\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:17.322705 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.322672 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57ddj\" (UniqueName: \"kubernetes.io/projected/0db3adaa-65f9-4928-bc0c-6f6111b57b77-kube-api-access-57ddj\") pod \"console-d4857b964-bkzmj\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:17.431631 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.431547 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:17.565743 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:17.565715 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d4857b964-bkzmj"] Jul 07 16:21:17.569541 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:21:17.569518 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0db3adaa_65f9_4928_bc0c_6f6111b57b77.slice/crio-91b7952d743e6979b8d05fdff879eb5d45c054c7266d476ff2baf481ee95c715 WatchSource:0}: Error finding container 91b7952d743e6979b8d05fdff879eb5d45c054c7266d476ff2baf481ee95c715: Status 404 returned error can't find the container with id 91b7952d743e6979b8d05fdff879eb5d45c054c7266d476ff2baf481ee95c715 Jul 07 16:21:18.188515 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:18.188470 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d4857b964-bkzmj" event={"ID":"0db3adaa-65f9-4928-bc0c-6f6111b57b77","Type":"ContainerStarted","Data":"8caa57cf54a74154e7b559d8754d6451031daa75d80fe688421545ced08c5a83"} Jul 07 16:21:18.188729 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:18.188528 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d4857b964-bkzmj" event={"ID":"0db3adaa-65f9-4928-bc0c-6f6111b57b77","Type":"ContainerStarted","Data":"91b7952d743e6979b8d05fdff879eb5d45c054c7266d476ff2baf481ee95c715"} Jul 07 16:21:18.209411 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:18.209356 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d4857b964-bkzmj" podStartSLOduration=1.209338327 podStartE2EDuration="1.209338327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:21:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:21:18.208539092 +0000 UTC m=+186.370352048" watchObservedRunningTime="2026-07-07 16:21:18.209338327 +0000 UTC m=+186.371151341" Jul 07 16:21:27.432771 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:27.432722 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:27.433207 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:27.432795 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:27.437506 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:27.437482 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:28.235398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:28.235368 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:21:28.285241 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:28.285206 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85d44b498d-lcts4"] Jul 07 16:21:53.311243 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:53.311184 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-85d44b498d-lcts4" podUID="41357fa4-338c-4a94-9cb2-1e058de74578" containerName="console" containerID="cri-o://7b96dea6916b004c212f5db78ad87dc651c887886cc751dba7bfa73052f6198c" gracePeriod=15 Jul 07 16:21:53.561004 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:53.560966 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:21:53.579219 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:53.579181 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-console-config\") pod \"41357fa4-338c-4a94-9cb2-1e058de74578\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " Jul 07 16:21:53.579442 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:53.579294 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gb2vg\" (UniqueName: \"kubernetes.io/projected/41357fa4-338c-4a94-9cb2-1e058de74578-kube-api-access-gb2vg\") pod \"41357fa4-338c-4a94-9cb2-1e058de74578\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " Jul 07 16:21:53.579442 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:53.579361 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41357fa4-338c-4a94-9cb2-1e058de74578-console-serving-cert\") pod \"41357fa4-338c-4a94-9cb2-1e058de74578\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " Jul 07 16:21:53.579442 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:53.579429 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-oauth-serving-cert\") pod \"41357fa4-338c-4a94-9cb2-1e058de74578\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " Jul 07 16:21:53.579662 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:53.579452 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41357fa4-338c-4a94-9cb2-1e058de74578-console-oauth-config\") pod \"41357fa4-338c-4a94-9cb2-1e058de74578\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " Jul 07 16:21:53.579662 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:53.579484 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-service-ca\") pod \"41357fa4-338c-4a94-9cb2-1e058de74578\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " Jul 07 16:21:53.579662 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:53.579521 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-trusted-ca-bundle\") pod \"41357fa4-338c-4a94-9cb2-1e058de74578\" (UID: \"41357fa4-338c-4a94-9cb2-1e058de74578\") " Jul 07 16:21:53.579662 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:53.579597 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-console-config" (OuterVolumeSpecName: "console-config") pod "41357fa4-338c-4a94-9cb2-1e058de74578" (UID: "41357fa4-338c-4a94-9cb2-1e058de74578"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:21:53.579851 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:53.579824 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-console-config\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:21:53.582843 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:53.582800 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-service-ca" (OuterVolumeSpecName: "service-ca") pod "41357fa4-338c-4a94-9cb2-1e058de74578" (UID: "41357fa4-338c-4a94-9cb2-1e058de74578"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:21:53.582996 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:53.582838 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "41357fa4-338c-4a94-9cb2-1e058de74578" (UID: "41357fa4-338c-4a94-9cb2-1e058de74578"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:21:53.583270 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:53.583248 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "41357fa4-338c-4a94-9cb2-1e058de74578" (UID: "41357fa4-338c-4a94-9cb2-1e058de74578"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:21:53.585126 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:53.585102 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41357fa4-338c-4a94-9cb2-1e058de74578-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "41357fa4-338c-4a94-9cb2-1e058de74578" (UID: "41357fa4-338c-4a94-9cb2-1e058de74578"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:21:53.585284 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:53.585266 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41357fa4-338c-4a94-9cb2-1e058de74578-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "41357fa4-338c-4a94-9cb2-1e058de74578" (UID: "41357fa4-338c-4a94-9cb2-1e058de74578"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:21:53.585409 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:53.585366 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41357fa4-338c-4a94-9cb2-1e058de74578-kube-api-access-gb2vg" (OuterVolumeSpecName: "kube-api-access-gb2vg") pod "41357fa4-338c-4a94-9cb2-1e058de74578" (UID: "41357fa4-338c-4a94-9cb2-1e058de74578"). InnerVolumeSpecName "kube-api-access-gb2vg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:21:53.680487 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:53.680446 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gb2vg\" (UniqueName: \"kubernetes.io/projected/41357fa4-338c-4a94-9cb2-1e058de74578-kube-api-access-gb2vg\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:21:53.680487 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:53.680480 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/41357fa4-338c-4a94-9cb2-1e058de74578-console-serving-cert\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:21:53.680487 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:53.680489 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-oauth-serving-cert\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:21:53.680487 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:53.680497 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/41357fa4-338c-4a94-9cb2-1e058de74578-console-oauth-config\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:21:53.680748 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:53.680507 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-service-ca\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:21:53.680748 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:53.680515 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41357fa4-338c-4a94-9cb2-1e058de74578-trusted-ca-bundle\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:21:54.335039 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:54.335009 2564 generic.go:356] "Generic (PLEG): container finished" podID="41357fa4-338c-4a94-9cb2-1e058de74578" containerID="7b96dea6916b004c212f5db78ad87dc651c887886cc751dba7bfa73052f6198c" exitCode=0 Jul 07 16:21:54.335441 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:54.335052 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d44b498d-lcts4" Jul 07 16:21:54.335441 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:54.335104 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d44b498d-lcts4" event={"ID":"41357fa4-338c-4a94-9cb2-1e058de74578","Type":"ContainerDied","Data":"7b96dea6916b004c212f5db78ad87dc651c887886cc751dba7bfa73052f6198c"} Jul 07 16:21:54.335441 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:54.335177 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d44b498d-lcts4" event={"ID":"41357fa4-338c-4a94-9cb2-1e058de74578","Type":"ContainerDied","Data":"f1ceb80c0e8ffbb20984008bde1a2d1e8cf9ff2cc457d08e569d87bebef2c6d7"} Jul 07 16:21:54.335441 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:54.335199 2564 scope.go:117] "RemoveContainer" containerID="7b96dea6916b004c212f5db78ad87dc651c887886cc751dba7bfa73052f6198c" Jul 07 16:21:54.343882 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:54.343863 2564 scope.go:117] "RemoveContainer" containerID="7b96dea6916b004c212f5db78ad87dc651c887886cc751dba7bfa73052f6198c" Jul 07 16:21:54.344184 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:21:54.344164 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b96dea6916b004c212f5db78ad87dc651c887886cc751dba7bfa73052f6198c\": container with ID starting with 7b96dea6916b004c212f5db78ad87dc651c887886cc751dba7bfa73052f6198c not found: ID does not exist" containerID="7b96dea6916b004c212f5db78ad87dc651c887886cc751dba7bfa73052f6198c" Jul 07 16:21:54.344245 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:54.344194 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b96dea6916b004c212f5db78ad87dc651c887886cc751dba7bfa73052f6198c"} err="failed to get container status \"7b96dea6916b004c212f5db78ad87dc651c887886cc751dba7bfa73052f6198c\": rpc error: code = NotFound desc = could not find container \"7b96dea6916b004c212f5db78ad87dc651c887886cc751dba7bfa73052f6198c\": container with ID starting with 7b96dea6916b004c212f5db78ad87dc651c887886cc751dba7bfa73052f6198c not found: ID does not exist" Jul 07 16:21:54.358920 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:54.358871 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85d44b498d-lcts4"] Jul 07 16:21:54.363631 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:54.363607 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85d44b498d-lcts4"] Jul 07 16:21:56.348046 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:21:56.348016 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41357fa4-338c-4a94-9cb2-1e058de74578" path="/var/lib/kubelet/pods/41357fa4-338c-4a94-9cb2-1e058de74578/volumes" Jul 07 16:22:23.084968 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.084932 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5989d5bcc8-cw2hh"] Jul 07 16:22:23.085339 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.085233 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41357fa4-338c-4a94-9cb2-1e058de74578" containerName="console" Jul 07 16:22:23.085339 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.085244 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="41357fa4-338c-4a94-9cb2-1e058de74578" containerName="console" Jul 07 16:22:23.085339 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.085296 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="41357fa4-338c-4a94-9cb2-1e058de74578" containerName="console" Jul 07 16:22:23.089494 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.089473 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:23.103598 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.103569 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5989d5bcc8-cw2hh"] Jul 07 16:22:23.226252 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.226215 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f57359a5-a9b6-4704-bd36-b7efa1f573b4-console-serving-cert\") pod \"console-5989d5bcc8-cw2hh\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:23.226252 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.226260 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-oauth-serving-cert\") pod \"console-5989d5bcc8-cw2hh\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:23.226522 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.226280 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-trusted-ca-bundle\") pod \"console-5989d5bcc8-cw2hh\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:23.226522 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.226389 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f57359a5-a9b6-4704-bd36-b7efa1f573b4-console-oauth-config\") pod \"console-5989d5bcc8-cw2hh\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:23.226522 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.226429 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-service-ca\") pod \"console-5989d5bcc8-cw2hh\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:23.226522 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.226488 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-console-config\") pod \"console-5989d5bcc8-cw2hh\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:23.226522 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.226506 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b7x6\" (UniqueName: \"kubernetes.io/projected/f57359a5-a9b6-4704-bd36-b7efa1f573b4-kube-api-access-6b7x6\") pod \"console-5989d5bcc8-cw2hh\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:23.327206 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.327164 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f57359a5-a9b6-4704-bd36-b7efa1f573b4-console-oauth-config\") pod \"console-5989d5bcc8-cw2hh\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:23.327206 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.327204 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-service-ca\") pod \"console-5989d5bcc8-cw2hh\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:23.327459 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.327244 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-console-config\") pod \"console-5989d5bcc8-cw2hh\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:23.327459 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.327272 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b7x6\" (UniqueName: \"kubernetes.io/projected/f57359a5-a9b6-4704-bd36-b7efa1f573b4-kube-api-access-6b7x6\") pod \"console-5989d5bcc8-cw2hh\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:23.327459 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.327320 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f57359a5-a9b6-4704-bd36-b7efa1f573b4-console-serving-cert\") pod \"console-5989d5bcc8-cw2hh\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:23.327459 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.327363 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-oauth-serving-cert\") pod \"console-5989d5bcc8-cw2hh\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:23.327459 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.327385 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-trusted-ca-bundle\") pod \"console-5989d5bcc8-cw2hh\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:23.328483 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.328450 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-service-ca\") pod \"console-5989d5bcc8-cw2hh\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:23.328654 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.328480 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-oauth-serving-cert\") pod \"console-5989d5bcc8-cw2hh\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:23.328759 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.328481 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-console-config\") pod \"console-5989d5bcc8-cw2hh\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:23.330607 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.330572 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f57359a5-a9b6-4704-bd36-b7efa1f573b4-console-oauth-config\") pod \"console-5989d5bcc8-cw2hh\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:23.332156 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.332129 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-trusted-ca-bundle\") pod \"console-5989d5bcc8-cw2hh\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:23.334581 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.334556 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f57359a5-a9b6-4704-bd36-b7efa1f573b4-console-serving-cert\") pod \"console-5989d5bcc8-cw2hh\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:23.337994 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.337900 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b7x6\" (UniqueName: \"kubernetes.io/projected/f57359a5-a9b6-4704-bd36-b7efa1f573b4-kube-api-access-6b7x6\") pod \"console-5989d5bcc8-cw2hh\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:23.398767 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.398713 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:23.537186 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:23.537156 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5989d5bcc8-cw2hh"] Jul 07 16:22:23.540231 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:22:23.540200 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf57359a5_a9b6_4704_bd36_b7efa1f573b4.slice/crio-828d8b05524e054b2388f38da62fd391a327a9bc481344912a7f9aaa33bff4d3 WatchSource:0}: Error finding container 828d8b05524e054b2388f38da62fd391a327a9bc481344912a7f9aaa33bff4d3: Status 404 returned error can't find the container with id 828d8b05524e054b2388f38da62fd391a327a9bc481344912a7f9aaa33bff4d3 Jul 07 16:22:24.453165 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:24.453127 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5989d5bcc8-cw2hh" event={"ID":"f57359a5-a9b6-4704-bd36-b7efa1f573b4","Type":"ContainerStarted","Data":"91f8a071f9debadb05785f47a82cdbce2636a0c1c5770f38d07459520909ba87"} Jul 07 16:22:24.453691 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:24.453178 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5989d5bcc8-cw2hh" event={"ID":"f57359a5-a9b6-4704-bd36-b7efa1f573b4","Type":"ContainerStarted","Data":"828d8b05524e054b2388f38da62fd391a327a9bc481344912a7f9aaa33bff4d3"} Jul 07 16:22:24.473183 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:24.473125 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5989d5bcc8-cw2hh" podStartSLOduration=1.473109652 podStartE2EDuration="1.473109652s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:22:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:22:24.472042891 +0000 UTC m=+252.633855847" watchObservedRunningTime="2026-07-07 16:22:24.473109652 +0000 UTC m=+252.634922608" Jul 07 16:22:33.398894 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:33.398852 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:33.399365 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:33.398923 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:33.403814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:33.403792 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:33.494015 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:33.493990 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:22:33.543986 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:33.543941 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d4857b964-bkzmj"] Jul 07 16:22:58.564692 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:58.564631 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-d4857b964-bkzmj" podUID="0db3adaa-65f9-4928-bc0c-6f6111b57b77" containerName="console" containerID="cri-o://8caa57cf54a74154e7b559d8754d6451031daa75d80fe688421545ced08c5a83" gracePeriod=15 Jul 07 16:22:58.817726 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:58.817656 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:22:58.916524 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:58.916486 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-console-config\") pod \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " Jul 07 16:22:58.916524 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:58.916530 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57ddj\" (UniqueName: \"kubernetes.io/projected/0db3adaa-65f9-4928-bc0c-6f6111b57b77-kube-api-access-57ddj\") pod \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " Jul 07 16:22:58.916786 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:58.916564 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-service-ca\") pod \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " Jul 07 16:22:58.916786 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:58.916595 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-trusted-ca-bundle\") pod \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " Jul 07 16:22:58.916786 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:58.916612 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0db3adaa-65f9-4928-bc0c-6f6111b57b77-console-serving-cert\") pod \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " Jul 07 16:22:58.916786 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:58.916635 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0db3adaa-65f9-4928-bc0c-6f6111b57b77-console-oauth-config\") pod \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " Jul 07 16:22:58.916786 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:58.916657 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-oauth-serving-cert\") pod \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\" (UID: \"0db3adaa-65f9-4928-bc0c-6f6111b57b77\") " Jul 07 16:22:58.917149 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:58.917115 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-console-config" (OuterVolumeSpecName: "console-config") pod "0db3adaa-65f9-4928-bc0c-6f6111b57b77" (UID: "0db3adaa-65f9-4928-bc0c-6f6111b57b77"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:22:58.917149 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:58.917126 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-service-ca" (OuterVolumeSpecName: "service-ca") pod "0db3adaa-65f9-4928-bc0c-6f6111b57b77" (UID: "0db3adaa-65f9-4928-bc0c-6f6111b57b77"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:22:58.917342 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:58.917239 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0db3adaa-65f9-4928-bc0c-6f6111b57b77" (UID: "0db3adaa-65f9-4928-bc0c-6f6111b57b77"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:22:58.917342 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:58.917313 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0db3adaa-65f9-4928-bc0c-6f6111b57b77" (UID: "0db3adaa-65f9-4928-bc0c-6f6111b57b77"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:22:58.918962 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:58.918933 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0db3adaa-65f9-4928-bc0c-6f6111b57b77-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0db3adaa-65f9-4928-bc0c-6f6111b57b77" (UID: "0db3adaa-65f9-4928-bc0c-6f6111b57b77"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:22:58.919048 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:58.918955 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0db3adaa-65f9-4928-bc0c-6f6111b57b77-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0db3adaa-65f9-4928-bc0c-6f6111b57b77" (UID: "0db3adaa-65f9-4928-bc0c-6f6111b57b77"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:22:58.919048 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:58.918967 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0db3adaa-65f9-4928-bc0c-6f6111b57b77-kube-api-access-57ddj" (OuterVolumeSpecName: "kube-api-access-57ddj") pod "0db3adaa-65f9-4928-bc0c-6f6111b57b77" (UID: "0db3adaa-65f9-4928-bc0c-6f6111b57b77"). InnerVolumeSpecName "kube-api-access-57ddj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:22:59.017567 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:59.017508 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57ddj\" (UniqueName: \"kubernetes.io/projected/0db3adaa-65f9-4928-bc0c-6f6111b57b77-kube-api-access-57ddj\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:22:59.017567 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:59.017562 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-service-ca\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:22:59.017567 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:59.017572 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-trusted-ca-bundle\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:22:59.017567 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:59.017581 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0db3adaa-65f9-4928-bc0c-6f6111b57b77-console-serving-cert\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:22:59.017843 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:59.017590 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0db3adaa-65f9-4928-bc0c-6f6111b57b77-console-oauth-config\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:22:59.017843 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:59.017598 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-oauth-serving-cert\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:22:59.017843 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:59.017607 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0db3adaa-65f9-4928-bc0c-6f6111b57b77-console-config\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:22:59.588109 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:59.588067 2564 generic.go:356] "Generic (PLEG): container finished" podID="0db3adaa-65f9-4928-bc0c-6f6111b57b77" containerID="8caa57cf54a74154e7b559d8754d6451031daa75d80fe688421545ced08c5a83" exitCode=0 Jul 07 16:22:59.588546 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:59.588132 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d4857b964-bkzmj" event={"ID":"0db3adaa-65f9-4928-bc0c-6f6111b57b77","Type":"ContainerDied","Data":"8caa57cf54a74154e7b559d8754d6451031daa75d80fe688421545ced08c5a83"} Jul 07 16:22:59.588546 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:59.588178 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d4857b964-bkzmj" event={"ID":"0db3adaa-65f9-4928-bc0c-6f6111b57b77","Type":"ContainerDied","Data":"91b7952d743e6979b8d05fdff879eb5d45c054c7266d476ff2baf481ee95c715"} Jul 07 16:22:59.588546 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:59.588199 2564 scope.go:117] "RemoveContainer" containerID="8caa57cf54a74154e7b559d8754d6451031daa75d80fe688421545ced08c5a83" Jul 07 16:22:59.588546 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:59.588136 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d4857b964-bkzmj" Jul 07 16:22:59.596965 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:59.596946 2564 scope.go:117] "RemoveContainer" containerID="8caa57cf54a74154e7b559d8754d6451031daa75d80fe688421545ced08c5a83" Jul 07 16:22:59.597236 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:22:59.597218 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8caa57cf54a74154e7b559d8754d6451031daa75d80fe688421545ced08c5a83\": container with ID starting with 8caa57cf54a74154e7b559d8754d6451031daa75d80fe688421545ced08c5a83 not found: ID does not exist" containerID="8caa57cf54a74154e7b559d8754d6451031daa75d80fe688421545ced08c5a83" Jul 07 16:22:59.597285 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:59.597245 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8caa57cf54a74154e7b559d8754d6451031daa75d80fe688421545ced08c5a83"} err="failed to get container status \"8caa57cf54a74154e7b559d8754d6451031daa75d80fe688421545ced08c5a83\": rpc error: code = NotFound desc = could not find container \"8caa57cf54a74154e7b559d8754d6451031daa75d80fe688421545ced08c5a83\": container with ID starting with 8caa57cf54a74154e7b559d8754d6451031daa75d80fe688421545ced08c5a83 not found: ID does not exist" Jul 07 16:22:59.610321 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:59.610290 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d4857b964-bkzmj"] Jul 07 16:22:59.616174 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:22:59.616149 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d4857b964-bkzmj"] Jul 07 16:23:00.348820 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:00.348784 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0db3adaa-65f9-4928-bc0c-6f6111b57b77" path="/var/lib/kubelet/pods/0db3adaa-65f9-4928-bc0c-6f6111b57b77/volumes" Jul 07 16:23:12.249606 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:12.249578 2564 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 16:23:38.045809 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:38.045776 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn"] Jul 07 16:23:38.048160 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:38.046123 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0db3adaa-65f9-4928-bc0c-6f6111b57b77" containerName="console" Jul 07 16:23:38.048160 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:38.046137 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0db3adaa-65f9-4928-bc0c-6f6111b57b77" containerName="console" Jul 07 16:23:38.048160 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:38.046202 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0db3adaa-65f9-4928-bc0c-6f6111b57b77" containerName="console" Jul 07 16:23:38.049090 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:38.049075 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn" Jul 07 16:23:38.051639 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:38.051612 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-t7vz2\"" Jul 07 16:23:38.051926 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:38.051892 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 16:23:38.052711 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:38.052696 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 16:23:38.066776 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:38.066746 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn"] Jul 07 16:23:38.132454 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:38.132420 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5a3a2ee9-a66c-40b7-a2d1-5665a3574855-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn\" (UID: \"5a3a2ee9-a66c-40b7-a2d1-5665a3574855\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn" Jul 07 16:23:38.132635 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:38.132464 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnlhb\" (UniqueName: \"kubernetes.io/projected/5a3a2ee9-a66c-40b7-a2d1-5665a3574855-kube-api-access-xnlhb\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn\" (UID: \"5a3a2ee9-a66c-40b7-a2d1-5665a3574855\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn" Jul 07 16:23:38.132635 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:38.132509 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5a3a2ee9-a66c-40b7-a2d1-5665a3574855-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn\" (UID: \"5a3a2ee9-a66c-40b7-a2d1-5665a3574855\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn" Jul 07 16:23:38.233824 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:38.233778 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5a3a2ee9-a66c-40b7-a2d1-5665a3574855-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn\" (UID: \"5a3a2ee9-a66c-40b7-a2d1-5665a3574855\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn" Jul 07 16:23:38.233824 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:38.233825 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnlhb\" (UniqueName: \"kubernetes.io/projected/5a3a2ee9-a66c-40b7-a2d1-5665a3574855-kube-api-access-xnlhb\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn\" (UID: \"5a3a2ee9-a66c-40b7-a2d1-5665a3574855\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn" Jul 07 16:23:38.234116 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:38.233867 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5a3a2ee9-a66c-40b7-a2d1-5665a3574855-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn\" (UID: \"5a3a2ee9-a66c-40b7-a2d1-5665a3574855\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn" Jul 07 16:23:38.234213 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:38.234189 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5a3a2ee9-a66c-40b7-a2d1-5665a3574855-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn\" (UID: \"5a3a2ee9-a66c-40b7-a2d1-5665a3574855\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn" Jul 07 16:23:38.234275 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:38.234245 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5a3a2ee9-a66c-40b7-a2d1-5665a3574855-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn\" (UID: \"5a3a2ee9-a66c-40b7-a2d1-5665a3574855\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn" Jul 07 16:23:38.244680 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:38.244646 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnlhb\" (UniqueName: \"kubernetes.io/projected/5a3a2ee9-a66c-40b7-a2d1-5665a3574855-kube-api-access-xnlhb\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn\" (UID: \"5a3a2ee9-a66c-40b7-a2d1-5665a3574855\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn" Jul 07 16:23:38.358834 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:38.358803 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn" Jul 07 16:23:38.494348 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:38.494325 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn"] Jul 07 16:23:38.496942 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:23:38.496917 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a3a2ee9_a66c_40b7_a2d1_5665a3574855.slice/crio-4ee199a2eaa4036e92f75e6511b73787347119cf1cd68c102e208ca25a037040 WatchSource:0}: Error finding container 4ee199a2eaa4036e92f75e6511b73787347119cf1cd68c102e208ca25a037040: Status 404 returned error can't find the container with id 4ee199a2eaa4036e92f75e6511b73787347119cf1cd68c102e208ca25a037040 Jul 07 16:23:38.498923 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:38.498889 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 16:23:38.736829 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:38.736744 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn" event={"ID":"5a3a2ee9-a66c-40b7-a2d1-5665a3574855","Type":"ContainerStarted","Data":"4ee199a2eaa4036e92f75e6511b73787347119cf1cd68c102e208ca25a037040"} Jul 07 16:23:43.759377 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:43.759340 2564 generic.go:356] "Generic (PLEG): container finished" podID="5a3a2ee9-a66c-40b7-a2d1-5665a3574855" containerID="cd6b3a0d98d39257134758af7fb0d8f1fff571b3a39d059fa771936c614ac876" exitCode=0 Jul 07 16:23:43.759827 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:43.759420 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn" event={"ID":"5a3a2ee9-a66c-40b7-a2d1-5665a3574855","Type":"ContainerDied","Data":"cd6b3a0d98d39257134758af7fb0d8f1fff571b3a39d059fa771936c614ac876"} Jul 07 16:23:46.773732 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:46.773690 2564 generic.go:356] "Generic (PLEG): container finished" podID="5a3a2ee9-a66c-40b7-a2d1-5665a3574855" containerID="cd70558af9f41601bfa2655aadaaff3bd0d889fb34137491ad333b253aa188e0" exitCode=0 Jul 07 16:23:46.774133 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:46.773752 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn" event={"ID":"5a3a2ee9-a66c-40b7-a2d1-5665a3574855","Type":"ContainerDied","Data":"cd70558af9f41601bfa2655aadaaff3bd0d889fb34137491ad333b253aa188e0"} Jul 07 16:23:55.816943 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:55.816835 2564 generic.go:356] "Generic (PLEG): container finished" podID="5a3a2ee9-a66c-40b7-a2d1-5665a3574855" containerID="e2933e0091b58a04d8dee2e700d5079658582588ee823006752e30074236f4b1" exitCode=0 Jul 07 16:23:55.817340 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:55.816930 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn" event={"ID":"5a3a2ee9-a66c-40b7-a2d1-5665a3574855","Type":"ContainerDied","Data":"e2933e0091b58a04d8dee2e700d5079658582588ee823006752e30074236f4b1"} Jul 07 16:23:56.963453 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:56.963421 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn" Jul 07 16:23:56.993921 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:56.993872 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnlhb\" (UniqueName: \"kubernetes.io/projected/5a3a2ee9-a66c-40b7-a2d1-5665a3574855-kube-api-access-xnlhb\") pod \"5a3a2ee9-a66c-40b7-a2d1-5665a3574855\" (UID: \"5a3a2ee9-a66c-40b7-a2d1-5665a3574855\") " Jul 07 16:23:56.993921 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:56.993926 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5a3a2ee9-a66c-40b7-a2d1-5665a3574855-util\") pod \"5a3a2ee9-a66c-40b7-a2d1-5665a3574855\" (UID: \"5a3a2ee9-a66c-40b7-a2d1-5665a3574855\") " Jul 07 16:23:56.994144 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:56.994004 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5a3a2ee9-a66c-40b7-a2d1-5665a3574855-bundle\") pod \"5a3a2ee9-a66c-40b7-a2d1-5665a3574855\" (UID: \"5a3a2ee9-a66c-40b7-a2d1-5665a3574855\") " Jul 07 16:23:56.994580 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:56.994550 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a3a2ee9-a66c-40b7-a2d1-5665a3574855-bundle" (OuterVolumeSpecName: "bundle") pod "5a3a2ee9-a66c-40b7-a2d1-5665a3574855" (UID: "5a3a2ee9-a66c-40b7-a2d1-5665a3574855"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:23:56.996234 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:56.996199 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a3a2ee9-a66c-40b7-a2d1-5665a3574855-kube-api-access-xnlhb" (OuterVolumeSpecName: "kube-api-access-xnlhb") pod "5a3a2ee9-a66c-40b7-a2d1-5665a3574855" (UID: "5a3a2ee9-a66c-40b7-a2d1-5665a3574855"). InnerVolumeSpecName "kube-api-access-xnlhb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:23:56.998358 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:56.998333 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a3a2ee9-a66c-40b7-a2d1-5665a3574855-util" (OuterVolumeSpecName: "util") pod "5a3a2ee9-a66c-40b7-a2d1-5665a3574855" (UID: "5a3a2ee9-a66c-40b7-a2d1-5665a3574855"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:23:57.095247 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:57.095162 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnlhb\" (UniqueName: \"kubernetes.io/projected/5a3a2ee9-a66c-40b7-a2d1-5665a3574855-kube-api-access-xnlhb\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:23:57.095247 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:57.095192 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5a3a2ee9-a66c-40b7-a2d1-5665a3574855-util\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:23:57.095247 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:57.095201 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5a3a2ee9-a66c-40b7-a2d1-5665a3574855-bundle\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:23:57.827007 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:57.826970 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn" event={"ID":"5a3a2ee9-a66c-40b7-a2d1-5665a3574855","Type":"ContainerDied","Data":"4ee199a2eaa4036e92f75e6511b73787347119cf1cd68c102e208ca25a037040"} Jul 07 16:23:57.827007 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:57.827006 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540xcbhn" Jul 07 16:23:57.827209 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:23:57.827017 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ee199a2eaa4036e92f75e6511b73787347119cf1cd68c102e208ca25a037040" Jul 07 16:24:01.240092 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:01.240058 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hb6z4"] Jul 07 16:24:01.240551 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:01.240384 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a3a2ee9-a66c-40b7-a2d1-5665a3574855" containerName="util" Jul 07 16:24:01.240551 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:01.240404 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a3a2ee9-a66c-40b7-a2d1-5665a3574855" containerName="util" Jul 07 16:24:01.240551 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:01.240412 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a3a2ee9-a66c-40b7-a2d1-5665a3574855" containerName="pull" Jul 07 16:24:01.240551 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:01.240417 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a3a2ee9-a66c-40b7-a2d1-5665a3574855" containerName="pull" Jul 07 16:24:01.240551 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:01.240426 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a3a2ee9-a66c-40b7-a2d1-5665a3574855" containerName="extract" Jul 07 16:24:01.240551 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:01.240432 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a3a2ee9-a66c-40b7-a2d1-5665a3574855" containerName="extract" Jul 07 16:24:01.240551 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:01.240485 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a3a2ee9-a66c-40b7-a2d1-5665a3574855" containerName="extract" Jul 07 16:24:01.242695 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:01.242681 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hb6z4" Jul 07 16:24:01.246298 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:01.246274 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 07 16:24:01.246298 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:01.246295 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-m27lq\"" Jul 07 16:24:01.246498 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:01.246296 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 07 16:24:01.264288 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:01.264260 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hb6z4"] Jul 07 16:24:01.335757 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:01.335717 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnh7k\" (UniqueName: \"kubernetes.io/projected/68aa18dd-02d3-44f5-bf02-73f3e45230e4-kube-api-access-gnh7k\") pod \"cert-manager-operator-controller-manager-55c444bf86-hb6z4\" (UID: \"68aa18dd-02d3-44f5-bf02-73f3e45230e4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hb6z4" Jul 07 16:24:01.335757 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:01.335760 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68aa18dd-02d3-44f5-bf02-73f3e45230e4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hb6z4\" (UID: \"68aa18dd-02d3-44f5-bf02-73f3e45230e4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hb6z4" Jul 07 16:24:01.436180 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:01.436142 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnh7k\" (UniqueName: \"kubernetes.io/projected/68aa18dd-02d3-44f5-bf02-73f3e45230e4-kube-api-access-gnh7k\") pod \"cert-manager-operator-controller-manager-55c444bf86-hb6z4\" (UID: \"68aa18dd-02d3-44f5-bf02-73f3e45230e4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hb6z4" Jul 07 16:24:01.436180 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:01.436182 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68aa18dd-02d3-44f5-bf02-73f3e45230e4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hb6z4\" (UID: \"68aa18dd-02d3-44f5-bf02-73f3e45230e4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hb6z4" Jul 07 16:24:01.436600 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:01.436583 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68aa18dd-02d3-44f5-bf02-73f3e45230e4-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hb6z4\" (UID: \"68aa18dd-02d3-44f5-bf02-73f3e45230e4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hb6z4" Jul 07 16:24:01.471756 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:01.471724 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnh7k\" (UniqueName: \"kubernetes.io/projected/68aa18dd-02d3-44f5-bf02-73f3e45230e4-kube-api-access-gnh7k\") pod \"cert-manager-operator-controller-manager-55c444bf86-hb6z4\" (UID: \"68aa18dd-02d3-44f5-bf02-73f3e45230e4\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hb6z4" Jul 07 16:24:01.552113 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:01.552016 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hb6z4" Jul 07 16:24:01.690749 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:01.690725 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hb6z4"] Jul 07 16:24:01.693583 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:24:01.693549 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68aa18dd_02d3_44f5_bf02_73f3e45230e4.slice/crio-a86a50ef407fcee46ab9f64a5d7826e33ff3c5487d38ab8b2e19fa70721b5df1 WatchSource:0}: Error finding container a86a50ef407fcee46ab9f64a5d7826e33ff3c5487d38ab8b2e19fa70721b5df1: Status 404 returned error can't find the container with id a86a50ef407fcee46ab9f64a5d7826e33ff3c5487d38ab8b2e19fa70721b5df1 Jul 07 16:24:01.843414 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:01.843325 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hb6z4" event={"ID":"68aa18dd-02d3-44f5-bf02-73f3e45230e4","Type":"ContainerStarted","Data":"a86a50ef407fcee46ab9f64a5d7826e33ff3c5487d38ab8b2e19fa70721b5df1"} Jul 07 16:24:03.854198 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:03.854163 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hb6z4" event={"ID":"68aa18dd-02d3-44f5-bf02-73f3e45230e4","Type":"ContainerStarted","Data":"e8cd6aeefaf1e3f44bee8182a37025fedb2c662844434665addec78ab393c781"} Jul 07 16:24:03.930699 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:03.930640 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hb6z4" podStartSLOduration=1.467144953 podStartE2EDuration="2.930623139s" totalImagesPullingTime="1.463478186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:24:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:24:03.930178541 +0000 UTC m=+352.091991497" watchObservedRunningTime="2026-07-07 16:24:03.930623139 +0000 UTC m=+352.092436093" Jul 07 16:24:04.352328 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:04.352246 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5xq8j"] Jul 07 16:24:04.354426 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:04.354410 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-5xq8j" Jul 07 16:24:04.360733 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:04.360714 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 16:24:04.360842 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:04.360714 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 16:24:04.361266 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:04.361245 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-9lmsx\"" Jul 07 16:24:04.378342 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:04.378304 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5xq8j"] Jul 07 16:24:04.464091 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:04.464050 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldhl9\" (UniqueName: \"kubernetes.io/projected/287156a8-c79d-410b-a488-9d82d6d58f09-kube-api-access-ldhl9\") pod \"cert-manager-webhook-6486ffd997-5xq8j\" (UID: \"287156a8-c79d-410b-a488-9d82d6d58f09\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5xq8j" Jul 07 16:24:04.464260 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:04.464183 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/287156a8-c79d-410b-a488-9d82d6d58f09-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5xq8j\" (UID: \"287156a8-c79d-410b-a488-9d82d6d58f09\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5xq8j" Jul 07 16:24:04.564789 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:04.564694 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldhl9\" (UniqueName: \"kubernetes.io/projected/287156a8-c79d-410b-a488-9d82d6d58f09-kube-api-access-ldhl9\") pod \"cert-manager-webhook-6486ffd997-5xq8j\" (UID: \"287156a8-c79d-410b-a488-9d82d6d58f09\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5xq8j" Jul 07 16:24:04.565010 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:04.564800 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/287156a8-c79d-410b-a488-9d82d6d58f09-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5xq8j\" (UID: \"287156a8-c79d-410b-a488-9d82d6d58f09\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5xq8j" Jul 07 16:24:04.592031 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:04.591996 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/287156a8-c79d-410b-a488-9d82d6d58f09-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5xq8j\" (UID: \"287156a8-c79d-410b-a488-9d82d6d58f09\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5xq8j" Jul 07 16:24:04.594200 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:04.594178 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldhl9\" (UniqueName: \"kubernetes.io/projected/287156a8-c79d-410b-a488-9d82d6d58f09-kube-api-access-ldhl9\") pod \"cert-manager-webhook-6486ffd997-5xq8j\" (UID: \"287156a8-c79d-410b-a488-9d82d6d58f09\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5xq8j" Jul 07 16:24:04.673627 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:04.673593 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-5xq8j" Jul 07 16:24:04.821715 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:04.821681 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5xq8j"] Jul 07 16:24:04.824757 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:24:04.824731 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod287156a8_c79d_410b_a488_9d82d6d58f09.slice/crio-9ec10da6e39ed7a6f6e4960568644e8c404b0f2ca5ae453170be983ff0403f21 WatchSource:0}: Error finding container 9ec10da6e39ed7a6f6e4960568644e8c404b0f2ca5ae453170be983ff0403f21: Status 404 returned error can't find the container with id 9ec10da6e39ed7a6f6e4960568644e8c404b0f2ca5ae453170be983ff0403f21 Jul 07 16:24:04.860133 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:04.860100 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-5xq8j" event={"ID":"287156a8-c79d-410b-a488-9d82d6d58f09","Type":"ContainerStarted","Data":"9ec10da6e39ed7a6f6e4960568644e8c404b0f2ca5ae453170be983ff0403f21"} Jul 07 16:24:05.623209 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:05.623173 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p"] Jul 07 16:24:05.627638 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:05.627619 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p" Jul 07 16:24:05.631679 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:05.631651 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-t7vz2\"" Jul 07 16:24:05.631974 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:05.631953 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 16:24:05.632689 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:05.632668 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 16:24:05.653130 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:05.653093 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p"] Jul 07 16:24:05.674796 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:05.674759 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45sfw\" (UniqueName: \"kubernetes.io/projected/cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2-kube-api-access-45sfw\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p\" (UID: \"cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p" Jul 07 16:24:05.674994 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:05.674818 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p\" (UID: \"cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p" Jul 07 16:24:05.674994 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:05.674854 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p\" (UID: \"cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p" Jul 07 16:24:05.776081 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:05.776041 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45sfw\" (UniqueName: \"kubernetes.io/projected/cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2-kube-api-access-45sfw\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p\" (UID: \"cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p" Jul 07 16:24:05.776278 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:05.776114 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p\" (UID: \"cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p" Jul 07 16:24:05.776278 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:05.776149 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p\" (UID: \"cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p" Jul 07 16:24:05.776551 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:05.776529 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p\" (UID: \"cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p" Jul 07 16:24:05.776551 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:05.776541 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p\" (UID: \"cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p" Jul 07 16:24:05.788345 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:05.788313 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45sfw\" (UniqueName: \"kubernetes.io/projected/cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2-kube-api-access-45sfw\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p\" (UID: \"cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p" Jul 07 16:24:05.937332 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:05.937240 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p" Jul 07 16:24:06.086429 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:06.086396 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p"] Jul 07 16:24:06.089895 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:24:06.089865 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc6e3b85_8c94_4fec_9cbf_dcbeda3323b2.slice/crio-54151baf25a9381d374c23670320d14d2b6bfa9a0b1e40742e749391c742abf7 WatchSource:0}: Error finding container 54151baf25a9381d374c23670320d14d2b6bfa9a0b1e40742e749391c742abf7: Status 404 returned error can't find the container with id 54151baf25a9381d374c23670320d14d2b6bfa9a0b1e40742e749391c742abf7 Jul 07 16:24:06.872412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:06.872376 2564 generic.go:356] "Generic (PLEG): container finished" podID="cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2" containerID="7d47da31623788bf4f3551a3c90d844b1850c7ea9bf6f3595efdae71fb746b62" exitCode=0 Jul 07 16:24:06.872611 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:06.872458 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p" event={"ID":"cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2","Type":"ContainerDied","Data":"7d47da31623788bf4f3551a3c90d844b1850c7ea9bf6f3595efdae71fb746b62"} Jul 07 16:24:06.872611 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:06.872500 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p" event={"ID":"cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2","Type":"ContainerStarted","Data":"54151baf25a9381d374c23670320d14d2b6bfa9a0b1e40742e749391c742abf7"} Jul 07 16:24:08.884067 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:08.884027 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-5xq8j" event={"ID":"287156a8-c79d-410b-a488-9d82d6d58f09","Type":"ContainerStarted","Data":"813b632372b88d61de4a64f3230d83e683652431de43939ce31b69c4fbc8e3a1"} Jul 07 16:24:08.884553 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:08.884162 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-5xq8j" Jul 07 16:24:10.740961 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:10.740893 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-5xq8j" podStartSLOduration=3.122735763 podStartE2EDuration="6.740873326s" totalImagesPullingTime="3.618137563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:24:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:24:08.921507615 +0000 UTC m=+357.083320573" watchObservedRunningTime="2026-07-07 16:24:10.740873326 +0000 UTC m=+358.902686290" Jul 07 16:24:10.741350 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:10.741216 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-7h2wt"] Jul 07 16:24:10.744304 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:10.744284 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7h2wt" Jul 07 16:24:10.747498 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:10.747480 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-dzk9n\"" Jul 07 16:24:10.764274 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:10.764247 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-7h2wt"] Jul 07 16:24:10.822656 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:10.822621 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2wgl\" (UniqueName: \"kubernetes.io/projected/bff8b016-24d0-49e4-be33-925f69e7c1b5-kube-api-access-t2wgl\") pod \"cert-manager-cainjector-559cdd8c87-7h2wt\" (UID: \"bff8b016-24d0-49e4-be33-925f69e7c1b5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7h2wt" Jul 07 16:24:10.822831 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:10.822666 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bff8b016-24d0-49e4-be33-925f69e7c1b5-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-7h2wt\" (UID: \"bff8b016-24d0-49e4-be33-925f69e7c1b5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7h2wt" Jul 07 16:24:10.893948 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:10.893894 2564 generic.go:356] "Generic (PLEG): container finished" podID="cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2" containerID="8ab8d8fb8b8a7d7d8957e4996aba6944ac29118965ab47cfd225dee96de5b99e" exitCode=0 Jul 07 16:24:10.894120 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:10.893954 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p" event={"ID":"cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2","Type":"ContainerDied","Data":"8ab8d8fb8b8a7d7d8957e4996aba6944ac29118965ab47cfd225dee96de5b99e"} Jul 07 16:24:10.923920 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:10.923862 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2wgl\" (UniqueName: \"kubernetes.io/projected/bff8b016-24d0-49e4-be33-925f69e7c1b5-kube-api-access-t2wgl\") pod \"cert-manager-cainjector-559cdd8c87-7h2wt\" (UID: \"bff8b016-24d0-49e4-be33-925f69e7c1b5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7h2wt" Jul 07 16:24:10.924052 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:10.924003 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bff8b016-24d0-49e4-be33-925f69e7c1b5-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-7h2wt\" (UID: \"bff8b016-24d0-49e4-be33-925f69e7c1b5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7h2wt" Jul 07 16:24:10.941098 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:10.941075 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2wgl\" (UniqueName: \"kubernetes.io/projected/bff8b016-24d0-49e4-be33-925f69e7c1b5-kube-api-access-t2wgl\") pod \"cert-manager-cainjector-559cdd8c87-7h2wt\" (UID: \"bff8b016-24d0-49e4-be33-925f69e7c1b5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7h2wt" Jul 07 16:24:10.948612 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:10.948588 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bff8b016-24d0-49e4-be33-925f69e7c1b5-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-7h2wt\" (UID: \"bff8b016-24d0-49e4-be33-925f69e7c1b5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-7h2wt" Jul 07 16:24:11.053439 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:11.053400 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7h2wt" Jul 07 16:24:11.190529 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:11.190497 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-7h2wt"] Jul 07 16:24:11.194664 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:24:11.194638 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbff8b016_24d0_49e4_be33_925f69e7c1b5.slice/crio-e3b71aa0c9eb08fb14dc72334fc96c0926af61f15607080d8fe778e90c490ff3 WatchSource:0}: Error finding container e3b71aa0c9eb08fb14dc72334fc96c0926af61f15607080d8fe778e90c490ff3: Status 404 returned error can't find the container with id e3b71aa0c9eb08fb14dc72334fc96c0926af61f15607080d8fe778e90c490ff3 Jul 07 16:24:11.900191 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:11.900158 2564 generic.go:356] "Generic (PLEG): container finished" podID="cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2" containerID="4316be71e77cfa974533993bd84cc9189cbf010eae2f7a763a86df3bde9215f1" exitCode=0 Jul 07 16:24:11.900605 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:11.900248 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p" event={"ID":"cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2","Type":"ContainerDied","Data":"4316be71e77cfa974533993bd84cc9189cbf010eae2f7a763a86df3bde9215f1"} Jul 07 16:24:11.901659 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:11.901635 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7h2wt" event={"ID":"bff8b016-24d0-49e4-be33-925f69e7c1b5","Type":"ContainerStarted","Data":"762c40409cddc9c46a91842857cdfd83b384ba14f1b6d37ab71c9d033d25a22b"} Jul 07 16:24:11.901787 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:11.901665 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7h2wt" event={"ID":"bff8b016-24d0-49e4-be33-925f69e7c1b5","Type":"ContainerStarted","Data":"e3b71aa0c9eb08fb14dc72334fc96c0926af61f15607080d8fe778e90c490ff3"} Jul 07 16:24:13.026571 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:13.026541 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p" Jul 07 16:24:13.077321 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:13.077261 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-7h2wt" podStartSLOduration=3.077243694 podStartE2EDuration="3.077243694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:24:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:24:11.941454395 +0000 UTC m=+360.103267350" watchObservedRunningTime="2026-07-07 16:24:13.077243694 +0000 UTC m=+361.239056670" Jul 07 16:24:13.144919 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:13.144886 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2-util\") pod \"cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2\" (UID: \"cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2\") " Jul 07 16:24:13.145071 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:13.144987 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45sfw\" (UniqueName: \"kubernetes.io/projected/cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2-kube-api-access-45sfw\") pod \"cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2\" (UID: \"cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2\") " Jul 07 16:24:13.145071 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:13.145022 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2-bundle\") pod \"cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2\" (UID: \"cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2\") " Jul 07 16:24:13.145429 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:13.145401 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2-bundle" (OuterVolumeSpecName: "bundle") pod "cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2" (UID: "cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:24:13.147065 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:13.147036 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2-kube-api-access-45sfw" (OuterVolumeSpecName: "kube-api-access-45sfw") pod "cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2" (UID: "cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2"). InnerVolumeSpecName "kube-api-access-45sfw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:24:13.150142 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:13.150103 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2-util" (OuterVolumeSpecName: "util") pod "cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2" (UID: "cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:24:13.245767 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:13.245691 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45sfw\" (UniqueName: \"kubernetes.io/projected/cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2-kube-api-access-45sfw\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:24:13.245767 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:13.245717 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2-bundle\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:24:13.245767 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:13.245726 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2-util\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:24:13.912086 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:13.912051 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p" event={"ID":"cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2","Type":"ContainerDied","Data":"54151baf25a9381d374c23670320d14d2b6bfa9a0b1e40742e749391c742abf7"} Jul 07 16:24:13.912239 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:13.912132 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54151baf25a9381d374c23670320d14d2b6bfa9a0b1e40742e749391c742abf7" Jul 07 16:24:13.912239 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:13.912065 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f44l6p" Jul 07 16:24:14.889932 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:14.889876 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-5xq8j" Jul 07 16:24:17.770714 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:17.770672 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpvdp"] Jul 07 16:24:17.771226 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:17.771136 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2" containerName="extract" Jul 07 16:24:17.771226 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:17.771154 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2" containerName="extract" Jul 07 16:24:17.771226 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:17.771172 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2" containerName="util" Jul 07 16:24:17.771226 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:17.771180 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2" containerName="util" Jul 07 16:24:17.771226 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:17.771203 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2" containerName="pull" Jul 07 16:24:17.771226 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:17.771211 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2" containerName="pull" Jul 07 16:24:17.771525 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:17.771282 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc6e3b85-8c94-4fec-9cbf-dcbeda3323b2" containerName="extract" Jul 07 16:24:17.775172 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:17.775152 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpvdp" Jul 07 16:24:17.778091 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:17.778070 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 16:24:17.778199 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:17.778112 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 16:24:17.779069 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:17.779050 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-n2sjf\"" Jul 07 16:24:17.784694 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:17.784669 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpvdp"] Jul 07 16:24:17.890606 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:17.890558 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l6pw\" (UniqueName: \"kubernetes.io/projected/1a1f1bb1-aa88-4eac-878c-d1f63c329a16-kube-api-access-2l6pw\") pod \"openshift-lws-operator-fd8ccff4c-tpvdp\" (UID: \"1a1f1bb1-aa88-4eac-878c-d1f63c329a16\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpvdp" Jul 07 16:24:17.890795 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:17.890697 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a1f1bb1-aa88-4eac-878c-d1f63c329a16-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tpvdp\" (UID: \"1a1f1bb1-aa88-4eac-878c-d1f63c329a16\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpvdp" Jul 07 16:24:17.991850 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:17.991809 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l6pw\" (UniqueName: \"kubernetes.io/projected/1a1f1bb1-aa88-4eac-878c-d1f63c329a16-kube-api-access-2l6pw\") pod \"openshift-lws-operator-fd8ccff4c-tpvdp\" (UID: \"1a1f1bb1-aa88-4eac-878c-d1f63c329a16\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpvdp" Jul 07 16:24:17.992089 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:17.991890 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a1f1bb1-aa88-4eac-878c-d1f63c329a16-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tpvdp\" (UID: \"1a1f1bb1-aa88-4eac-878c-d1f63c329a16\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpvdp" Jul 07 16:24:17.992288 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:17.992264 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a1f1bb1-aa88-4eac-878c-d1f63c329a16-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tpvdp\" (UID: \"1a1f1bb1-aa88-4eac-878c-d1f63c329a16\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpvdp" Jul 07 16:24:18.002057 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:18.002026 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l6pw\" (UniqueName: \"kubernetes.io/projected/1a1f1bb1-aa88-4eac-878c-d1f63c329a16-kube-api-access-2l6pw\") pod \"openshift-lws-operator-fd8ccff4c-tpvdp\" (UID: \"1a1f1bb1-aa88-4eac-878c-d1f63c329a16\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpvdp" Jul 07 16:24:18.085800 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:18.085709 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpvdp" Jul 07 16:24:18.220149 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:18.220100 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpvdp"] Jul 07 16:24:18.223138 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:24:18.223106 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a1f1bb1_aa88_4eac_878c_d1f63c329a16.slice/crio-d497f176ae3745e23dd980b71fb5f66dbb96ecf2f56a6db69ff80a2dd308a26e WatchSource:0}: Error finding container d497f176ae3745e23dd980b71fb5f66dbb96ecf2f56a6db69ff80a2dd308a26e: Status 404 returned error can't find the container with id d497f176ae3745e23dd980b71fb5f66dbb96ecf2f56a6db69ff80a2dd308a26e Jul 07 16:24:18.935250 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:18.935210 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpvdp" event={"ID":"1a1f1bb1-aa88-4eac-878c-d1f63c329a16","Type":"ContainerStarted","Data":"d497f176ae3745e23dd980b71fb5f66dbb96ecf2f56a6db69ff80a2dd308a26e"} Jul 07 16:24:20.946506 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:20.946467 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpvdp" event={"ID":"1a1f1bb1-aa88-4eac-878c-d1f63c329a16","Type":"ContainerStarted","Data":"07e0f08b73e7e64405a37bf4754f55427ffa49ac59524b315d7b197be0379061"} Jul 07 16:24:20.967883 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:20.967826 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tpvdp" podStartSLOduration=2.104019058 podStartE2EDuration="3.96780975s" totalImagesPullingTime="1.863790692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:24:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:24:20.966352175 +0000 UTC m=+369.128165132" watchObservedRunningTime="2026-07-07 16:24:20.96780975 +0000 UTC m=+369.129622787" Jul 07 16:24:23.207465 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:23.207431 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd"] Jul 07 16:24:23.256112 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:23.256073 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd"] Jul 07 16:24:23.256282 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:23.256189 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd" Jul 07 16:24:23.259543 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:23.259496 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 16:24:23.259699 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:23.259510 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 16:24:23.259699 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:23.259627 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-t7vz2\"" Jul 07 16:24:23.444535 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:23.444489 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99d8cd83-1d97-42ba-8791-70b58e5aa28d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd\" (UID: \"99d8cd83-1d97-42ba-8791-70b58e5aa28d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd" Jul 07 16:24:23.444535 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:23.444537 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9hm6\" (UniqueName: \"kubernetes.io/projected/99d8cd83-1d97-42ba-8791-70b58e5aa28d-kube-api-access-p9hm6\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd\" (UID: \"99d8cd83-1d97-42ba-8791-70b58e5aa28d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd" Jul 07 16:24:23.444771 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:23.444604 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99d8cd83-1d97-42ba-8791-70b58e5aa28d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd\" (UID: \"99d8cd83-1d97-42ba-8791-70b58e5aa28d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd" Jul 07 16:24:23.545492 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:23.545398 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99d8cd83-1d97-42ba-8791-70b58e5aa28d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd\" (UID: \"99d8cd83-1d97-42ba-8791-70b58e5aa28d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd" Jul 07 16:24:23.545492 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:23.545489 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99d8cd83-1d97-42ba-8791-70b58e5aa28d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd\" (UID: \"99d8cd83-1d97-42ba-8791-70b58e5aa28d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd" Jul 07 16:24:23.545738 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:23.545510 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9hm6\" (UniqueName: \"kubernetes.io/projected/99d8cd83-1d97-42ba-8791-70b58e5aa28d-kube-api-access-p9hm6\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd\" (UID: \"99d8cd83-1d97-42ba-8791-70b58e5aa28d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd" Jul 07 16:24:23.545813 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:23.545790 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99d8cd83-1d97-42ba-8791-70b58e5aa28d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd\" (UID: \"99d8cd83-1d97-42ba-8791-70b58e5aa28d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd" Jul 07 16:24:23.545900 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:23.545879 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99d8cd83-1d97-42ba-8791-70b58e5aa28d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd\" (UID: \"99d8cd83-1d97-42ba-8791-70b58e5aa28d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd" Jul 07 16:24:23.554252 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:23.554228 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9hm6\" (UniqueName: \"kubernetes.io/projected/99d8cd83-1d97-42ba-8791-70b58e5aa28d-kube-api-access-p9hm6\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd\" (UID: \"99d8cd83-1d97-42ba-8791-70b58e5aa28d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd" Jul 07 16:24:23.572266 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:23.572232 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd" Jul 07 16:24:23.729240 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:23.729171 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd"] Jul 07 16:24:23.731356 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:24:23.731331 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99d8cd83_1d97_42ba_8791_70b58e5aa28d.slice/crio-df479df645b298afce709c6bfc54b2e8a1f7785a1707372121784797fb2f3ed2 WatchSource:0}: Error finding container df479df645b298afce709c6bfc54b2e8a1f7785a1707372121784797fb2f3ed2: Status 404 returned error can't find the container with id df479df645b298afce709c6bfc54b2e8a1f7785a1707372121784797fb2f3ed2 Jul 07 16:24:23.964365 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:23.964325 2564 generic.go:356] "Generic (PLEG): container finished" podID="99d8cd83-1d97-42ba-8791-70b58e5aa28d" containerID="ec211ecfe5ef1a2c2479194d5cb2ce60b2b26f2b43c23c7bb8c4e8f915606e6e" exitCode=0 Jul 07 16:24:23.964545 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:23.964412 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd" event={"ID":"99d8cd83-1d97-42ba-8791-70b58e5aa28d","Type":"ContainerDied","Data":"ec211ecfe5ef1a2c2479194d5cb2ce60b2b26f2b43c23c7bb8c4e8f915606e6e"} Jul 07 16:24:23.964545 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:23.964459 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd" event={"ID":"99d8cd83-1d97-42ba-8791-70b58e5aa28d","Type":"ContainerStarted","Data":"df479df645b298afce709c6bfc54b2e8a1f7785a1707372121784797fb2f3ed2"} Jul 07 16:24:25.976303 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:25.976271 2564 generic.go:356] "Generic (PLEG): container finished" podID="99d8cd83-1d97-42ba-8791-70b58e5aa28d" containerID="d20eacc0498605e9f6aeb76a8b04349c79011438fd6651be0efe0904bd58b759" exitCode=0 Jul 07 16:24:25.976714 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:25.976362 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd" event={"ID":"99d8cd83-1d97-42ba-8791-70b58e5aa28d","Type":"ContainerDied","Data":"d20eacc0498605e9f6aeb76a8b04349c79011438fd6651be0efe0904bd58b759"} Jul 07 16:24:26.983348 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:26.983316 2564 generic.go:356] "Generic (PLEG): container finished" podID="99d8cd83-1d97-42ba-8791-70b58e5aa28d" containerID="8dc415df74cf57c7b79602d084f8e7fe53d2c7f18ea76253242273dbc70e664c" exitCode=0 Jul 07 16:24:26.983702 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:26.983400 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd" event={"ID":"99d8cd83-1d97-42ba-8791-70b58e5aa28d","Type":"ContainerDied","Data":"8dc415df74cf57c7b79602d084f8e7fe53d2c7f18ea76253242273dbc70e664c"} Jul 07 16:24:28.116017 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:28.115988 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd" Jul 07 16:24:28.284176 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:28.284084 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99d8cd83-1d97-42ba-8791-70b58e5aa28d-util\") pod \"99d8cd83-1d97-42ba-8791-70b58e5aa28d\" (UID: \"99d8cd83-1d97-42ba-8791-70b58e5aa28d\") " Jul 07 16:24:28.284176 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:28.284138 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9hm6\" (UniqueName: \"kubernetes.io/projected/99d8cd83-1d97-42ba-8791-70b58e5aa28d-kube-api-access-p9hm6\") pod \"99d8cd83-1d97-42ba-8791-70b58e5aa28d\" (UID: \"99d8cd83-1d97-42ba-8791-70b58e5aa28d\") " Jul 07 16:24:28.284402 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:28.284189 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99d8cd83-1d97-42ba-8791-70b58e5aa28d-bundle\") pod \"99d8cd83-1d97-42ba-8791-70b58e5aa28d\" (UID: \"99d8cd83-1d97-42ba-8791-70b58e5aa28d\") " Jul 07 16:24:28.284983 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:28.284951 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99d8cd83-1d97-42ba-8791-70b58e5aa28d-bundle" (OuterVolumeSpecName: "bundle") pod "99d8cd83-1d97-42ba-8791-70b58e5aa28d" (UID: "99d8cd83-1d97-42ba-8791-70b58e5aa28d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:24:28.286355 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:28.286321 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99d8cd83-1d97-42ba-8791-70b58e5aa28d-kube-api-access-p9hm6" (OuterVolumeSpecName: "kube-api-access-p9hm6") pod "99d8cd83-1d97-42ba-8791-70b58e5aa28d" (UID: "99d8cd83-1d97-42ba-8791-70b58e5aa28d"). InnerVolumeSpecName "kube-api-access-p9hm6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:24:28.289975 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:28.289941 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99d8cd83-1d97-42ba-8791-70b58e5aa28d-util" (OuterVolumeSpecName: "util") pod "99d8cd83-1d97-42ba-8791-70b58e5aa28d" (UID: "99d8cd83-1d97-42ba-8791-70b58e5aa28d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:24:28.385492 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:28.385451 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/99d8cd83-1d97-42ba-8791-70b58e5aa28d-bundle\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:24:28.385492 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:28.385482 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/99d8cd83-1d97-42ba-8791-70b58e5aa28d-util\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:24:28.385492 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:28.385493 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9hm6\" (UniqueName: \"kubernetes.io/projected/99d8cd83-1d97-42ba-8791-70b58e5aa28d-kube-api-access-p9hm6\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:24:28.994370 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:28.994341 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd" Jul 07 16:24:28.994544 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:28.994378 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af92dsd" event={"ID":"99d8cd83-1d97-42ba-8791-70b58e5aa28d","Type":"ContainerDied","Data":"df479df645b298afce709c6bfc54b2e8a1f7785a1707372121784797fb2f3ed2"} Jul 07 16:24:28.994544 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:28.994423 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df479df645b298afce709c6bfc54b2e8a1f7785a1707372121784797fb2f3ed2" Jul 07 16:24:39.617668 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.617627 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb"] Jul 07 16:24:39.618309 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.618154 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99d8cd83-1d97-42ba-8791-70b58e5aa28d" containerName="util" Jul 07 16:24:39.618309 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.618175 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="99d8cd83-1d97-42ba-8791-70b58e5aa28d" containerName="util" Jul 07 16:24:39.618309 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.618210 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99d8cd83-1d97-42ba-8791-70b58e5aa28d" containerName="extract" Jul 07 16:24:39.618309 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.618219 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="99d8cd83-1d97-42ba-8791-70b58e5aa28d" containerName="extract" Jul 07 16:24:39.618309 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.618238 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99d8cd83-1d97-42ba-8791-70b58e5aa28d" containerName="pull" Jul 07 16:24:39.618309 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.618247 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="99d8cd83-1d97-42ba-8791-70b58e5aa28d" containerName="pull" Jul 07 16:24:39.618610 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.618331 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="99d8cd83-1d97-42ba-8791-70b58e5aa28d" containerName="extract" Jul 07 16:24:39.620651 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.620629 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" Jul 07 16:24:39.624703 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.624681 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 16:24:39.625026 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.625007 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 16:24:39.625164 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.625142 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gg9gr\"" Jul 07 16:24:39.625277 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.625198 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 16:24:39.625277 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.625254 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 16:24:39.637923 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.637857 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb"] Jul 07 16:24:39.669826 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.669791 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwhmz\" (UniqueName: \"kubernetes.io/projected/bdb26e74-5407-41be-a50f-c9ecfbd8836f-kube-api-access-jwhmz\") pod \"opendatahub-operator-controller-manager-6f64778974-22mbb\" (UID: \"bdb26e74-5407-41be-a50f-c9ecfbd8836f\") " pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" Jul 07 16:24:39.669826 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.669831 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdb26e74-5407-41be-a50f-c9ecfbd8836f-tmp\") pod \"opendatahub-operator-controller-manager-6f64778974-22mbb\" (UID: \"bdb26e74-5407-41be-a50f-c9ecfbd8836f\") " pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" Jul 07 16:24:39.670074 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.669858 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bdb26e74-5407-41be-a50f-c9ecfbd8836f-manifests\") pod \"opendatahub-operator-controller-manager-6f64778974-22mbb\" (UID: \"bdb26e74-5407-41be-a50f-c9ecfbd8836f\") " pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" Jul 07 16:24:39.670074 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.669942 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bdb26e74-5407-41be-a50f-c9ecfbd8836f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f64778974-22mbb\" (UID: \"bdb26e74-5407-41be-a50f-c9ecfbd8836f\") " pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" Jul 07 16:24:39.670074 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.669971 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bdb26e74-5407-41be-a50f-c9ecfbd8836f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f64778974-22mbb\" (UID: \"bdb26e74-5407-41be-a50f-c9ecfbd8836f\") " pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" Jul 07 16:24:39.771038 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.770995 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwhmz\" (UniqueName: \"kubernetes.io/projected/bdb26e74-5407-41be-a50f-c9ecfbd8836f-kube-api-access-jwhmz\") pod \"opendatahub-operator-controller-manager-6f64778974-22mbb\" (UID: \"bdb26e74-5407-41be-a50f-c9ecfbd8836f\") " pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" Jul 07 16:24:39.771038 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.771043 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdb26e74-5407-41be-a50f-c9ecfbd8836f-tmp\") pod \"opendatahub-operator-controller-manager-6f64778974-22mbb\" (UID: \"bdb26e74-5407-41be-a50f-c9ecfbd8836f\") " pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" Jul 07 16:24:39.771308 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.771069 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bdb26e74-5407-41be-a50f-c9ecfbd8836f-manifests\") pod \"opendatahub-operator-controller-manager-6f64778974-22mbb\" (UID: \"bdb26e74-5407-41be-a50f-c9ecfbd8836f\") " pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" Jul 07 16:24:39.771308 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.771088 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bdb26e74-5407-41be-a50f-c9ecfbd8836f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f64778974-22mbb\" (UID: \"bdb26e74-5407-41be-a50f-c9ecfbd8836f\") " pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" Jul 07 16:24:39.771308 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.771111 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bdb26e74-5407-41be-a50f-c9ecfbd8836f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f64778974-22mbb\" (UID: \"bdb26e74-5407-41be-a50f-c9ecfbd8836f\") " pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" Jul 07 16:24:39.771460 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.771430 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bdb26e74-5407-41be-a50f-c9ecfbd8836f-manifests\") pod \"opendatahub-operator-controller-manager-6f64778974-22mbb\" (UID: \"bdb26e74-5407-41be-a50f-c9ecfbd8836f\") " pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" Jul 07 16:24:39.773417 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.773386 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bdb26e74-5407-41be-a50f-c9ecfbd8836f-tmp\") pod \"opendatahub-operator-controller-manager-6f64778974-22mbb\" (UID: \"bdb26e74-5407-41be-a50f-c9ecfbd8836f\") " pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" Jul 07 16:24:39.773522 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.773475 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bdb26e74-5407-41be-a50f-c9ecfbd8836f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6f64778974-22mbb\" (UID: \"bdb26e74-5407-41be-a50f-c9ecfbd8836f\") " pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" Jul 07 16:24:39.773578 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.773519 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bdb26e74-5407-41be-a50f-c9ecfbd8836f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6f64778974-22mbb\" (UID: \"bdb26e74-5407-41be-a50f-c9ecfbd8836f\") " pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" Jul 07 16:24:39.781074 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.781044 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwhmz\" (UniqueName: \"kubernetes.io/projected/bdb26e74-5407-41be-a50f-c9ecfbd8836f-kube-api-access-jwhmz\") pod \"opendatahub-operator-controller-manager-6f64778974-22mbb\" (UID: \"bdb26e74-5407-41be-a50f-c9ecfbd8836f\") " pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" Jul 07 16:24:39.931086 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:39.930991 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" Jul 07 16:24:40.070705 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:40.070678 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb"] Jul 07 16:24:40.073112 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:24:40.073084 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdb26e74_5407_41be_a50f_c9ecfbd8836f.slice/crio-77bf5fbedb3143a029dd2e3f67dd5e95d52e34df176b2462caf64d1c1e32d083 WatchSource:0}: Error finding container 77bf5fbedb3143a029dd2e3f67dd5e95d52e34df176b2462caf64d1c1e32d083: Status 404 returned error can't find the container with id 77bf5fbedb3143a029dd2e3f67dd5e95d52e34df176b2462caf64d1c1e32d083 Jul 07 16:24:41.052954 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:41.052888 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" event={"ID":"bdb26e74-5407-41be-a50f-c9ecfbd8836f","Type":"ContainerStarted","Data":"77bf5fbedb3143a029dd2e3f67dd5e95d52e34df176b2462caf64d1c1e32d083"} Jul 07 16:24:44.071162 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:44.071127 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" event={"ID":"bdb26e74-5407-41be-a50f-c9ecfbd8836f","Type":"ContainerStarted","Data":"79602b9949c8301a559ae4ab22a719e08286c20a8f6f26614b6b6823827dc412"} Jul 07 16:24:45.077414 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:45.077379 2564 generic.go:356] "Generic (PLEG): container finished" podID="bdb26e74-5407-41be-a50f-c9ecfbd8836f" containerID="79602b9949c8301a559ae4ab22a719e08286c20a8f6f26614b6b6823827dc412" exitCode=0 Jul 07 16:24:45.077858 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:45.077474 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" event={"ID":"bdb26e74-5407-41be-a50f-c9ecfbd8836f","Type":"ContainerDied","Data":"79602b9949c8301a559ae4ab22a719e08286c20a8f6f26614b6b6823827dc412"} Jul 07 16:24:46.083401 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:46.083363 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" event={"ID":"bdb26e74-5407-41be-a50f-c9ecfbd8836f","Type":"ContainerStarted","Data":"26bdd9afc1f620731525dfaf94e06f2156b8811486bccab712827336c80beff7"} Jul 07 16:24:46.083830 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:46.083564 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" Jul 07 16:24:46.111529 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:46.111474 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" podStartSLOduration=2.396116139 podStartE2EDuration="7.111457749s" totalImagesPullingTime="3.679198947s" totalInitContainerRuntime="1.036142663s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:24:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:24:46.109093801 +0000 UTC m=+394.270906784" watchObservedRunningTime="2026-07-07 16:24:46.111457749 +0000 UTC m=+394.273270704" Jul 07 16:24:57.090570 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:57.090534 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6f64778974-22mbb" Jul 07 16:24:59.721367 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.721325 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7cbbdb79bb-lbrks"] Jul 07 16:24:59.730292 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.730259 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7cbbdb79bb-lbrks" Jul 07 16:24:59.733679 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.733650 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 07 16:24:59.733679 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.733673 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-6xdvb\"" Jul 07 16:24:59.734722 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.734703 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 16:24:59.734857 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.734704 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 16:24:59.734857 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.734708 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 16:24:59.738388 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.738366 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7cbbdb79bb-lbrks"] Jul 07 16:24:59.850763 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.850726 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6gzws"] Jul 07 16:24:59.854117 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.854101 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-6gzws" Jul 07 16:24:59.858313 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.858288 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 07 16:24:59.858441 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.858361 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-5dnnj\"" Jul 07 16:24:59.859655 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.859627 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6f791a8-4e75-4f89-9b8f-c65fdd57aa21-tls-certs\") pod \"kube-auth-proxy-7cbbdb79bb-lbrks\" (UID: \"d6f791a8-4e75-4f89-9b8f-c65fdd57aa21\") " pod="openshift-ingress/kube-auth-proxy-7cbbdb79bb-lbrks" Jul 07 16:24:59.859747 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.859671 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6f791a8-4e75-4f89-9b8f-c65fdd57aa21-tmp\") pod \"kube-auth-proxy-7cbbdb79bb-lbrks\" (UID: \"d6f791a8-4e75-4f89-9b8f-c65fdd57aa21\") " pod="openshift-ingress/kube-auth-proxy-7cbbdb79bb-lbrks" Jul 07 16:24:59.859811 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.859779 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvpbl\" (UniqueName: \"kubernetes.io/projected/d6f791a8-4e75-4f89-9b8f-c65fdd57aa21-kube-api-access-hvpbl\") pod \"kube-auth-proxy-7cbbdb79bb-lbrks\" (UID: \"d6f791a8-4e75-4f89-9b8f-c65fdd57aa21\") " pod="openshift-ingress/kube-auth-proxy-7cbbdb79bb-lbrks" Jul 07 16:24:59.870591 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.870564 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6gzws"] Jul 07 16:24:59.960621 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.960591 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvpbl\" (UniqueName: \"kubernetes.io/projected/d6f791a8-4e75-4f89-9b8f-c65fdd57aa21-kube-api-access-hvpbl\") pod \"kube-auth-proxy-7cbbdb79bb-lbrks\" (UID: \"d6f791a8-4e75-4f89-9b8f-c65fdd57aa21\") " pod="openshift-ingress/kube-auth-proxy-7cbbdb79bb-lbrks" Jul 07 16:24:59.960822 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.960636 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bce89f98-6a41-4485-a4c7-f1bb3368e25a-cert\") pod \"odh-model-controller-7998c67dc4-6gzws\" (UID: \"bce89f98-6a41-4485-a4c7-f1bb3368e25a\") " pod="opendatahub/odh-model-controller-7998c67dc4-6gzws" Jul 07 16:24:59.960822 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.960675 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6f791a8-4e75-4f89-9b8f-c65fdd57aa21-tls-certs\") pod \"kube-auth-proxy-7cbbdb79bb-lbrks\" (UID: \"d6f791a8-4e75-4f89-9b8f-c65fdd57aa21\") " pod="openshift-ingress/kube-auth-proxy-7cbbdb79bb-lbrks" Jul 07 16:24:59.960822 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.960711 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6f791a8-4e75-4f89-9b8f-c65fdd57aa21-tmp\") pod \"kube-auth-proxy-7cbbdb79bb-lbrks\" (UID: \"d6f791a8-4e75-4f89-9b8f-c65fdd57aa21\") " pod="openshift-ingress/kube-auth-proxy-7cbbdb79bb-lbrks" Jul 07 16:24:59.960822 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.960736 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjv55\" (UniqueName: \"kubernetes.io/projected/bce89f98-6a41-4485-a4c7-f1bb3368e25a-kube-api-access-cjv55\") pod \"odh-model-controller-7998c67dc4-6gzws\" (UID: \"bce89f98-6a41-4485-a4c7-f1bb3368e25a\") " pod="opendatahub/odh-model-controller-7998c67dc4-6gzws" Jul 07 16:24:59.963052 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.963026 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d6f791a8-4e75-4f89-9b8f-c65fdd57aa21-tmp\") pod \"kube-auth-proxy-7cbbdb79bb-lbrks\" (UID: \"d6f791a8-4e75-4f89-9b8f-c65fdd57aa21\") " pod="openshift-ingress/kube-auth-proxy-7cbbdb79bb-lbrks" Jul 07 16:24:59.963258 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.963242 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6f791a8-4e75-4f89-9b8f-c65fdd57aa21-tls-certs\") pod \"kube-auth-proxy-7cbbdb79bb-lbrks\" (UID: \"d6f791a8-4e75-4f89-9b8f-c65fdd57aa21\") " pod="openshift-ingress/kube-auth-proxy-7cbbdb79bb-lbrks" Jul 07 16:24:59.969643 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:24:59.969615 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvpbl\" (UniqueName: \"kubernetes.io/projected/d6f791a8-4e75-4f89-9b8f-c65fdd57aa21-kube-api-access-hvpbl\") pod \"kube-auth-proxy-7cbbdb79bb-lbrks\" (UID: \"d6f791a8-4e75-4f89-9b8f-c65fdd57aa21\") " pod="openshift-ingress/kube-auth-proxy-7cbbdb79bb-lbrks" Jul 07 16:25:00.041382 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:00.041290 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7cbbdb79bb-lbrks" Jul 07 16:25:00.062024 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:00.061992 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjv55\" (UniqueName: \"kubernetes.io/projected/bce89f98-6a41-4485-a4c7-f1bb3368e25a-kube-api-access-cjv55\") pod \"odh-model-controller-7998c67dc4-6gzws\" (UID: \"bce89f98-6a41-4485-a4c7-f1bb3368e25a\") " pod="opendatahub/odh-model-controller-7998c67dc4-6gzws" Jul 07 16:25:00.062162 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:00.062083 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bce89f98-6a41-4485-a4c7-f1bb3368e25a-cert\") pod \"odh-model-controller-7998c67dc4-6gzws\" (UID: \"bce89f98-6a41-4485-a4c7-f1bb3368e25a\") " pod="opendatahub/odh-model-controller-7998c67dc4-6gzws" Jul 07 16:25:00.062254 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:25:00.062233 2564 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 07 16:25:00.062311 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:25:00.062300 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bce89f98-6a41-4485-a4c7-f1bb3368e25a-cert podName:bce89f98-6a41-4485-a4c7-f1bb3368e25a nodeName:}" failed. No retries permitted until 2026-07-07 16:25:00.56228473 +0000 UTC m=+408.724097663 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bce89f98-6a41-4485-a4c7-f1bb3368e25a-cert") pod "odh-model-controller-7998c67dc4-6gzws" (UID: "bce89f98-6a41-4485-a4c7-f1bb3368e25a") : secret "odh-model-controller-webhook-cert" not found Jul 07 16:25:00.072935 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:00.072892 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjv55\" (UniqueName: \"kubernetes.io/projected/bce89f98-6a41-4485-a4c7-f1bb3368e25a-kube-api-access-cjv55\") pod \"odh-model-controller-7998c67dc4-6gzws\" (UID: \"bce89f98-6a41-4485-a4c7-f1bb3368e25a\") " pod="opendatahub/odh-model-controller-7998c67dc4-6gzws" Jul 07 16:25:00.187244 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:00.187210 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7cbbdb79bb-lbrks"] Jul 07 16:25:00.190632 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:25:00.190605 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6f791a8_4e75_4f89_9b8f_c65fdd57aa21.slice/crio-f251f68e3ed711fa120caafea2946a421444cada71d9a441f0af8611ffc3e823 WatchSource:0}: Error finding container f251f68e3ed711fa120caafea2946a421444cada71d9a441f0af8611ffc3e823: Status 404 returned error can't find the container with id f251f68e3ed711fa120caafea2946a421444cada71d9a441f0af8611ffc3e823 Jul 07 16:25:00.567471 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:00.567430 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bce89f98-6a41-4485-a4c7-f1bb3368e25a-cert\") pod \"odh-model-controller-7998c67dc4-6gzws\" (UID: \"bce89f98-6a41-4485-a4c7-f1bb3368e25a\") " pod="opendatahub/odh-model-controller-7998c67dc4-6gzws" Jul 07 16:25:00.570015 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:00.569994 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bce89f98-6a41-4485-a4c7-f1bb3368e25a-cert\") pod \"odh-model-controller-7998c67dc4-6gzws\" (UID: \"bce89f98-6a41-4485-a4c7-f1bb3368e25a\") " pod="opendatahub/odh-model-controller-7998c67dc4-6gzws" Jul 07 16:25:00.764622 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:00.764586 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-6gzws" Jul 07 16:25:00.901625 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:00.901599 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6gzws"] Jul 07 16:25:00.904286 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:25:00.904257 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbce89f98_6a41_4485_a4c7_f1bb3368e25a.slice/crio-f287297a44dca270ec428dfa3cc54675818c39e75010e0470bcc651a058308d5 WatchSource:0}: Error finding container f287297a44dca270ec428dfa3cc54675818c39e75010e0470bcc651a058308d5: Status 404 returned error can't find the container with id f287297a44dca270ec428dfa3cc54675818c39e75010e0470bcc651a058308d5 Jul 07 16:25:01.154643 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:01.154595 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7cbbdb79bb-lbrks" event={"ID":"d6f791a8-4e75-4f89-9b8f-c65fdd57aa21","Type":"ContainerStarted","Data":"f251f68e3ed711fa120caafea2946a421444cada71d9a441f0af8611ffc3e823"} Jul 07 16:25:01.155974 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:01.155942 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-6gzws" event={"ID":"bce89f98-6a41-4485-a4c7-f1bb3368e25a","Type":"ContainerStarted","Data":"f287297a44dca270ec428dfa3cc54675818c39e75010e0470bcc651a058308d5"} Jul 07 16:25:05.184744 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.184709 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-6gzws" event={"ID":"bce89f98-6a41-4485-a4c7-f1bb3368e25a","Type":"ContainerStarted","Data":"bc4da4d1e115d157af3127c216656bdbf8586557e28cdc64f7604709de728ac9"} Jul 07 16:25:05.185289 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.184779 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-6gzws" Jul 07 16:25:05.186223 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.186198 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7cbbdb79bb-lbrks" event={"ID":"d6f791a8-4e75-4f89-9b8f-c65fdd57aa21","Type":"ContainerStarted","Data":"9fe8851469c511c72bf5a1e8e405667615d4d66b188e17e9c20c4e04bf4b38ef"} Jul 07 16:25:05.209492 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.209439 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-6gzws" podStartSLOduration=2.451298032 podStartE2EDuration="6.209422805s" totalImagesPullingTime="3.758124773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:25:05.208467793 +0000 UTC m=+413.370280749" watchObservedRunningTime="2026-07-07 16:25:05.209422805 +0000 UTC m=+413.371235760" Jul 07 16:25:05.228397 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.228344 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7cbbdb79bb-lbrks" podStartSLOduration=1.8038001989999999 podStartE2EDuration="6.228327733s" totalImagesPullingTime="4.424527534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:25:05.228249094 +0000 UTC m=+413.390062069" watchObservedRunningTime="2026-07-07 16:25:05.228327733 +0000 UTC m=+413.390140733" Jul 07 16:25:05.399873 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.399834 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2"] Jul 07 16:25:05.403252 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.403230 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.405839 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.405813 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 16:25:05.405982 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.405949 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-n5l44\"" Jul 07 16:25:05.433332 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.433301 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2"] Jul 07 16:25:05.506779 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.506685 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.506779 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.506742 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5s69\" (UniqueName: \"kubernetes.io/projected/2469130b-3bcf-40a7-9e1f-57097ff363ac-kube-api-access-z5s69\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.507043 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.506822 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.507043 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.506855 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.507043 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.506896 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2469130b-3bcf-40a7-9e1f-57097ff363ac-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.507043 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.506962 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.507210 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.507053 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.507210 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.507106 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.507210 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.507151 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.608115 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.608083 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.608298 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.608124 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5s69\" (UniqueName: \"kubernetes.io/projected/2469130b-3bcf-40a7-9e1f-57097ff363ac-kube-api-access-z5s69\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.608298 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.608165 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.608298 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.608191 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.608298 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.608213 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2469130b-3bcf-40a7-9e1f-57097ff363ac-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.608298 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.608250 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.608558 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.608318 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.608558 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.608369 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.608558 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.608406 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.608831 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.608771 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-data\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.608831 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.608772 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.608831 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.608817 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.609043 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.608847 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.609423 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.609401 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2469130b-3bcf-40a7-9e1f-57097ff363ac-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.610584 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.610557 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.610864 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.610845 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.633879 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.633828 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5s69\" (UniqueName: \"kubernetes.io/projected/2469130b-3bcf-40a7-9e1f-57097ff363ac-kube-api-access-z5s69\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.634876 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.634857 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-token\") pod \"data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.713739 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.713688 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:05.861227 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:05.861195 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2"] Jul 07 16:25:05.863742 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:25:05.863718 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2469130b_3bcf_40a7_9e1f_57097ff363ac.slice/crio-fac2c6b7361e14bffa79386c2adec4685156b7ce883545e5d1f2bd3669c55b6a WatchSource:0}: Error finding container fac2c6b7361e14bffa79386c2adec4685156b7ce883545e5d1f2bd3669c55b6a: Status 404 returned error can't find the container with id fac2c6b7361e14bffa79386c2adec4685156b7ce883545e5d1f2bd3669c55b6a Jul 07 16:25:06.191897 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:06.191856 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" event={"ID":"2469130b-3bcf-40a7-9e1f-57097ff363ac","Type":"ContainerStarted","Data":"fac2c6b7361e14bffa79386c2adec4685156b7ce883545e5d1f2bd3669c55b6a"} Jul 07 16:25:08.285934 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.285880 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz"] Jul 07 16:25:08.293003 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.292977 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:08.306116 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.306088 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 07 16:25:08.306272 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.306125 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 07 16:25:08.307188 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.307145 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-mnzlh\"" Jul 07 16:25:08.307188 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.307175 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 07 16:25:08.339022 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.338984 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz"] Jul 07 16:25:08.439274 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.439238 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2cbd34f1-8649-41be-9595-798f5e903723-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-pdhmz\" (UID: \"2cbd34f1-8649-41be-9595-798f5e903723\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:08.439274 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.439281 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cbd34f1-8649-41be-9595-798f5e903723-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-pdhmz\" (UID: \"2cbd34f1-8649-41be-9595-798f5e903723\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:08.439560 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.439379 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cbd34f1-8649-41be-9595-798f5e903723-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-pdhmz\" (UID: \"2cbd34f1-8649-41be-9595-798f5e903723\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:08.439560 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.439425 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2cbd34f1-8649-41be-9595-798f5e903723-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-pdhmz\" (UID: \"2cbd34f1-8649-41be-9595-798f5e903723\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:08.439560 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.439447 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2cbd34f1-8649-41be-9595-798f5e903723-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-pdhmz\" (UID: \"2cbd34f1-8649-41be-9595-798f5e903723\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:08.439560 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.439469 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nc4d\" (UniqueName: \"kubernetes.io/projected/2cbd34f1-8649-41be-9595-798f5e903723-kube-api-access-7nc4d\") pod \"istiod-openshift-gateway-94bb8fbfd-pdhmz\" (UID: \"2cbd34f1-8649-41be-9595-798f5e903723\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:08.439560 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.439496 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2cbd34f1-8649-41be-9595-798f5e903723-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-pdhmz\" (UID: \"2cbd34f1-8649-41be-9595-798f5e903723\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:08.540263 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.540178 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cbd34f1-8649-41be-9595-798f5e903723-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-pdhmz\" (UID: \"2cbd34f1-8649-41be-9595-798f5e903723\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:08.540263 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.540221 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2cbd34f1-8649-41be-9595-798f5e903723-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-pdhmz\" (UID: \"2cbd34f1-8649-41be-9595-798f5e903723\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:08.540263 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.540250 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2cbd34f1-8649-41be-9595-798f5e903723-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-pdhmz\" (UID: \"2cbd34f1-8649-41be-9595-798f5e903723\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:08.540538 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.540275 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nc4d\" (UniqueName: \"kubernetes.io/projected/2cbd34f1-8649-41be-9595-798f5e903723-kube-api-access-7nc4d\") pod \"istiod-openshift-gateway-94bb8fbfd-pdhmz\" (UID: \"2cbd34f1-8649-41be-9595-798f5e903723\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:08.540538 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.540299 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2cbd34f1-8649-41be-9595-798f5e903723-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-pdhmz\" (UID: \"2cbd34f1-8649-41be-9595-798f5e903723\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:08.540538 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.540400 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2cbd34f1-8649-41be-9595-798f5e903723-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-pdhmz\" (UID: \"2cbd34f1-8649-41be-9595-798f5e903723\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:08.540538 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.540432 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cbd34f1-8649-41be-9595-798f5e903723-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-pdhmz\" (UID: \"2cbd34f1-8649-41be-9595-798f5e903723\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:08.540989 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.540955 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2cbd34f1-8649-41be-9595-798f5e903723-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-pdhmz\" (UID: \"2cbd34f1-8649-41be-9595-798f5e903723\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:08.542729 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.542703 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2cbd34f1-8649-41be-9595-798f5e903723-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-pdhmz\" (UID: \"2cbd34f1-8649-41be-9595-798f5e903723\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:08.542849 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.542754 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cbd34f1-8649-41be-9595-798f5e903723-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-pdhmz\" (UID: \"2cbd34f1-8649-41be-9595-798f5e903723\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:08.543037 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.543015 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2cbd34f1-8649-41be-9595-798f5e903723-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-pdhmz\" (UID: \"2cbd34f1-8649-41be-9595-798f5e903723\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:08.543143 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.543124 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2cbd34f1-8649-41be-9595-798f5e903723-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-pdhmz\" (UID: \"2cbd34f1-8649-41be-9595-798f5e903723\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:08.550921 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.550879 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cbd34f1-8649-41be-9595-798f5e903723-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-pdhmz\" (UID: \"2cbd34f1-8649-41be-9595-798f5e903723\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:08.554144 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.554120 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nc4d\" (UniqueName: \"kubernetes.io/projected/2cbd34f1-8649-41be-9595-798f5e903723-kube-api-access-7nc4d\") pod \"istiod-openshift-gateway-94bb8fbfd-pdhmz\" (UID: \"2cbd34f1-8649-41be-9595-798f5e903723\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:08.575072 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.575031 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:25:08.575161 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.575112 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:25:08.575161 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.575142 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:25:08.605042 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.605014 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:08.761979 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:08.761946 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz"] Jul 07 16:25:08.767735 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:25:08.767706 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cbd34f1_8649_41be_9595_798f5e903723.slice/crio-68f7f23115515a62c045669eaeb7c878dc6bac8c1cc92f7ed2f6970545c489f9 WatchSource:0}: Error finding container 68f7f23115515a62c045669eaeb7c878dc6bac8c1cc92f7ed2f6970545c489f9: Status 404 returned error can't find the container with id 68f7f23115515a62c045669eaeb7c878dc6bac8c1cc92f7ed2f6970545c489f9 Jul 07 16:25:09.209062 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:09.209027 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" event={"ID":"2cbd34f1-8649-41be-9595-798f5e903723","Type":"ContainerStarted","Data":"68f7f23115515a62c045669eaeb7c878dc6bac8c1cc92f7ed2f6970545c489f9"} Jul 07 16:25:09.211170 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:09.211138 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" event={"ID":"2469130b-3bcf-40a7-9e1f-57097ff363ac","Type":"ContainerStarted","Data":"58f73def9d4f033e9c605f680745e9a3f1140e76d9c762a6652202d083955343"} Jul 07 16:25:09.238129 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:09.238082 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" podStartSLOduration=1.529531472 podStartE2EDuration="4.238064917s" totalImagesPullingTime="2.708533445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:25:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:25:09.236233881 +0000 UTC m=+417.398046839" watchObservedRunningTime="2026-07-07 16:25:09.238064917 +0000 UTC m=+417.399877875" Jul 07 16:25:09.714339 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:09.714300 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:09.716094 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:09.716063 2564 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 07 16:25:09.716230 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:09.716125 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" podUID="2469130b-3bcf-40a7-9e1f-57097ff363ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 07 16:25:10.154685 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.154644 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2"] Jul 07 16:25:10.491814 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.491725 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xcq85"] Jul 07 16:25:10.499135 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.499103 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq85" Jul 07 16:25:10.512324 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.510499 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-wlmt2\"" Jul 07 16:25:10.512324 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.510786 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 07 16:25:10.517620 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.515845 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lnck9"] Jul 07 16:25:10.523155 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.523128 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xcq85"] Jul 07 16:25:10.523501 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.523447 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lnck9" Jul 07 16:25:10.528959 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.528937 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 07 16:25:10.529354 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.529338 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-xs4nt\"" Jul 07 16:25:10.536116 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.536091 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lnck9"] Jul 07 16:25:10.661340 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.661298 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8492c6fd-ea82-4bb1-ab8e-4455c75dbe6d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lnck9\" (UID: \"8492c6fd-ea82-4bb1-ab8e-4455c75dbe6d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lnck9" Jul 07 16:25:10.661546 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.661353 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5abd0a46-2e65-49fd-a06d-30ee5cd38791-cert\") pod \"kserve-controller-manager-d6f4b65b8-xcq85\" (UID: \"5abd0a46-2e65-49fd-a06d-30ee5cd38791\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq85" Jul 07 16:25:10.661546 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.661382 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqwkp\" (UniqueName: \"kubernetes.io/projected/8492c6fd-ea82-4bb1-ab8e-4455c75dbe6d-kube-api-access-dqwkp\") pod \"llmisvc-controller-manager-777c6bd7d5-lnck9\" (UID: \"8492c6fd-ea82-4bb1-ab8e-4455c75dbe6d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lnck9" Jul 07 16:25:10.661546 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.661477 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv7qg\" (UniqueName: \"kubernetes.io/projected/5abd0a46-2e65-49fd-a06d-30ee5cd38791-kube-api-access-sv7qg\") pod \"kserve-controller-manager-d6f4b65b8-xcq85\" (UID: \"5abd0a46-2e65-49fd-a06d-30ee5cd38791\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq85" Jul 07 16:25:10.714647 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.714610 2564 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" start-of-body= Jul 07 16:25:10.715170 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.714670 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" podUID="2469130b-3bcf-40a7-9e1f-57097ff363ac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.35:15021/healthz/ready\": dial tcp 10.132.0.35:15021: connect: connection refused" Jul 07 16:25:10.762245 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.762151 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8492c6fd-ea82-4bb1-ab8e-4455c75dbe6d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lnck9\" (UID: \"8492c6fd-ea82-4bb1-ab8e-4455c75dbe6d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lnck9" Jul 07 16:25:10.762245 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.762203 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5abd0a46-2e65-49fd-a06d-30ee5cd38791-cert\") pod \"kserve-controller-manager-d6f4b65b8-xcq85\" (UID: \"5abd0a46-2e65-49fd-a06d-30ee5cd38791\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq85" Jul 07 16:25:10.762245 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.762223 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqwkp\" (UniqueName: \"kubernetes.io/projected/8492c6fd-ea82-4bb1-ab8e-4455c75dbe6d-kube-api-access-dqwkp\") pod \"llmisvc-controller-manager-777c6bd7d5-lnck9\" (UID: \"8492c6fd-ea82-4bb1-ab8e-4455c75dbe6d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lnck9" Jul 07 16:25:10.762575 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.762267 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv7qg\" (UniqueName: \"kubernetes.io/projected/5abd0a46-2e65-49fd-a06d-30ee5cd38791-kube-api-access-sv7qg\") pod \"kserve-controller-manager-d6f4b65b8-xcq85\" (UID: \"5abd0a46-2e65-49fd-a06d-30ee5cd38791\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq85" Jul 07 16:25:10.765024 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.764994 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5abd0a46-2e65-49fd-a06d-30ee5cd38791-cert\") pod \"kserve-controller-manager-d6f4b65b8-xcq85\" (UID: \"5abd0a46-2e65-49fd-a06d-30ee5cd38791\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq85" Jul 07 16:25:10.765164 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.765005 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8492c6fd-ea82-4bb1-ab8e-4455c75dbe6d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lnck9\" (UID: \"8492c6fd-ea82-4bb1-ab8e-4455c75dbe6d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lnck9" Jul 07 16:25:10.775842 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.775820 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqwkp\" (UniqueName: \"kubernetes.io/projected/8492c6fd-ea82-4bb1-ab8e-4455c75dbe6d-kube-api-access-dqwkp\") pod \"llmisvc-controller-manager-777c6bd7d5-lnck9\" (UID: \"8492c6fd-ea82-4bb1-ab8e-4455c75dbe6d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lnck9" Jul 07 16:25:10.782585 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.782559 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv7qg\" (UniqueName: \"kubernetes.io/projected/5abd0a46-2e65-49fd-a06d-30ee5cd38791-kube-api-access-sv7qg\") pod \"kserve-controller-manager-d6f4b65b8-xcq85\" (UID: \"5abd0a46-2e65-49fd-a06d-30ee5cd38791\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq85" Jul 07 16:25:10.817087 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.817050 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq85" Jul 07 16:25:10.854155 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.854113 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lnck9" Jul 07 16:25:10.956927 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.956856 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:25:10.957057 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:10.956982 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:25:11.030152 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:11.030125 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lnck9"] Jul 07 16:25:11.032206 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:25:11.032100 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8492c6fd_ea82_4bb1_ab8e_4455c75dbe6d.slice/crio-30f39f8430759f4a3d85caaec5e535fbbd451696488d26b43ed2b7c7594d47f7 WatchSource:0}: Error finding container 30f39f8430759f4a3d85caaec5e535fbbd451696488d26b43ed2b7c7594d47f7: Status 404 returned error can't find the container with id 30f39f8430759f4a3d85caaec5e535fbbd451696488d26b43ed2b7c7594d47f7 Jul 07 16:25:11.049029 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:11.049004 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-xcq85"] Jul 07 16:25:11.051823 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:25:11.051796 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5abd0a46_2e65_49fd_a06d_30ee5cd38791.slice/crio-18d3b4e3d4bbdd3947eda472da12588759d5bc360ebffd2358816ef5bf9533bf WatchSource:0}: Error finding container 18d3b4e3d4bbdd3947eda472da12588759d5bc360ebffd2358816ef5bf9533bf: Status 404 returned error can't find the container with id 18d3b4e3d4bbdd3947eda472da12588759d5bc360ebffd2358816ef5bf9533bf Jul 07 16:25:11.227550 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:11.227510 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lnck9" event={"ID":"8492c6fd-ea82-4bb1-ab8e-4455c75dbe6d","Type":"ContainerStarted","Data":"30f39f8430759f4a3d85caaec5e535fbbd451696488d26b43ed2b7c7594d47f7"} Jul 07 16:25:11.229035 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:11.229002 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" event={"ID":"2cbd34f1-8649-41be-9595-798f5e903723","Type":"ContainerStarted","Data":"32386ff8ffdde35212e4c8ab8aaeafa230698ff58798b2ca79284e8e73831578"} Jul 07 16:25:11.229194 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:11.229118 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:11.230263 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:11.230238 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq85" event={"ID":"5abd0a46-2e65-49fd-a06d-30ee5cd38791","Type":"ContainerStarted","Data":"18d3b4e3d4bbdd3947eda472da12588759d5bc360ebffd2358816ef5bf9533bf"} Jul 07 16:25:11.230386 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:11.230321 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" podUID="2469130b-3bcf-40a7-9e1f-57097ff363ac" containerName="istio-proxy" containerID="cri-o://58f73def9d4f033e9c605f680745e9a3f1140e76d9c762a6652202d083955343" gracePeriod=30 Jul 07 16:25:11.263107 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:11.263060 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" podStartSLOduration=1.076726824 podStartE2EDuration="3.263041974s" totalImagesPullingTime="2.18631515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:25:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:25:11.26230742 +0000 UTC m=+419.424120376" watchObservedRunningTime="2026-07-07 16:25:11.263041974 +0000 UTC m=+419.424854928" Jul 07 16:25:12.239175 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:12.239140 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-pdhmz" Jul 07 16:25:15.261265 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:15.261230 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq85" event={"ID":"5abd0a46-2e65-49fd-a06d-30ee5cd38791","Type":"ContainerStarted","Data":"1c51e14b3d0f8079d66fc0c877557241bf7c5887fc2521249887444b44715719"} Jul 07 16:25:15.261725 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:15.261309 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq85" Jul 07 16:25:15.282841 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:15.282783 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq85" podStartSLOduration=2.103257223 podStartE2EDuration="5.282767245s" totalImagesPullingTime="3.179510022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:25:15.279830066 +0000 UTC m=+423.441643021" watchObservedRunningTime="2026-07-07 16:25:15.282767245 +0000 UTC m=+423.444580200" Jul 07 16:25:16.194565 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.194535 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-6gzws" Jul 07 16:25:16.267651 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.267614 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lnck9" event={"ID":"8492c6fd-ea82-4bb1-ab8e-4455c75dbe6d","Type":"ContainerStarted","Data":"9066fe721a4f0a9959a2e5a604b2eff5c13bfa87c2b3fb7841bb5381823579b7"} Jul 07 16:25:16.268090 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.267884 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lnck9" Jul 07 16:25:16.286586 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.286539 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lnck9" podStartSLOduration=2.021003238 podStartE2EDuration="6.286524269s" totalImagesPullingTime="4.265521031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:25:16.284183898 +0000 UTC m=+424.445996843" watchObservedRunningTime="2026-07-07 16:25:16.286524269 +0000 UTC m=+424.448337255" Jul 07 16:25:16.487958 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.487932 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:16.616340 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.616302 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-data\") pod \"2469130b-3bcf-40a7-9e1f-57097ff363ac\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " Jul 07 16:25:16.616340 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.616347 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-envoy\") pod \"2469130b-3bcf-40a7-9e1f-57097ff363ac\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " Jul 07 16:25:16.616619 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.616379 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2469130b-3bcf-40a7-9e1f-57097ff363ac-istiod-ca-cert\") pod \"2469130b-3bcf-40a7-9e1f-57097ff363ac\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " Jul 07 16:25:16.616619 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.616424 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-workload-socket\") pod \"2469130b-3bcf-40a7-9e1f-57097ff363ac\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " Jul 07 16:25:16.616619 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.616462 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-workload-certs\") pod \"2469130b-3bcf-40a7-9e1f-57097ff363ac\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " Jul 07 16:25:16.616619 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.616492 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-credential-socket\") pod \"2469130b-3bcf-40a7-9e1f-57097ff363ac\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " Jul 07 16:25:16.616619 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.616520 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-podinfo\") pod \"2469130b-3bcf-40a7-9e1f-57097ff363ac\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " Jul 07 16:25:16.616619 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.616594 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-token\") pod \"2469130b-3bcf-40a7-9e1f-57097ff363ac\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " Jul 07 16:25:16.616946 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.616628 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5s69\" (UniqueName: \"kubernetes.io/projected/2469130b-3bcf-40a7-9e1f-57097ff363ac-kube-api-access-z5s69\") pod \"2469130b-3bcf-40a7-9e1f-57097ff363ac\" (UID: \"2469130b-3bcf-40a7-9e1f-57097ff363ac\") " Jul 07 16:25:16.616946 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.616772 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-data" (OuterVolumeSpecName: "istio-data") pod "2469130b-3bcf-40a7-9e1f-57097ff363ac" (UID: "2469130b-3bcf-40a7-9e1f-57097ff363ac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:25:16.616946 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.616932 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-data\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:25:16.617087 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.617051 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2469130b-3bcf-40a7-9e1f-57097ff363ac" (UID: "2469130b-3bcf-40a7-9e1f-57097ff363ac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:25:16.617167 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.617094 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2469130b-3bcf-40a7-9e1f-57097ff363ac" (UID: "2469130b-3bcf-40a7-9e1f-57097ff363ac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:25:16.617285 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.617243 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2469130b-3bcf-40a7-9e1f-57097ff363ac-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2469130b-3bcf-40a7-9e1f-57097ff363ac" (UID: "2469130b-3bcf-40a7-9e1f-57097ff363ac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:25:16.617402 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.617314 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2469130b-3bcf-40a7-9e1f-57097ff363ac" (UID: "2469130b-3bcf-40a7-9e1f-57097ff363ac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:25:16.619210 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.619154 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2469130b-3bcf-40a7-9e1f-57097ff363ac" (UID: "2469130b-3bcf-40a7-9e1f-57097ff363ac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 16:25:16.619210 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.619172 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2469130b-3bcf-40a7-9e1f-57097ff363ac-kube-api-access-z5s69" (OuterVolumeSpecName: "kube-api-access-z5s69") pod "2469130b-3bcf-40a7-9e1f-57097ff363ac" (UID: "2469130b-3bcf-40a7-9e1f-57097ff363ac"). InnerVolumeSpecName "kube-api-access-z5s69". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:25:16.619210 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.619198 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2469130b-3bcf-40a7-9e1f-57097ff363ac" (UID: "2469130b-3bcf-40a7-9e1f-57097ff363ac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:25:16.619429 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.619348 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-token" (OuterVolumeSpecName: "istio-token") pod "2469130b-3bcf-40a7-9e1f-57097ff363ac" (UID: "2469130b-3bcf-40a7-9e1f-57097ff363ac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:25:16.718377 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.718279 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-workload-socket\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:25:16.718377 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.718318 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-workload-certs\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:25:16.718377 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.718328 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-credential-socket\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:25:16.718377 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.718338 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-podinfo\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:25:16.718377 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.718349 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-token\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:25:16.718377 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.718359 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5s69\" (UniqueName: \"kubernetes.io/projected/2469130b-3bcf-40a7-9e1f-57097ff363ac-kube-api-access-z5s69\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:25:16.718377 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.718367 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2469130b-3bcf-40a7-9e1f-57097ff363ac-istio-envoy\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:25:16.718377 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:16.718376 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2469130b-3bcf-40a7-9e1f-57097ff363ac-istiod-ca-cert\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:25:17.274371 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:17.274341 2564 generic.go:356] "Generic (PLEG): container finished" podID="2469130b-3bcf-40a7-9e1f-57097ff363ac" containerID="58f73def9d4f033e9c605f680745e9a3f1140e76d9c762a6652202d083955343" exitCode=0 Jul 07 16:25:17.274773 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:17.274388 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" Jul 07 16:25:17.274773 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:17.274424 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" event={"ID":"2469130b-3bcf-40a7-9e1f-57097ff363ac","Type":"ContainerDied","Data":"58f73def9d4f033e9c605f680745e9a3f1140e76d9c762a6652202d083955343"} Jul 07 16:25:17.274773 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:17.274472 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2" event={"ID":"2469130b-3bcf-40a7-9e1f-57097ff363ac","Type":"ContainerDied","Data":"fac2c6b7361e14bffa79386c2adec4685156b7ce883545e5d1f2bd3669c55b6a"} Jul 07 16:25:17.274773 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:17.274494 2564 scope.go:117] "RemoveContainer" containerID="58f73def9d4f033e9c605f680745e9a3f1140e76d9c762a6652202d083955343" Jul 07 16:25:17.284234 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:17.284217 2564 scope.go:117] "RemoveContainer" containerID="58f73def9d4f033e9c605f680745e9a3f1140e76d9c762a6652202d083955343" Jul 07 16:25:17.284510 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:25:17.284494 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58f73def9d4f033e9c605f680745e9a3f1140e76d9c762a6652202d083955343\": container with ID starting with 58f73def9d4f033e9c605f680745e9a3f1140e76d9c762a6652202d083955343 not found: ID does not exist" containerID="58f73def9d4f033e9c605f680745e9a3f1140e76d9c762a6652202d083955343" Jul 07 16:25:17.284583 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:17.284520 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58f73def9d4f033e9c605f680745e9a3f1140e76d9c762a6652202d083955343"} err="failed to get container status \"58f73def9d4f033e9c605f680745e9a3f1140e76d9c762a6652202d083955343\": rpc error: code = NotFound desc = could not find container \"58f73def9d4f033e9c605f680745e9a3f1140e76d9c762a6652202d083955343\": container with ID starting with 58f73def9d4f033e9c605f680745e9a3f1140e76d9c762a6652202d083955343 not found: ID does not exist" Jul 07 16:25:17.304890 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:17.304857 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2"] Jul 07 16:25:17.315305 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:17.315271 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-84cc848bd-nb2c2"] Jul 07 16:25:18.355662 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:18.355622 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2469130b-3bcf-40a7-9e1f-57097ff363ac" path="/var/lib/kubelet/pods/2469130b-3bcf-40a7-9e1f-57097ff363ac/volumes" Jul 07 16:25:46.273799 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:46.273715 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-xcq85" Jul 07 16:25:47.276608 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:25:47.276578 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lnck9" Jul 07 16:26:16.073923 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.073866 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4"] Jul 07 16:26:16.074352 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.074258 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2469130b-3bcf-40a7-9e1f-57097ff363ac" containerName="istio-proxy" Jul 07 16:26:16.074352 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.074272 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2469130b-3bcf-40a7-9e1f-57097ff363ac" containerName="istio-proxy" Jul 07 16:26:16.074352 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.074335 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2469130b-3bcf-40a7-9e1f-57097ff363ac" containerName="istio-proxy" Jul 07 16:26:16.078786 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.078768 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4" Jul 07 16:26:16.081276 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.081254 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 16:26:16.082524 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.082484 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 16:26:16.082742 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.082725 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-468st\"" Jul 07 16:26:16.085725 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.085705 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4"] Jul 07 16:26:16.136601 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.136557 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b330d35-9651-498e-abaa-3ae7b4831c6a-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4\" (UID: \"1b330d35-9651-498e-abaa-3ae7b4831c6a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4" Jul 07 16:26:16.136773 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.136658 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b330d35-9651-498e-abaa-3ae7b4831c6a-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4\" (UID: \"1b330d35-9651-498e-abaa-3ae7b4831c6a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4" Jul 07 16:26:16.136773 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.136690 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4nsl\" (UniqueName: \"kubernetes.io/projected/1b330d35-9651-498e-abaa-3ae7b4831c6a-kube-api-access-p4nsl\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4\" (UID: \"1b330d35-9651-498e-abaa-3ae7b4831c6a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4" Jul 07 16:26:16.237459 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.237423 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b330d35-9651-498e-abaa-3ae7b4831c6a-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4\" (UID: \"1b330d35-9651-498e-abaa-3ae7b4831c6a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4" Jul 07 16:26:16.237662 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.237479 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b330d35-9651-498e-abaa-3ae7b4831c6a-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4\" (UID: \"1b330d35-9651-498e-abaa-3ae7b4831c6a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4" Jul 07 16:26:16.237662 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.237508 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4nsl\" (UniqueName: \"kubernetes.io/projected/1b330d35-9651-498e-abaa-3ae7b4831c6a-kube-api-access-p4nsl\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4\" (UID: \"1b330d35-9651-498e-abaa-3ae7b4831c6a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4" Jul 07 16:26:16.237795 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.237779 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b330d35-9651-498e-abaa-3ae7b4831c6a-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4\" (UID: \"1b330d35-9651-498e-abaa-3ae7b4831c6a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4" Jul 07 16:26:16.237891 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.237872 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b330d35-9651-498e-abaa-3ae7b4831c6a-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4\" (UID: \"1b330d35-9651-498e-abaa-3ae7b4831c6a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4" Jul 07 16:26:16.247080 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.247058 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4nsl\" (UniqueName: \"kubernetes.io/projected/1b330d35-9651-498e-abaa-3ae7b4831c6a-kube-api-access-p4nsl\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4\" (UID: \"1b330d35-9651-498e-abaa-3ae7b4831c6a\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4" Jul 07 16:26:16.389689 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.389643 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4" Jul 07 16:26:16.470392 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.470355 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf"] Jul 07 16:26:16.475283 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.475258 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf" Jul 07 16:26:16.481022 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.480992 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf"] Jul 07 16:26:16.535527 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.535502 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4"] Jul 07 16:26:16.537366 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:26:16.537339 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b330d35_9651_498e_abaa_3ae7b4831c6a.slice/crio-ee284eceb7dcf435700ca4e315f486e1aef13595ad3b4e65e5d297bcf1a45264 WatchSource:0}: Error finding container ee284eceb7dcf435700ca4e315f486e1aef13595ad3b4e65e5d297bcf1a45264: Status 404 returned error can't find the container with id ee284eceb7dcf435700ca4e315f486e1aef13595ad3b4e65e5d297bcf1a45264 Jul 07 16:26:16.539527 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.539503 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/372fdb21-39c4-4378-af66-8c3728384e8c-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf\" (UID: \"372fdb21-39c4-4378-af66-8c3728384e8c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf" Jul 07 16:26:16.539631 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.539537 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-792pm\" (UniqueName: \"kubernetes.io/projected/372fdb21-39c4-4378-af66-8c3728384e8c-kube-api-access-792pm\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf\" (UID: \"372fdb21-39c4-4378-af66-8c3728384e8c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf" Jul 07 16:26:16.539631 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.539562 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/372fdb21-39c4-4378-af66-8c3728384e8c-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf\" (UID: \"372fdb21-39c4-4378-af66-8c3728384e8c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf" Jul 07 16:26:16.575755 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.575693 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4" event={"ID":"1b330d35-9651-498e-abaa-3ae7b4831c6a","Type":"ContainerStarted","Data":"ee284eceb7dcf435700ca4e315f486e1aef13595ad3b4e65e5d297bcf1a45264"} Jul 07 16:26:16.641008 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.640948 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/372fdb21-39c4-4378-af66-8c3728384e8c-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf\" (UID: \"372fdb21-39c4-4378-af66-8c3728384e8c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf" Jul 07 16:26:16.641143 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.641074 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/372fdb21-39c4-4378-af66-8c3728384e8c-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf\" (UID: \"372fdb21-39c4-4378-af66-8c3728384e8c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf" Jul 07 16:26:16.641143 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.641116 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-792pm\" (UniqueName: \"kubernetes.io/projected/372fdb21-39c4-4378-af66-8c3728384e8c-kube-api-access-792pm\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf\" (UID: \"372fdb21-39c4-4378-af66-8c3728384e8c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf" Jul 07 16:26:16.641416 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.641375 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/372fdb21-39c4-4378-af66-8c3728384e8c-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf\" (UID: \"372fdb21-39c4-4378-af66-8c3728384e8c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf" Jul 07 16:26:16.641546 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.641411 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/372fdb21-39c4-4378-af66-8c3728384e8c-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf\" (UID: \"372fdb21-39c4-4378-af66-8c3728384e8c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf" Jul 07 16:26:16.650249 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.650220 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-792pm\" (UniqueName: \"kubernetes.io/projected/372fdb21-39c4-4378-af66-8c3728384e8c-kube-api-access-792pm\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf\" (UID: \"372fdb21-39c4-4378-af66-8c3728384e8c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf" Jul 07 16:26:16.788734 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.788678 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf" Jul 07 16:26:16.866012 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.865981 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v"] Jul 07 16:26:16.869895 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.869874 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v" Jul 07 16:26:16.878289 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.877803 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v"] Jul 07 16:26:16.932005 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.931923 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf"] Jul 07 16:26:16.935321 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:26:16.935298 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod372fdb21_39c4_4378_af66_8c3728384e8c.slice/crio-40a2826a872210cbdff7d75081dba6b5c927fd852ba6c95f9c1bbaea0dbdefda WatchSource:0}: Error finding container 40a2826a872210cbdff7d75081dba6b5c927fd852ba6c95f9c1bbaea0dbdefda: Status 404 returned error can't find the container with id 40a2826a872210cbdff7d75081dba6b5c927fd852ba6c95f9c1bbaea0dbdefda Jul 07 16:26:16.942662 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.942641 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35b8f5f2-9e59-4c68-a767-a6e6d45ba798-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v\" (UID: \"35b8f5f2-9e59-4c68-a767-a6e6d45ba798\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v" Jul 07 16:26:16.942771 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.942681 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdkhl\" (UniqueName: \"kubernetes.io/projected/35b8f5f2-9e59-4c68-a767-a6e6d45ba798-kube-api-access-qdkhl\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v\" (UID: \"35b8f5f2-9e59-4c68-a767-a6e6d45ba798\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v" Jul 07 16:26:16.942771 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:16.942707 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35b8f5f2-9e59-4c68-a767-a6e6d45ba798-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v\" (UID: \"35b8f5f2-9e59-4c68-a767-a6e6d45ba798\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v" Jul 07 16:26:17.043266 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.043235 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdkhl\" (UniqueName: \"kubernetes.io/projected/35b8f5f2-9e59-4c68-a767-a6e6d45ba798-kube-api-access-qdkhl\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v\" (UID: \"35b8f5f2-9e59-4c68-a767-a6e6d45ba798\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v" Jul 07 16:26:17.043424 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.043292 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35b8f5f2-9e59-4c68-a767-a6e6d45ba798-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v\" (UID: \"35b8f5f2-9e59-4c68-a767-a6e6d45ba798\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v" Jul 07 16:26:17.043424 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.043378 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35b8f5f2-9e59-4c68-a767-a6e6d45ba798-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v\" (UID: \"35b8f5f2-9e59-4c68-a767-a6e6d45ba798\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v" Jul 07 16:26:17.043680 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.043658 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35b8f5f2-9e59-4c68-a767-a6e6d45ba798-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v\" (UID: \"35b8f5f2-9e59-4c68-a767-a6e6d45ba798\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v" Jul 07 16:26:17.043753 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.043718 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35b8f5f2-9e59-4c68-a767-a6e6d45ba798-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v\" (UID: \"35b8f5f2-9e59-4c68-a767-a6e6d45ba798\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v" Jul 07 16:26:17.052268 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.052239 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdkhl\" (UniqueName: \"kubernetes.io/projected/35b8f5f2-9e59-4c68-a767-a6e6d45ba798-kube-api-access-qdkhl\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v\" (UID: \"35b8f5f2-9e59-4c68-a767-a6e6d45ba798\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v" Jul 07 16:26:17.181367 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.181329 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v" Jul 07 16:26:17.301689 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.301653 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp"] Jul 07 16:26:17.307118 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.307086 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp" Jul 07 16:26:17.313163 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.313135 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp"] Jul 07 16:26:17.323484 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.323447 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v"] Jul 07 16:26:17.324047 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:26:17.324022 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35b8f5f2_9e59_4c68_a767_a6e6d45ba798.slice/crio-ea49a80abaf6a9a8f457268f7e13e990bdcddb74e34e9cd856a58eceda1415a0 WatchSource:0}: Error finding container ea49a80abaf6a9a8f457268f7e13e990bdcddb74e34e9cd856a58eceda1415a0: Status 404 returned error can't find the container with id ea49a80abaf6a9a8f457268f7e13e990bdcddb74e34e9cd856a58eceda1415a0 Jul 07 16:26:17.346082 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.346051 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfkth\" (UniqueName: \"kubernetes.io/projected/54d4386c-0b67-4cac-a953-23cc4175999b-kube-api-access-zfkth\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp\" (UID: \"54d4386c-0b67-4cac-a953-23cc4175999b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp" Jul 07 16:26:17.346187 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.346095 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54d4386c-0b67-4cac-a953-23cc4175999b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp\" (UID: \"54d4386c-0b67-4cac-a953-23cc4175999b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp" Jul 07 16:26:17.346330 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.346250 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54d4386c-0b67-4cac-a953-23cc4175999b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp\" (UID: \"54d4386c-0b67-4cac-a953-23cc4175999b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp" Jul 07 16:26:17.447529 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.447448 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54d4386c-0b67-4cac-a953-23cc4175999b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp\" (UID: \"54d4386c-0b67-4cac-a953-23cc4175999b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp" Jul 07 16:26:17.447681 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.447536 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfkth\" (UniqueName: \"kubernetes.io/projected/54d4386c-0b67-4cac-a953-23cc4175999b-kube-api-access-zfkth\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp\" (UID: \"54d4386c-0b67-4cac-a953-23cc4175999b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp" Jul 07 16:26:17.447681 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.447558 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54d4386c-0b67-4cac-a953-23cc4175999b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp\" (UID: \"54d4386c-0b67-4cac-a953-23cc4175999b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp" Jul 07 16:26:17.447833 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.447813 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54d4386c-0b67-4cac-a953-23cc4175999b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp\" (UID: \"54d4386c-0b67-4cac-a953-23cc4175999b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp" Jul 07 16:26:17.447876 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.447851 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54d4386c-0b67-4cac-a953-23cc4175999b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp\" (UID: \"54d4386c-0b67-4cac-a953-23cc4175999b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp" Jul 07 16:26:17.456452 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.456420 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfkth\" (UniqueName: \"kubernetes.io/projected/54d4386c-0b67-4cac-a953-23cc4175999b-kube-api-access-zfkth\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp\" (UID: \"54d4386c-0b67-4cac-a953-23cc4175999b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp" Jul 07 16:26:17.582622 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.582583 2564 generic.go:356] "Generic (PLEG): container finished" podID="35b8f5f2-9e59-4c68-a767-a6e6d45ba798" containerID="164e17499a68095210e0bf8379aebb0bcb716b18e1550825a709e0f65a63d6df" exitCode=0 Jul 07 16:26:17.582885 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.582676 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v" event={"ID":"35b8f5f2-9e59-4c68-a767-a6e6d45ba798","Type":"ContainerDied","Data":"164e17499a68095210e0bf8379aebb0bcb716b18e1550825a709e0f65a63d6df"} Jul 07 16:26:17.582885 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.582734 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v" event={"ID":"35b8f5f2-9e59-4c68-a767-a6e6d45ba798","Type":"ContainerStarted","Data":"ea49a80abaf6a9a8f457268f7e13e990bdcddb74e34e9cd856a58eceda1415a0"} Jul 07 16:26:17.584405 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.584383 2564 generic.go:356] "Generic (PLEG): container finished" podID="372fdb21-39c4-4378-af66-8c3728384e8c" containerID="f490ae16868a6fa0da0925f1223ae4b7bd912f21213dffa5e6f5e65c08a2dadd" exitCode=0 Jul 07 16:26:17.584505 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.584434 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf" event={"ID":"372fdb21-39c4-4378-af66-8c3728384e8c","Type":"ContainerDied","Data":"f490ae16868a6fa0da0925f1223ae4b7bd912f21213dffa5e6f5e65c08a2dadd"} Jul 07 16:26:17.584505 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.584472 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf" event={"ID":"372fdb21-39c4-4378-af66-8c3728384e8c","Type":"ContainerStarted","Data":"40a2826a872210cbdff7d75081dba6b5c927fd852ba6c95f9c1bbaea0dbdefda"} Jul 07 16:26:17.586024 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.585999 2564 generic.go:356] "Generic (PLEG): container finished" podID="1b330d35-9651-498e-abaa-3ae7b4831c6a" containerID="75d55374b08740f13a4ccac03b03c25d24aa872ee09ddd39f28844de41821ac0" exitCode=0 Jul 07 16:26:17.586143 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.586065 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4" event={"ID":"1b330d35-9651-498e-abaa-3ae7b4831c6a","Type":"ContainerDied","Data":"75d55374b08740f13a4ccac03b03c25d24aa872ee09ddd39f28844de41821ac0"} Jul 07 16:26:17.617980 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.617947 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp" Jul 07 16:26:17.758002 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:17.757975 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp"] Jul 07 16:26:17.759436 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:26:17.759411 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54d4386c_0b67_4cac_a953_23cc4175999b.slice/crio-0f0e8207a42c1b8a8dd679c8b7dfae43972e42bc13db3646800f1e86f82b4334 WatchSource:0}: Error finding container 0f0e8207a42c1b8a8dd679c8b7dfae43972e42bc13db3646800f1e86f82b4334: Status 404 returned error can't find the container with id 0f0e8207a42c1b8a8dd679c8b7dfae43972e42bc13db3646800f1e86f82b4334 Jul 07 16:26:18.593950 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:18.593898 2564 generic.go:356] "Generic (PLEG): container finished" podID="54d4386c-0b67-4cac-a953-23cc4175999b" containerID="67890f1b6d574d1a9ad67d01c3bc05eda14f02ef7422fb300593740a6e83af76" exitCode=0 Jul 07 16:26:18.594380 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:18.594002 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp" event={"ID":"54d4386c-0b67-4cac-a953-23cc4175999b","Type":"ContainerDied","Data":"67890f1b6d574d1a9ad67d01c3bc05eda14f02ef7422fb300593740a6e83af76"} Jul 07 16:26:18.594380 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:18.594056 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp" event={"ID":"54d4386c-0b67-4cac-a953-23cc4175999b","Type":"ContainerStarted","Data":"0f0e8207a42c1b8a8dd679c8b7dfae43972e42bc13db3646800f1e86f82b4334"} Jul 07 16:26:18.596294 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:18.596269 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4" event={"ID":"1b330d35-9651-498e-abaa-3ae7b4831c6a","Type":"ContainerStarted","Data":"3cb7203db8a2800baff7ff1af0dc7da94afac9a479cc3a4ae0ade2e2d5453cf5"} Jul 07 16:26:18.599634 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:18.599545 2564 generic.go:356] "Generic (PLEG): container finished" podID="35b8f5f2-9e59-4c68-a767-a6e6d45ba798" containerID="d5eadde987bc7a9957c33c6bc3dcc50f27d8d1d90095749931297ebbfa6b408c" exitCode=0 Jul 07 16:26:18.599634 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:18.599614 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v" event={"ID":"35b8f5f2-9e59-4c68-a767-a6e6d45ba798","Type":"ContainerDied","Data":"d5eadde987bc7a9957c33c6bc3dcc50f27d8d1d90095749931297ebbfa6b408c"} Jul 07 16:26:18.602467 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:18.602445 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf" event={"ID":"372fdb21-39c4-4378-af66-8c3728384e8c","Type":"ContainerStarted","Data":"f30cc7faabd1dadd87a9a70a8a995c364de8936a389c7e3787c26f208788490e"} Jul 07 16:26:19.610150 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:19.610112 2564 generic.go:356] "Generic (PLEG): container finished" podID="35b8f5f2-9e59-4c68-a767-a6e6d45ba798" containerID="910420be7a7ec8ab9cfbf08713004dcbd1ae388dba4ec03a5c23cd2702351b92" exitCode=0 Jul 07 16:26:19.610580 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:19.610187 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v" event={"ID":"35b8f5f2-9e59-4c68-a767-a6e6d45ba798","Type":"ContainerDied","Data":"910420be7a7ec8ab9cfbf08713004dcbd1ae388dba4ec03a5c23cd2702351b92"} Jul 07 16:26:19.612167 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:19.612144 2564 generic.go:356] "Generic (PLEG): container finished" podID="372fdb21-39c4-4378-af66-8c3728384e8c" containerID="f30cc7faabd1dadd87a9a70a8a995c364de8936a389c7e3787c26f208788490e" exitCode=0 Jul 07 16:26:19.612167 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:19.612163 2564 generic.go:356] "Generic (PLEG): container finished" podID="372fdb21-39c4-4378-af66-8c3728384e8c" containerID="52ac74639c21f418e37730a488efe4351f987512b347b5bb58a42f88a9dc3765" exitCode=0 Jul 07 16:26:19.612336 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:19.612226 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf" event={"ID":"372fdb21-39c4-4378-af66-8c3728384e8c","Type":"ContainerDied","Data":"f30cc7faabd1dadd87a9a70a8a995c364de8936a389c7e3787c26f208788490e"} Jul 07 16:26:19.612336 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:19.612274 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf" event={"ID":"372fdb21-39c4-4378-af66-8c3728384e8c","Type":"ContainerDied","Data":"52ac74639c21f418e37730a488efe4351f987512b347b5bb58a42f88a9dc3765"} Jul 07 16:26:19.614033 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:19.614009 2564 generic.go:356] "Generic (PLEG): container finished" podID="54d4386c-0b67-4cac-a953-23cc4175999b" containerID="26a37d556c7211c2c3dab2fb65d9c52a10956b91440070c263c401b723461a4a" exitCode=0 Jul 07 16:26:19.614132 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:19.614091 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp" event={"ID":"54d4386c-0b67-4cac-a953-23cc4175999b","Type":"ContainerDied","Data":"26a37d556c7211c2c3dab2fb65d9c52a10956b91440070c263c401b723461a4a"} Jul 07 16:26:19.616048 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:19.616029 2564 generic.go:356] "Generic (PLEG): container finished" podID="1b330d35-9651-498e-abaa-3ae7b4831c6a" containerID="3cb7203db8a2800baff7ff1af0dc7da94afac9a479cc3a4ae0ade2e2d5453cf5" exitCode=0 Jul 07 16:26:19.616133 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:19.616074 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4" event={"ID":"1b330d35-9651-498e-abaa-3ae7b4831c6a","Type":"ContainerDied","Data":"3cb7203db8a2800baff7ff1af0dc7da94afac9a479cc3a4ae0ade2e2d5453cf5"} Jul 07 16:26:20.623494 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.623461 2564 generic.go:356] "Generic (PLEG): container finished" podID="1b330d35-9651-498e-abaa-3ae7b4831c6a" containerID="e55fa8b689979674b72c504371cd84b6607c27cb4b96a971c4452cc7f5acb6b2" exitCode=0 Jul 07 16:26:20.623925 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.623541 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4" event={"ID":"1b330d35-9651-498e-abaa-3ae7b4831c6a","Type":"ContainerDied","Data":"e55fa8b689979674b72c504371cd84b6607c27cb4b96a971c4452cc7f5acb6b2"} Jul 07 16:26:20.625502 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.625465 2564 generic.go:356] "Generic (PLEG): container finished" podID="54d4386c-0b67-4cac-a953-23cc4175999b" containerID="ece890a990e556ac967729862ac4b581826869562729997a123d5a280e5f82dd" exitCode=0 Jul 07 16:26:20.625608 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.625544 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp" event={"ID":"54d4386c-0b67-4cac-a953-23cc4175999b","Type":"ContainerDied","Data":"ece890a990e556ac967729862ac4b581826869562729997a123d5a280e5f82dd"} Jul 07 16:26:20.761756 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.761732 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v" Jul 07 16:26:20.779265 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.779241 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35b8f5f2-9e59-4c68-a767-a6e6d45ba798-bundle\") pod \"35b8f5f2-9e59-4c68-a767-a6e6d45ba798\" (UID: \"35b8f5f2-9e59-4c68-a767-a6e6d45ba798\") " Jul 07 16:26:20.779379 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.779334 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdkhl\" (UniqueName: \"kubernetes.io/projected/35b8f5f2-9e59-4c68-a767-a6e6d45ba798-kube-api-access-qdkhl\") pod \"35b8f5f2-9e59-4c68-a767-a6e6d45ba798\" (UID: \"35b8f5f2-9e59-4c68-a767-a6e6d45ba798\") " Jul 07 16:26:20.779449 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.779417 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35b8f5f2-9e59-4c68-a767-a6e6d45ba798-util\") pod \"35b8f5f2-9e59-4c68-a767-a6e6d45ba798\" (UID: \"35b8f5f2-9e59-4c68-a767-a6e6d45ba798\") " Jul 07 16:26:20.779791 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.779767 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35b8f5f2-9e59-4c68-a767-a6e6d45ba798-bundle" (OuterVolumeSpecName: "bundle") pod "35b8f5f2-9e59-4c68-a767-a6e6d45ba798" (UID: "35b8f5f2-9e59-4c68-a767-a6e6d45ba798"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:26:20.782082 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.782010 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35b8f5f2-9e59-4c68-a767-a6e6d45ba798-kube-api-access-qdkhl" (OuterVolumeSpecName: "kube-api-access-qdkhl") pod "35b8f5f2-9e59-4c68-a767-a6e6d45ba798" (UID: "35b8f5f2-9e59-4c68-a767-a6e6d45ba798"). InnerVolumeSpecName "kube-api-access-qdkhl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:26:20.782435 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.782378 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf" Jul 07 16:26:20.788624 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.788597 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35b8f5f2-9e59-4c68-a767-a6e6d45ba798-util" (OuterVolumeSpecName: "util") pod "35b8f5f2-9e59-4c68-a767-a6e6d45ba798" (UID: "35b8f5f2-9e59-4c68-a767-a6e6d45ba798"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:26:20.880613 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.880507 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/372fdb21-39c4-4378-af66-8c3728384e8c-util\") pod \"372fdb21-39c4-4378-af66-8c3728384e8c\" (UID: \"372fdb21-39c4-4378-af66-8c3728384e8c\") " Jul 07 16:26:20.880613 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.880551 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-792pm\" (UniqueName: \"kubernetes.io/projected/372fdb21-39c4-4378-af66-8c3728384e8c-kube-api-access-792pm\") pod \"372fdb21-39c4-4378-af66-8c3728384e8c\" (UID: \"372fdb21-39c4-4378-af66-8c3728384e8c\") " Jul 07 16:26:20.880613 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.880613 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/372fdb21-39c4-4378-af66-8c3728384e8c-bundle\") pod \"372fdb21-39c4-4378-af66-8c3728384e8c\" (UID: \"372fdb21-39c4-4378-af66-8c3728384e8c\") " Jul 07 16:26:20.880930 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.880827 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35b8f5f2-9e59-4c68-a767-a6e6d45ba798-bundle\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:26:20.880930 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.880839 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdkhl\" (UniqueName: \"kubernetes.io/projected/35b8f5f2-9e59-4c68-a767-a6e6d45ba798-kube-api-access-qdkhl\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:26:20.880930 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.880848 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35b8f5f2-9e59-4c68-a767-a6e6d45ba798-util\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:26:20.881224 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.881193 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/372fdb21-39c4-4378-af66-8c3728384e8c-bundle" (OuterVolumeSpecName: "bundle") pod "372fdb21-39c4-4378-af66-8c3728384e8c" (UID: "372fdb21-39c4-4378-af66-8c3728384e8c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:26:20.882742 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.882718 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/372fdb21-39c4-4378-af66-8c3728384e8c-kube-api-access-792pm" (OuterVolumeSpecName: "kube-api-access-792pm") pod "372fdb21-39c4-4378-af66-8c3728384e8c" (UID: "372fdb21-39c4-4378-af66-8c3728384e8c"). InnerVolumeSpecName "kube-api-access-792pm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:26:20.886119 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.886096 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/372fdb21-39c4-4378-af66-8c3728384e8c-util" (OuterVolumeSpecName: "util") pod "372fdb21-39c4-4378-af66-8c3728384e8c" (UID: "372fdb21-39c4-4378-af66-8c3728384e8c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:26:20.982132 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.982097 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/372fdb21-39c4-4378-af66-8c3728384e8c-util\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:26:20.982132 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.982131 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-792pm\" (UniqueName: \"kubernetes.io/projected/372fdb21-39c4-4378-af66-8c3728384e8c-kube-api-access-792pm\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:26:20.982132 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:20.982142 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/372fdb21-39c4-4378-af66-8c3728384e8c-bundle\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:26:21.633436 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.633410 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v" Jul 07 16:26:21.633861 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.633408 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v" event={"ID":"35b8f5f2-9e59-4c68-a767-a6e6d45ba798","Type":"ContainerDied","Data":"ea49a80abaf6a9a8f457268f7e13e990bdcddb74e34e9cd856a58eceda1415a0"} Jul 07 16:26:21.633861 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.633539 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ea49a80abaf6a9a8f457268f7e13e990bdcddb74e34e9cd856a58eceda1415a0" Jul 07 16:26:21.635461 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.635433 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf" Jul 07 16:26:21.635461 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.635445 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf" event={"ID":"372fdb21-39c4-4378-af66-8c3728384e8c","Type":"ContainerDied","Data":"40a2826a872210cbdff7d75081dba6b5c927fd852ba6c95f9c1bbaea0dbdefda"} Jul 07 16:26:21.635637 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.635491 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="40a2826a872210cbdff7d75081dba6b5c927fd852ba6c95f9c1bbaea0dbdefda" Jul 07 16:26:21.794857 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.794831 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4" Jul 07 16:26:21.819561 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.819527 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp" Jul 07 16:26:21.890073 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.889987 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54d4386c-0b67-4cac-a953-23cc4175999b-util\") pod \"54d4386c-0b67-4cac-a953-23cc4175999b\" (UID: \"54d4386c-0b67-4cac-a953-23cc4175999b\") " Jul 07 16:26:21.890073 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.890035 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfkth\" (UniqueName: \"kubernetes.io/projected/54d4386c-0b67-4cac-a953-23cc4175999b-kube-api-access-zfkth\") pod \"54d4386c-0b67-4cac-a953-23cc4175999b\" (UID: \"54d4386c-0b67-4cac-a953-23cc4175999b\") " Jul 07 16:26:21.890241 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.890147 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b330d35-9651-498e-abaa-3ae7b4831c6a-util\") pod \"1b330d35-9651-498e-abaa-3ae7b4831c6a\" (UID: \"1b330d35-9651-498e-abaa-3ae7b4831c6a\") " Jul 07 16:26:21.890241 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.890173 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54d4386c-0b67-4cac-a953-23cc4175999b-bundle\") pod \"54d4386c-0b67-4cac-a953-23cc4175999b\" (UID: \"54d4386c-0b67-4cac-a953-23cc4175999b\") " Jul 07 16:26:21.890241 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.890204 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b330d35-9651-498e-abaa-3ae7b4831c6a-bundle\") pod \"1b330d35-9651-498e-abaa-3ae7b4831c6a\" (UID: \"1b330d35-9651-498e-abaa-3ae7b4831c6a\") " Jul 07 16:26:21.890390 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.890246 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4nsl\" (UniqueName: \"kubernetes.io/projected/1b330d35-9651-498e-abaa-3ae7b4831c6a-kube-api-access-p4nsl\") pod \"1b330d35-9651-498e-abaa-3ae7b4831c6a\" (UID: \"1b330d35-9651-498e-abaa-3ae7b4831c6a\") " Jul 07 16:26:21.890724 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.890683 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54d4386c-0b67-4cac-a953-23cc4175999b-bundle" (OuterVolumeSpecName: "bundle") pod "54d4386c-0b67-4cac-a953-23cc4175999b" (UID: "54d4386c-0b67-4cac-a953-23cc4175999b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:26:21.890961 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.890936 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b330d35-9651-498e-abaa-3ae7b4831c6a-bundle" (OuterVolumeSpecName: "bundle") pod "1b330d35-9651-498e-abaa-3ae7b4831c6a" (UID: "1b330d35-9651-498e-abaa-3ae7b4831c6a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:26:21.892597 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.892576 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54d4386c-0b67-4cac-a953-23cc4175999b-kube-api-access-zfkth" (OuterVolumeSpecName: "kube-api-access-zfkth") pod "54d4386c-0b67-4cac-a953-23cc4175999b" (UID: "54d4386c-0b67-4cac-a953-23cc4175999b"). InnerVolumeSpecName "kube-api-access-zfkth". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:26:21.892696 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.892677 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b330d35-9651-498e-abaa-3ae7b4831c6a-kube-api-access-p4nsl" (OuterVolumeSpecName: "kube-api-access-p4nsl") pod "1b330d35-9651-498e-abaa-3ae7b4831c6a" (UID: "1b330d35-9651-498e-abaa-3ae7b4831c6a"). InnerVolumeSpecName "kube-api-access-p4nsl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:26:21.895843 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.895822 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b330d35-9651-498e-abaa-3ae7b4831c6a-util" (OuterVolumeSpecName: "util") pod "1b330d35-9651-498e-abaa-3ae7b4831c6a" (UID: "1b330d35-9651-498e-abaa-3ae7b4831c6a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:26:21.896267 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.896247 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54d4386c-0b67-4cac-a953-23cc4175999b-util" (OuterVolumeSpecName: "util") pod "54d4386c-0b67-4cac-a953-23cc4175999b" (UID: "54d4386c-0b67-4cac-a953-23cc4175999b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:26:21.991880 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.991838 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1b330d35-9651-498e-abaa-3ae7b4831c6a-util\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:26:21.991880 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.991870 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54d4386c-0b67-4cac-a953-23cc4175999b-bundle\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:26:21.991880 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.991879 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1b330d35-9651-498e-abaa-3ae7b4831c6a-bundle\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:26:21.991880 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.991888 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4nsl\" (UniqueName: \"kubernetes.io/projected/1b330d35-9651-498e-abaa-3ae7b4831c6a-kube-api-access-p4nsl\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:26:21.992159 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.991896 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54d4386c-0b67-4cac-a953-23cc4175999b-util\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:26:21.992159 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:21.991933 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfkth\" (UniqueName: \"kubernetes.io/projected/54d4386c-0b67-4cac-a953-23cc4175999b-kube-api-access-zfkth\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:26:22.642527 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:22.642497 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp" Jul 07 16:26:22.642527 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:22.642517 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp" event={"ID":"54d4386c-0b67-4cac-a953-23cc4175999b","Type":"ContainerDied","Data":"0f0e8207a42c1b8a8dd679c8b7dfae43972e42bc13db3646800f1e86f82b4334"} Jul 07 16:26:22.642986 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:22.642566 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f0e8207a42c1b8a8dd679c8b7dfae43972e42bc13db3646800f1e86f82b4334" Jul 07 16:26:22.644427 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:22.644397 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4" event={"ID":"1b330d35-9651-498e-abaa-3ae7b4831c6a","Type":"ContainerDied","Data":"ee284eceb7dcf435700ca4e315f486e1aef13595ad3b4e65e5d297bcf1a45264"} Jul 07 16:26:22.644549 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:22.644446 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee284eceb7dcf435700ca4e315f486e1aef13595ad3b4e65e5d297bcf1a45264" Jul 07 16:26:22.644549 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:22.644405 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4" Jul 07 16:26:39.622195 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622157 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hf9ll"] Jul 07 16:26:39.622712 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622597 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="372fdb21-39c4-4378-af66-8c3728384e8c" containerName="pull" Jul 07 16:26:39.622712 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622614 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="372fdb21-39c4-4378-af66-8c3728384e8c" containerName="pull" Jul 07 16:26:39.622712 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622626 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="372fdb21-39c4-4378-af66-8c3728384e8c" containerName="extract" Jul 07 16:26:39.622712 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622634 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="372fdb21-39c4-4378-af66-8c3728384e8c" containerName="extract" Jul 07 16:26:39.622712 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622649 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54d4386c-0b67-4cac-a953-23cc4175999b" containerName="util" Jul 07 16:26:39.622712 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622656 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="54d4386c-0b67-4cac-a953-23cc4175999b" containerName="util" Jul 07 16:26:39.622712 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622666 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54d4386c-0b67-4cac-a953-23cc4175999b" containerName="pull" Jul 07 16:26:39.622712 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622675 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="54d4386c-0b67-4cac-a953-23cc4175999b" containerName="pull" Jul 07 16:26:39.622712 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622689 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="372fdb21-39c4-4378-af66-8c3728384e8c" containerName="util" Jul 07 16:26:39.622712 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622697 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="372fdb21-39c4-4378-af66-8c3728384e8c" containerName="util" Jul 07 16:26:39.622712 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622709 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35b8f5f2-9e59-4c68-a767-a6e6d45ba798" containerName="util" Jul 07 16:26:39.622712 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622717 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="35b8f5f2-9e59-4c68-a767-a6e6d45ba798" containerName="util" Jul 07 16:26:39.623147 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622729 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b330d35-9651-498e-abaa-3ae7b4831c6a" containerName="pull" Jul 07 16:26:39.623147 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622738 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b330d35-9651-498e-abaa-3ae7b4831c6a" containerName="pull" Jul 07 16:26:39.623147 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622747 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54d4386c-0b67-4cac-a953-23cc4175999b" containerName="extract" Jul 07 16:26:39.623147 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622754 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="54d4386c-0b67-4cac-a953-23cc4175999b" containerName="extract" Jul 07 16:26:39.623147 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622766 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35b8f5f2-9e59-4c68-a767-a6e6d45ba798" containerName="extract" Jul 07 16:26:39.623147 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622774 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="35b8f5f2-9e59-4c68-a767-a6e6d45ba798" containerName="extract" Jul 07 16:26:39.623147 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622787 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35b8f5f2-9e59-4c68-a767-a6e6d45ba798" containerName="pull" Jul 07 16:26:39.623147 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622795 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="35b8f5f2-9e59-4c68-a767-a6e6d45ba798" containerName="pull" Jul 07 16:26:39.623147 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622803 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b330d35-9651-498e-abaa-3ae7b4831c6a" containerName="extract" Jul 07 16:26:39.623147 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622810 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b330d35-9651-498e-abaa-3ae7b4831c6a" containerName="extract" Jul 07 16:26:39.623147 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622832 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b330d35-9651-498e-abaa-3ae7b4831c6a" containerName="util" Jul 07 16:26:39.623147 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622839 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b330d35-9651-498e-abaa-3ae7b4831c6a" containerName="util" Jul 07 16:26:39.623147 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622926 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="35b8f5f2-9e59-4c68-a767-a6e6d45ba798" containerName="extract" Jul 07 16:26:39.623147 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622938 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="54d4386c-0b67-4cac-a953-23cc4175999b" containerName="extract" Jul 07 16:26:39.623147 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622947 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="372fdb21-39c4-4378-af66-8c3728384e8c" containerName="extract" Jul 07 16:26:39.623147 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.622952 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b330d35-9651-498e-abaa-3ae7b4831c6a" containerName="extract" Jul 07 16:26:39.628331 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.628298 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hf9ll" Jul 07 16:26:39.631495 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.631467 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 16:26:39.631495 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.631480 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-4x89j\"" Jul 07 16:26:39.631495 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.631476 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 07 16:26:39.632292 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.632279 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 16:26:39.645716 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.645681 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hf9ll"] Jul 07 16:26:39.765146 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.765107 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8jxd\" (UniqueName: \"kubernetes.io/projected/eb40afc6-0746-4753-bec1-7088e65076af-kube-api-access-m8jxd\") pod \"dns-operator-controller-manager-864dd78c4c-hf9ll\" (UID: \"eb40afc6-0746-4753-bec1-7088e65076af\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hf9ll" Jul 07 16:26:39.865842 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.865806 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8jxd\" (UniqueName: \"kubernetes.io/projected/eb40afc6-0746-4753-bec1-7088e65076af-kube-api-access-m8jxd\") pod \"dns-operator-controller-manager-864dd78c4c-hf9ll\" (UID: \"eb40afc6-0746-4753-bec1-7088e65076af\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hf9ll" Jul 07 16:26:39.876802 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.876732 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8jxd\" (UniqueName: \"kubernetes.io/projected/eb40afc6-0746-4753-bec1-7088e65076af-kube-api-access-m8jxd\") pod \"dns-operator-controller-manager-864dd78c4c-hf9ll\" (UID: \"eb40afc6-0746-4753-bec1-7088e65076af\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hf9ll" Jul 07 16:26:39.939844 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:39.939811 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hf9ll" Jul 07 16:26:40.085832 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:40.085806 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-hf9ll"] Jul 07 16:26:40.087898 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:26:40.087859 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb40afc6_0746_4753_bec1_7088e65076af.slice/crio-7f39f534fff3eb51017f35181ad722ee562a588b524513a6d1b3ad616e068489 WatchSource:0}: Error finding container 7f39f534fff3eb51017f35181ad722ee562a588b524513a6d1b3ad616e068489: Status 404 returned error can't find the container with id 7f39f534fff3eb51017f35181ad722ee562a588b524513a6d1b3ad616e068489 Jul 07 16:26:40.741103 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:40.741072 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hf9ll" event={"ID":"eb40afc6-0746-4753-bec1-7088e65076af","Type":"ContainerStarted","Data":"7f39f534fff3eb51017f35181ad722ee562a588b524513a6d1b3ad616e068489"} Jul 07 16:26:42.754253 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:42.754219 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hf9ll" event={"ID":"eb40afc6-0746-4753-bec1-7088e65076af","Type":"ContainerStarted","Data":"442767e4369185e3ac7ef511e05cff9e44bde8407b800197eb85a85e800d3aff"} Jul 07 16:26:42.754644 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:42.754318 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hf9ll" Jul 07 16:26:42.771742 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:42.771687 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hf9ll" podStartSLOduration=1.700105668 podStartE2EDuration="3.771670662s" totalImagesPullingTime="2.071564994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:26:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:26:42.770016375 +0000 UTC m=+510.931829330" watchObservedRunningTime="2026-07-07 16:26:42.771670662 +0000 UTC m=+510.933483616" Jul 07 16:26:44.656559 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:44.656518 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj"] Jul 07 16:26:44.660233 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:44.660211 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj" Jul 07 16:26:44.663003 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:44.662979 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kbc57\"" Jul 07 16:26:44.671667 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:44.671639 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj"] Jul 07 16:26:44.808418 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:44.808368 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b142b12-ef91-43e7-b1fe-07706b2a092c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mhcrj\" (UID: \"9b142b12-ef91-43e7-b1fe-07706b2a092c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj" Jul 07 16:26:44.808610 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:44.808475 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z2ck\" (UniqueName: \"kubernetes.io/projected/9b142b12-ef91-43e7-b1fe-07706b2a092c-kube-api-access-4z2ck\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mhcrj\" (UID: \"9b142b12-ef91-43e7-b1fe-07706b2a092c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj" Jul 07 16:26:44.909440 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:44.909339 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b142b12-ef91-43e7-b1fe-07706b2a092c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mhcrj\" (UID: \"9b142b12-ef91-43e7-b1fe-07706b2a092c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj" Jul 07 16:26:44.909440 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:44.909403 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z2ck\" (UniqueName: \"kubernetes.io/projected/9b142b12-ef91-43e7-b1fe-07706b2a092c-kube-api-access-4z2ck\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mhcrj\" (UID: \"9b142b12-ef91-43e7-b1fe-07706b2a092c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj" Jul 07 16:26:44.909816 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:44.909791 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b142b12-ef91-43e7-b1fe-07706b2a092c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mhcrj\" (UID: \"9b142b12-ef91-43e7-b1fe-07706b2a092c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj" Jul 07 16:26:44.919389 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:44.919357 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z2ck\" (UniqueName: \"kubernetes.io/projected/9b142b12-ef91-43e7-b1fe-07706b2a092c-kube-api-access-4z2ck\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mhcrj\" (UID: \"9b142b12-ef91-43e7-b1fe-07706b2a092c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj" Jul 07 16:26:44.971021 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:44.970983 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj" Jul 07 16:26:45.108859 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:45.108835 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj"] Jul 07 16:26:45.110814 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:26:45.110774 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b142b12_ef91_43e7_b1fe_07706b2a092c.slice/crio-4cc3ccadf3772f48e3b0bcbf354f1af3f5267fffce49b77b063bab5772c35612 WatchSource:0}: Error finding container 4cc3ccadf3772f48e3b0bcbf354f1af3f5267fffce49b77b063bab5772c35612: Status 404 returned error can't find the container with id 4cc3ccadf3772f48e3b0bcbf354f1af3f5267fffce49b77b063bab5772c35612 Jul 07 16:26:45.772743 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:45.772706 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj" event={"ID":"9b142b12-ef91-43e7-b1fe-07706b2a092c","Type":"ContainerStarted","Data":"4cc3ccadf3772f48e3b0bcbf354f1af3f5267fffce49b77b063bab5772c35612"} Jul 07 16:26:51.808989 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:51.808951 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj" event={"ID":"9b142b12-ef91-43e7-b1fe-07706b2a092c","Type":"ContainerStarted","Data":"5a69b09a0e3bf4e5590e2792084eeeff0daa65ac87d6590f2465d7390291fb04"} Jul 07 16:26:51.809405 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:51.809053 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj" Jul 07 16:26:51.830711 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:51.830657 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj" podStartSLOduration=1.6506187209999998 podStartE2EDuration="7.830640623s" totalImagesPullingTime="6.180021902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:26:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:26:51.828512969 +0000 UTC m=+519.990325942" watchObservedRunningTime="2026-07-07 16:26:51.830640623 +0000 UTC m=+519.992453578" Jul 07 16:26:53.762015 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:26:53.761985 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-hf9ll" Jul 07 16:27:02.816485 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:02.816454 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj" Jul 07 16:27:04.643902 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.643816 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj"] Jul 07 16:27:04.644290 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.644039 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj" podUID="9b142b12-ef91-43e7-b1fe-07706b2a092c" containerName="manager" containerID="cri-o://5a69b09a0e3bf4e5590e2792084eeeff0daa65ac87d6590f2465d7390291fb04" gracePeriod=2 Jul 07 16:27:04.659721 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.659676 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj"] Jul 07 16:27:04.662515 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.662485 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl"] Jul 07 16:27:04.663048 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.663032 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b142b12-ef91-43e7-b1fe-07706b2a092c" containerName="manager" Jul 07 16:27:04.663107 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.663052 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b142b12-ef91-43e7-b1fe-07706b2a092c" containerName="manager" Jul 07 16:27:04.663175 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.663165 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b142b12-ef91-43e7-b1fe-07706b2a092c" containerName="manager" Jul 07 16:27:04.666602 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.666574 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl" Jul 07 16:27:04.678625 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:27:04.678579 2564 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"9b142b12-ef91-43e7-b1fe-07706b2a092c\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-mhcrj\": pods \"kuadrant-operator-controller-manager-8684c9ff49-mhcrj\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj" Jul 07 16:27:04.680962 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:27:04.680925 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mhcrj\" is forbidden: User \"system:node:ip-10-0-129-174.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-174.ec2.internal' and this object" podUID="9b142b12-ef91-43e7-b1fe-07706b2a092c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj" Jul 07 16:27:04.681734 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.681708 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9"] Jul 07 16:27:04.686664 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.686641 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl"] Jul 07 16:27:04.686776 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.686761 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9" Jul 07 16:27:04.700710 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.700681 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9"] Jul 07 16:27:04.714748 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:27:04.714707 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mhcrj\" is forbidden: User \"system:node:ip-10-0-129-174.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-174.ec2.internal' and this object" podUID="9b142b12-ef91-43e7-b1fe-07706b2a092c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj" Jul 07 16:27:04.772380 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.772336 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b81b71f-98ff-4608-bb2d-4ea6f81cb66c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2jnn9\" (UID: \"7b81b71f-98ff-4608-bb2d-4ea6f81cb66c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9" Jul 07 16:27:04.772531 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.772507 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a4d7f2c-b89f-4c21-906f-b81bb7517d24-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lvwrl\" (UID: \"0a4d7f2c-b89f-4c21-906f-b81bb7517d24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl" Jul 07 16:27:04.772593 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.772553 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqfht\" (UniqueName: \"kubernetes.io/projected/7b81b71f-98ff-4608-bb2d-4ea6f81cb66c-kube-api-access-hqfht\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2jnn9\" (UID: \"7b81b71f-98ff-4608-bb2d-4ea6f81cb66c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9" Jul 07 16:27:04.772593 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.772571 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8nbp\" (UniqueName: \"kubernetes.io/projected/0a4d7f2c-b89f-4c21-906f-b81bb7517d24-kube-api-access-h8nbp\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lvwrl\" (UID: \"0a4d7f2c-b89f-4c21-906f-b81bb7517d24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl" Jul 07 16:27:04.873578 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.873548 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a4d7f2c-b89f-4c21-906f-b81bb7517d24-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lvwrl\" (UID: \"0a4d7f2c-b89f-4c21-906f-b81bb7517d24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl" Jul 07 16:27:04.873732 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.873588 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqfht\" (UniqueName: \"kubernetes.io/projected/7b81b71f-98ff-4608-bb2d-4ea6f81cb66c-kube-api-access-hqfht\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2jnn9\" (UID: \"7b81b71f-98ff-4608-bb2d-4ea6f81cb66c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9" Jul 07 16:27:04.873732 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.873605 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8nbp\" (UniqueName: \"kubernetes.io/projected/0a4d7f2c-b89f-4c21-906f-b81bb7517d24-kube-api-access-h8nbp\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lvwrl\" (UID: \"0a4d7f2c-b89f-4c21-906f-b81bb7517d24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl" Jul 07 16:27:04.873732 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.873648 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b81b71f-98ff-4608-bb2d-4ea6f81cb66c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2jnn9\" (UID: \"7b81b71f-98ff-4608-bb2d-4ea6f81cb66c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9" Jul 07 16:27:04.874028 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.874006 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b81b71f-98ff-4608-bb2d-4ea6f81cb66c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2jnn9\" (UID: \"7b81b71f-98ff-4608-bb2d-4ea6f81cb66c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9" Jul 07 16:27:04.874120 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.874012 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a4d7f2c-b89f-4c21-906f-b81bb7517d24-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lvwrl\" (UID: \"0a4d7f2c-b89f-4c21-906f-b81bb7517d24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl" Jul 07 16:27:04.886271 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.886243 2564 generic.go:356] "Generic (PLEG): container finished" podID="9b142b12-ef91-43e7-b1fe-07706b2a092c" containerID="5a69b09a0e3bf4e5590e2792084eeeff0daa65ac87d6590f2465d7390291fb04" exitCode=0 Jul 07 16:27:04.886432 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.886283 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4cc3ccadf3772f48e3b0bcbf354f1af3f5267fffce49b77b063bab5772c35612" Jul 07 16:27:04.891841 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.891808 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqfht\" (UniqueName: \"kubernetes.io/projected/7b81b71f-98ff-4608-bb2d-4ea6f81cb66c-kube-api-access-hqfht\") pod \"kuadrant-operator-controller-manager-8684c9ff49-2jnn9\" (UID: \"7b81b71f-98ff-4608-bb2d-4ea6f81cb66c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9" Jul 07 16:27:04.892479 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.892459 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8nbp\" (UniqueName: \"kubernetes.io/projected/0a4d7f2c-b89f-4c21-906f-b81bb7517d24-kube-api-access-h8nbp\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-lvwrl\" (UID: \"0a4d7f2c-b89f-4c21-906f-b81bb7517d24\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl" Jul 07 16:27:04.894311 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.894254 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj" Jul 07 16:27:04.896548 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:27:04.896514 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mhcrj\" is forbidden: User \"system:node:ip-10-0-129-174.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-174.ec2.internal' and this object" podUID="9b142b12-ef91-43e7-b1fe-07706b2a092c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj" Jul 07 16:27:04.974027 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.973985 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4z2ck\" (UniqueName: \"kubernetes.io/projected/9b142b12-ef91-43e7-b1fe-07706b2a092c-kube-api-access-4z2ck\") pod \"9b142b12-ef91-43e7-b1fe-07706b2a092c\" (UID: \"9b142b12-ef91-43e7-b1fe-07706b2a092c\") " Jul 07 16:27:04.974239 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.974076 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b142b12-ef91-43e7-b1fe-07706b2a092c-extensions-socket-volume\") pod \"9b142b12-ef91-43e7-b1fe-07706b2a092c\" (UID: \"9b142b12-ef91-43e7-b1fe-07706b2a092c\") " Jul 07 16:27:04.974606 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.974572 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b142b12-ef91-43e7-b1fe-07706b2a092c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9b142b12-ef91-43e7-b1fe-07706b2a092c" (UID: "9b142b12-ef91-43e7-b1fe-07706b2a092c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:27:04.976046 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:04.976024 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b142b12-ef91-43e7-b1fe-07706b2a092c-kube-api-access-4z2ck" (OuterVolumeSpecName: "kube-api-access-4z2ck") pod "9b142b12-ef91-43e7-b1fe-07706b2a092c" (UID: "9b142b12-ef91-43e7-b1fe-07706b2a092c"). InnerVolumeSpecName "kube-api-access-4z2ck". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:27:05.033789 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:05.033750 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl" Jul 07 16:27:05.042224 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:05.042201 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9" Jul 07 16:27:05.075593 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:05.075561 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4z2ck\" (UniqueName: \"kubernetes.io/projected/9b142b12-ef91-43e7-b1fe-07706b2a092c-kube-api-access-4z2ck\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:27:05.075593 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:05.075591 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9b142b12-ef91-43e7-b1fe-07706b2a092c-extensions-socket-volume\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:27:05.193976 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:05.193703 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl"] Jul 07 16:27:05.227297 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:05.227270 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9"] Jul 07 16:27:05.239154 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:27:05.239124 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b81b71f_98ff_4608_bb2d_4ea6f81cb66c.slice/crio-8f0b9c7923905399e7777ae48c2e24c10517ddec7d93046838cba7534e7b1a05 WatchSource:0}: Error finding container 8f0b9c7923905399e7777ae48c2e24c10517ddec7d93046838cba7534e7b1a05: Status 404 returned error can't find the container with id 8f0b9c7923905399e7777ae48c2e24c10517ddec7d93046838cba7534e7b1a05 Jul 07 16:27:05.895285 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:05.895247 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9" event={"ID":"7b81b71f-98ff-4608-bb2d-4ea6f81cb66c","Type":"ContainerStarted","Data":"f4ca739256c6defa5661e2a0c66f255df1856111a7f2819153b5dd1d0c50c274"} Jul 07 16:27:05.895764 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:05.895716 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9" event={"ID":"7b81b71f-98ff-4608-bb2d-4ea6f81cb66c","Type":"ContainerStarted","Data":"8f0b9c7923905399e7777ae48c2e24c10517ddec7d93046838cba7534e7b1a05"} Jul 07 16:27:05.895764 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:05.895756 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9" Jul 07 16:27:05.897383 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:05.897356 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl" event={"ID":"0a4d7f2c-b89f-4c21-906f-b81bb7517d24","Type":"ContainerStarted","Data":"adba6413bd61da5a63ce7b26f8adfec1f573c124ac8326df944a998b0bf98af0"} Jul 07 16:27:05.897383 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:05.897378 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj" Jul 07 16:27:05.897560 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:05.897408 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl" event={"ID":"0a4d7f2c-b89f-4c21-906f-b81bb7517d24","Type":"ContainerStarted","Data":"bee49b9ee3859adeb92dfbee6b02f73cdae09a75f990bfbf4088b5fefd6479e8"} Jul 07 16:27:05.897602 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:05.897562 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl" Jul 07 16:27:05.897988 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:27:05.897963 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mhcrj\" is forbidden: User \"system:node:ip-10-0-129-174.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-174.ec2.internal' and this object" podUID="9b142b12-ef91-43e7-b1fe-07706b2a092c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj" Jul 07 16:27:05.933365 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:05.933235 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9" podStartSLOduration=1.933216554 podStartE2EDuration="1.933216554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:27:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:27:05.93179309 +0000 UTC m=+534.093606046" watchObservedRunningTime="2026-07-07 16:27:05.933216554 +0000 UTC m=+534.095029509" Jul 07 16:27:05.934762 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:27:05.934728 2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mhcrj\" is forbidden: User \"system:node:ip-10-0-129-174.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-174.ec2.internal' and this object" podUID="9b142b12-ef91-43e7-b1fe-07706b2a092c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mhcrj" Jul 07 16:27:05.962829 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:05.962776 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl" podStartSLOduration=1.962759341 podStartE2EDuration="1.962759341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:27:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:27:05.961519036 +0000 UTC m=+534.123331993" watchObservedRunningTime="2026-07-07 16:27:05.962759341 +0000 UTC m=+534.124572296" Jul 07 16:27:06.349940 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:06.349840 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b142b12-ef91-43e7-b1fe-07706b2a092c" path="/var/lib/kubelet/pods/9b142b12-ef91-43e7-b1fe-07706b2a092c/volumes" Jul 07 16:27:16.906055 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:16.906023 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl" Jul 07 16:27:16.962074 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:16.962045 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9"] Jul 07 16:27:16.962280 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:16.962236 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9" podUID="7b81b71f-98ff-4608-bb2d-4ea6f81cb66c" containerName="manager" containerID="cri-o://f4ca739256c6defa5661e2a0c66f255df1856111a7f2819153b5dd1d0c50c274" gracePeriod=10 Jul 07 16:27:16.964236 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:16.964208 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9" Jul 07 16:27:17.215184 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.215146 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f"] Jul 07 16:27:17.219488 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.219467 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f" Jul 07 16:27:17.235758 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.235726 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f"] Jul 07 16:27:17.242453 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.242423 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9" Jul 07 16:27:17.282733 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.282699 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqfht\" (UniqueName: \"kubernetes.io/projected/7b81b71f-98ff-4608-bb2d-4ea6f81cb66c-kube-api-access-hqfht\") pod \"7b81b71f-98ff-4608-bb2d-4ea6f81cb66c\" (UID: \"7b81b71f-98ff-4608-bb2d-4ea6f81cb66c\") " Jul 07 16:27:17.282955 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.282799 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b81b71f-98ff-4608-bb2d-4ea6f81cb66c-extensions-socket-volume\") pod \"7b81b71f-98ff-4608-bb2d-4ea6f81cb66c\" (UID: \"7b81b71f-98ff-4608-bb2d-4ea6f81cb66c\") " Jul 07 16:27:17.282955 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.282899 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fec340b-6734-419c-9813-8a84ef5481b6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9d97f\" (UID: \"8fec340b-6734-419c-9813-8a84ef5481b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f" Jul 07 16:27:17.283079 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.283038 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cj6k\" (UniqueName: \"kubernetes.io/projected/8fec340b-6734-419c-9813-8a84ef5481b6-kube-api-access-5cj6k\") pod \"kuadrant-operator-controller-manager-86878c769f-9d97f\" (UID: \"8fec340b-6734-419c-9813-8a84ef5481b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f" Jul 07 16:27:17.283233 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.283207 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b81b71f-98ff-4608-bb2d-4ea6f81cb66c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7b81b71f-98ff-4608-bb2d-4ea6f81cb66c" (UID: "7b81b71f-98ff-4608-bb2d-4ea6f81cb66c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:27:17.285023 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.284992 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b81b71f-98ff-4608-bb2d-4ea6f81cb66c-kube-api-access-hqfht" (OuterVolumeSpecName: "kube-api-access-hqfht") pod "7b81b71f-98ff-4608-bb2d-4ea6f81cb66c" (UID: "7b81b71f-98ff-4608-bb2d-4ea6f81cb66c"). InnerVolumeSpecName "kube-api-access-hqfht". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:27:17.384287 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.384243 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cj6k\" (UniqueName: \"kubernetes.io/projected/8fec340b-6734-419c-9813-8a84ef5481b6-kube-api-access-5cj6k\") pod \"kuadrant-operator-controller-manager-86878c769f-9d97f\" (UID: \"8fec340b-6734-419c-9813-8a84ef5481b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f" Jul 07 16:27:17.384480 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.384336 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fec340b-6734-419c-9813-8a84ef5481b6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9d97f\" (UID: \"8fec340b-6734-419c-9813-8a84ef5481b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f" Jul 07 16:27:17.384540 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.384522 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b81b71f-98ff-4608-bb2d-4ea6f81cb66c-extensions-socket-volume\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:27:17.384629 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.384610 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqfht\" (UniqueName: \"kubernetes.io/projected/7b81b71f-98ff-4608-bb2d-4ea6f81cb66c-kube-api-access-hqfht\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:27:17.384795 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.384777 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fec340b-6734-419c-9813-8a84ef5481b6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9d97f\" (UID: \"8fec340b-6734-419c-9813-8a84ef5481b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f" Jul 07 16:27:17.399488 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.399459 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cj6k\" (UniqueName: \"kubernetes.io/projected/8fec340b-6734-419c-9813-8a84ef5481b6-kube-api-access-5cj6k\") pod \"kuadrant-operator-controller-manager-86878c769f-9d97f\" (UID: \"8fec340b-6734-419c-9813-8a84ef5481b6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f" Jul 07 16:27:17.547375 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.547281 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f" Jul 07 16:27:17.696291 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.696265 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f"] Jul 07 16:27:17.698969 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:27:17.698942 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fec340b_6734_419c_9813_8a84ef5481b6.slice/crio-63ea70a530ec5f38df91ccd41e0e20c280633735d95aa2a96d26e3ea147fba69 WatchSource:0}: Error finding container 63ea70a530ec5f38df91ccd41e0e20c280633735d95aa2a96d26e3ea147fba69: Status 404 returned error can't find the container with id 63ea70a530ec5f38df91ccd41e0e20c280633735d95aa2a96d26e3ea147fba69 Jul 07 16:27:17.964269 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.964240 2564 generic.go:356] "Generic (PLEG): container finished" podID="7b81b71f-98ff-4608-bb2d-4ea6f81cb66c" containerID="f4ca739256c6defa5661e2a0c66f255df1856111a7f2819153b5dd1d0c50c274" exitCode=0 Jul 07 16:27:17.964725 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.964288 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9" Jul 07 16:27:17.964725 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.964335 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9" event={"ID":"7b81b71f-98ff-4608-bb2d-4ea6f81cb66c","Type":"ContainerDied","Data":"f4ca739256c6defa5661e2a0c66f255df1856111a7f2819153b5dd1d0c50c274"} Jul 07 16:27:17.964725 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.964383 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9" event={"ID":"7b81b71f-98ff-4608-bb2d-4ea6f81cb66c","Type":"ContainerDied","Data":"8f0b9c7923905399e7777ae48c2e24c10517ddec7d93046838cba7534e7b1a05"} Jul 07 16:27:17.964725 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.964410 2564 scope.go:117] "RemoveContainer" containerID="f4ca739256c6defa5661e2a0c66f255df1856111a7f2819153b5dd1d0c50c274" Jul 07 16:27:17.966231 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.966212 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f" event={"ID":"8fec340b-6734-419c-9813-8a84ef5481b6","Type":"ContainerStarted","Data":"1047bce0c40670e71573b61240aeea433ee2c9c9dad47366b1262aeff76951c0"} Jul 07 16:27:17.966329 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.966253 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f" event={"ID":"8fec340b-6734-419c-9813-8a84ef5481b6","Type":"ContainerStarted","Data":"63ea70a530ec5f38df91ccd41e0e20c280633735d95aa2a96d26e3ea147fba69"} Jul 07 16:27:17.966329 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.966305 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f" Jul 07 16:27:17.974557 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.974527 2564 scope.go:117] "RemoveContainer" containerID="f4ca739256c6defa5661e2a0c66f255df1856111a7f2819153b5dd1d0c50c274" Jul 07 16:27:17.974853 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:27:17.974835 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4ca739256c6defa5661e2a0c66f255df1856111a7f2819153b5dd1d0c50c274\": container with ID starting with f4ca739256c6defa5661e2a0c66f255df1856111a7f2819153b5dd1d0c50c274 not found: ID does not exist" containerID="f4ca739256c6defa5661e2a0c66f255df1856111a7f2819153b5dd1d0c50c274" Jul 07 16:27:17.974963 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.974860 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4ca739256c6defa5661e2a0c66f255df1856111a7f2819153b5dd1d0c50c274"} err="failed to get container status \"f4ca739256c6defa5661e2a0c66f255df1856111a7f2819153b5dd1d0c50c274\": rpc error: code = NotFound desc = could not find container \"f4ca739256c6defa5661e2a0c66f255df1856111a7f2819153b5dd1d0c50c274\": container with ID starting with f4ca739256c6defa5661e2a0c66f255df1856111a7f2819153b5dd1d0c50c274 not found: ID does not exist" Jul 07 16:27:17.991433 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:17.991381 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f" podStartSLOduration=0.991364722 podStartE2EDuration="991.364722ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:27:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:27:17.989035419 +0000 UTC m=+546.150848392" watchObservedRunningTime="2026-07-07 16:27:17.991364722 +0000 UTC m=+546.153177676" Jul 07 16:27:18.006748 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:18.006710 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9"] Jul 07 16:27:18.012089 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:18.012052 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-2jnn9"] Jul 07 16:27:18.350067 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:18.349992 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b81b71f-98ff-4608-bb2d-4ea6f81cb66c" path="/var/lib/kubelet/pods/7b81b71f-98ff-4608-bb2d-4ea6f81cb66c/volumes" Jul 07 16:27:28.975430 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:28.975400 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f" Jul 07 16:27:29.036994 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:29.036957 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl"] Jul 07 16:27:29.037208 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:29.037182 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl" podUID="0a4d7f2c-b89f-4c21-906f-b81bb7517d24" containerName="manager" containerID="cri-o://adba6413bd61da5a63ce7b26f8adfec1f573c124ac8326df944a998b0bf98af0" gracePeriod=10 Jul 07 16:27:29.288249 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:29.288225 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl" Jul 07 16:27:29.395244 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:29.395203 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a4d7f2c-b89f-4c21-906f-b81bb7517d24-extensions-socket-volume\") pod \"0a4d7f2c-b89f-4c21-906f-b81bb7517d24\" (UID: \"0a4d7f2c-b89f-4c21-906f-b81bb7517d24\") " Jul 07 16:27:29.395426 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:29.395283 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8nbp\" (UniqueName: \"kubernetes.io/projected/0a4d7f2c-b89f-4c21-906f-b81bb7517d24-kube-api-access-h8nbp\") pod \"0a4d7f2c-b89f-4c21-906f-b81bb7517d24\" (UID: \"0a4d7f2c-b89f-4c21-906f-b81bb7517d24\") " Jul 07 16:27:29.395690 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:29.395664 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a4d7f2c-b89f-4c21-906f-b81bb7517d24-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0a4d7f2c-b89f-4c21-906f-b81bb7517d24" (UID: "0a4d7f2c-b89f-4c21-906f-b81bb7517d24"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:27:29.397446 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:29.397425 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a4d7f2c-b89f-4c21-906f-b81bb7517d24-kube-api-access-h8nbp" (OuterVolumeSpecName: "kube-api-access-h8nbp") pod "0a4d7f2c-b89f-4c21-906f-b81bb7517d24" (UID: "0a4d7f2c-b89f-4c21-906f-b81bb7517d24"). InnerVolumeSpecName "kube-api-access-h8nbp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:27:29.496894 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:29.496850 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a4d7f2c-b89f-4c21-906f-b81bb7517d24-extensions-socket-volume\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:27:29.496894 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:29.496885 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8nbp\" (UniqueName: \"kubernetes.io/projected/0a4d7f2c-b89f-4c21-906f-b81bb7517d24-kube-api-access-h8nbp\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:27:30.035999 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:30.035960 2564 generic.go:356] "Generic (PLEG): container finished" podID="0a4d7f2c-b89f-4c21-906f-b81bb7517d24" containerID="adba6413bd61da5a63ce7b26f8adfec1f573c124ac8326df944a998b0bf98af0" exitCode=0 Jul 07 16:27:30.036491 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:30.036011 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl" Jul 07 16:27:30.036491 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:30.036048 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl" event={"ID":"0a4d7f2c-b89f-4c21-906f-b81bb7517d24","Type":"ContainerDied","Data":"adba6413bd61da5a63ce7b26f8adfec1f573c124ac8326df944a998b0bf98af0"} Jul 07 16:27:30.036491 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:30.036106 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl" event={"ID":"0a4d7f2c-b89f-4c21-906f-b81bb7517d24","Type":"ContainerDied","Data":"bee49b9ee3859adeb92dfbee6b02f73cdae09a75f990bfbf4088b5fefd6479e8"} Jul 07 16:27:30.036491 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:30.036130 2564 scope.go:117] "RemoveContainer" containerID="adba6413bd61da5a63ce7b26f8adfec1f573c124ac8326df944a998b0bf98af0" Jul 07 16:27:30.045815 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:30.045618 2564 scope.go:117] "RemoveContainer" containerID="adba6413bd61da5a63ce7b26f8adfec1f573c124ac8326df944a998b0bf98af0" Jul 07 16:27:30.045976 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:27:30.045954 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adba6413bd61da5a63ce7b26f8adfec1f573c124ac8326df944a998b0bf98af0\": container with ID starting with adba6413bd61da5a63ce7b26f8adfec1f573c124ac8326df944a998b0bf98af0 not found: ID does not exist" containerID="adba6413bd61da5a63ce7b26f8adfec1f573c124ac8326df944a998b0bf98af0" Jul 07 16:27:30.046039 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:30.045988 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adba6413bd61da5a63ce7b26f8adfec1f573c124ac8326df944a998b0bf98af0"} err="failed to get container status \"adba6413bd61da5a63ce7b26f8adfec1f573c124ac8326df944a998b0bf98af0\": rpc error: code = NotFound desc = could not find container \"adba6413bd61da5a63ce7b26f8adfec1f573c124ac8326df944a998b0bf98af0\": container with ID starting with adba6413bd61da5a63ce7b26f8adfec1f573c124ac8326df944a998b0bf98af0 not found: ID does not exist" Jul 07 16:27:30.062947 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:30.062918 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl"] Jul 07 16:27:30.071293 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:30.071264 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-lvwrl"] Jul 07 16:27:30.350347 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:30.350269 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a4d7f2c-b89f-4c21-906f-b81bb7517d24" path="/var/lib/kubelet/pods/0a4d7f2c-b89f-4c21-906f-b81bb7517d24/volumes" Jul 07 16:27:33.302110 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.302069 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp"] Jul 07 16:27:33.302530 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.302513 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b81b71f-98ff-4608-bb2d-4ea6f81cb66c" containerName="manager" Jul 07 16:27:33.302580 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.302533 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b81b71f-98ff-4608-bb2d-4ea6f81cb66c" containerName="manager" Jul 07 16:27:33.302580 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.302552 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a4d7f2c-b89f-4c21-906f-b81bb7517d24" containerName="manager" Jul 07 16:27:33.302580 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.302557 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a4d7f2c-b89f-4c21-906f-b81bb7517d24" containerName="manager" Jul 07 16:27:33.302672 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.302623 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a4d7f2c-b89f-4c21-906f-b81bb7517d24" containerName="manager" Jul 07 16:27:33.302672 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.302630 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b81b71f-98ff-4608-bb2d-4ea6f81cb66c" containerName="manager" Jul 07 16:27:33.305286 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.305253 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.307887 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.307868 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-gcw8r\"" Jul 07 16:27:33.318731 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.318705 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp"] Jul 07 16:27:33.433209 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.433166 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.433359 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.433218 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.433359 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.433295 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.433359 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.433351 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.433525 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.433373 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.433525 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.433388 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.433525 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.433419 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.433525 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.433481 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4m79\" (UniqueName: \"kubernetes.io/projected/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-kube-api-access-m4m79\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.433694 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.433584 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.534209 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.534170 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.534209 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.534214 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.534469 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.534305 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.534469 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.534354 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.534469 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.534373 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4m79\" (UniqueName: \"kubernetes.io/projected/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-kube-api-access-m4m79\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.534469 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.534423 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.534469 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.534459 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.534821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.534513 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.534821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.534557 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.534821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.534676 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.534821 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.534800 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.535009 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.534851 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.535009 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.534989 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.535084 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.535019 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.536649 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.536632 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.536772 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.536753 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.543724 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.543698 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.543953 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.543934 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4m79\" (UniqueName: \"kubernetes.io/projected/d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f-kube-api-access-m4m79\") pod \"maas-default-gateway-openshift-default-56654dcf7-9rftp\" (UID: \"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.615624 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.615594 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:33.761861 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.761828 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp"] Jul 07 16:27:33.763291 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:27:33.763260 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd85ae2f7_2c71_4c67_820d_6fcdd23b9d6f.slice/crio-e374df9820969b576b99f77aeab815a39aa519f6471e9c82c9ec8455ef2ef311 WatchSource:0}: Error finding container e374df9820969b576b99f77aeab815a39aa519f6471e9c82c9ec8455ef2ef311: Status 404 returned error can't find the container with id e374df9820969b576b99f77aeab815a39aa519f6471e9c82c9ec8455ef2ef311 Jul 07 16:27:33.765398 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.765358 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:27:33.765492 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.765429 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:27:33.765492 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:33.765456 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:27:34.061150 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:34.061057 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" event={"ID":"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f","Type":"ContainerStarted","Data":"c6789ceea618969f903f59beffceb57b69812667dce5c0e023a3f362c6310949"} Jul 07 16:27:34.061150 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:34.061114 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" event={"ID":"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f","Type":"ContainerStarted","Data":"e374df9820969b576b99f77aeab815a39aa519f6471e9c82c9ec8455ef2ef311"} Jul 07 16:27:34.085692 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:34.085636 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" podStartSLOduration=1.08561901 podStartE2EDuration="1.08561901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:27:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:27:34.084955717 +0000 UTC m=+562.246768672" watchObservedRunningTime="2026-07-07 16:27:34.08561901 +0000 UTC m=+562.247431964" Jul 07 16:27:34.616562 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:34.616521 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:34.621790 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:34.621760 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:35.067794 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:35.067702 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:35.068652 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:35.068636 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:27:48.943308 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:48.943273 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:27:48.947013 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:48.946991 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-4dwvn" Jul 07 16:27:48.949538 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:48.949501 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 16:27:48.949674 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:48.949655 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-468st\"" Jul 07 16:27:48.959690 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:48.959666 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:27:48.986959 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:48.986920 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:27:49.069763 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:49.069722 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9f66\" (UniqueName: \"kubernetes.io/projected/d50601fa-a47c-4404-8e35-871b6a2e593d-kube-api-access-f9f66\") pod \"limitador-limitador-54d6d988d5-4dwvn\" (UID: \"d50601fa-a47c-4404-8e35-871b6a2e593d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4dwvn" Jul 07 16:27:49.069984 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:49.069774 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d50601fa-a47c-4404-8e35-871b6a2e593d-config-file\") pod \"limitador-limitador-54d6d988d5-4dwvn\" (UID: \"d50601fa-a47c-4404-8e35-871b6a2e593d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4dwvn" Jul 07 16:27:49.170469 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:49.170432 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9f66\" (UniqueName: \"kubernetes.io/projected/d50601fa-a47c-4404-8e35-871b6a2e593d-kube-api-access-f9f66\") pod \"limitador-limitador-54d6d988d5-4dwvn\" (UID: \"d50601fa-a47c-4404-8e35-871b6a2e593d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4dwvn" Jul 07 16:27:49.170663 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:49.170486 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d50601fa-a47c-4404-8e35-871b6a2e593d-config-file\") pod \"limitador-limitador-54d6d988d5-4dwvn\" (UID: \"d50601fa-a47c-4404-8e35-871b6a2e593d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4dwvn" Jul 07 16:27:49.171127 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:49.171109 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d50601fa-a47c-4404-8e35-871b6a2e593d-config-file\") pod \"limitador-limitador-54d6d988d5-4dwvn\" (UID: \"d50601fa-a47c-4404-8e35-871b6a2e593d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4dwvn" Jul 07 16:27:49.178566 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:49.178538 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9f66\" (UniqueName: \"kubernetes.io/projected/d50601fa-a47c-4404-8e35-871b6a2e593d-kube-api-access-f9f66\") pod \"limitador-limitador-54d6d988d5-4dwvn\" (UID: \"d50601fa-a47c-4404-8e35-871b6a2e593d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-4dwvn" Jul 07 16:27:49.258707 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:49.258608 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-4dwvn" Jul 07 16:27:49.400697 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:49.400665 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:27:49.402318 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:27:49.402289 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd50601fa_a47c_4404_8e35_871b6a2e593d.slice/crio-94cc0b38e1622139902aeeaefc2a4649ae6afa7023ba8f2ff327ef4f4d131389 WatchSource:0}: Error finding container 94cc0b38e1622139902aeeaefc2a4649ae6afa7023ba8f2ff327ef4f4d131389: Status 404 returned error can't find the container with id 94cc0b38e1622139902aeeaefc2a4649ae6afa7023ba8f2ff327ef4f4d131389 Jul 07 16:27:50.168508 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:50.168465 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-4dwvn" event={"ID":"d50601fa-a47c-4404-8e35-871b6a2e593d","Type":"ContainerStarted","Data":"94cc0b38e1622139902aeeaefc2a4649ae6afa7023ba8f2ff327ef4f4d131389"} Jul 07 16:27:52.182041 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:52.182000 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-4dwvn" event={"ID":"d50601fa-a47c-4404-8e35-871b6a2e593d","Type":"ContainerStarted","Data":"12ed5e3e538aa99f4ba4bc28235e12590a50ab8520269df8330dfc806a9a95ed"} Jul 07 16:27:52.182419 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:52.182144 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-4dwvn" Jul 07 16:27:52.200037 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:27:52.199983 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-4dwvn" podStartSLOduration=1.6412410849999999 podStartE2EDuration="4.19996866s" totalImagesPullingTime="2.558727575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:27:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:27:52.197492464 +0000 UTC m=+580.359305418" watchObservedRunningTime="2026-07-07 16:27:52.19996866 +0000 UTC m=+580.361781614" Jul 07 16:28:03.189537 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:28:03.189495 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-4dwvn" Jul 07 16:29:02.381290 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:02.381248 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:29:10.642568 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:10.642530 2564 generic.go:356] "Generic (PLEG): container finished" podID="d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f" containerID="c6789ceea618969f903f59beffceb57b69812667dce5c0e023a3f362c6310949" exitCode=0 Jul 07 16:29:10.643037 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:10.642607 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" event={"ID":"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f","Type":"ContainerDied","Data":"c6789ceea618969f903f59beffceb57b69812667dce5c0e023a3f362c6310949"} Jul 07 16:29:10.643037 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:10.643011 2564 scope.go:117] "RemoveContainer" containerID="c6789ceea618969f903f59beffceb57b69812667dce5c0e023a3f362c6310949" Jul 07 16:29:10.644007 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:10.643979 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:29:10.644083 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:10.644052 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:29:10.644122 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:10.644082 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 07 16:29:11.650700 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:11.650666 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" event={"ID":"d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f","Type":"ContainerStarted","Data":"c927934aab87b8cdfffc69374a611a88ae9ae8040d612c13b2135fbfb8398e7d"} Jul 07 16:29:12.617400 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:12.617360 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:29:12.617667 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:12.617646 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-9rftp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 07 16:29:12.617796 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:12.617695 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" podUID="d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 07 16:29:13.616047 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:13.616003 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-9rftp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 07 16:29:13.616551 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:13.616122 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" podUID="d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 07 16:29:14.616571 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:14.616530 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-9rftp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 07 16:29:14.617220 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:14.617186 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" podUID="d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 07 16:29:15.620114 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:15.620075 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:29:15.620502 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:15.620448 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:29:15.621283 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:15.621263 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-9rftp" Jul 07 16:29:17.273753 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:17.273714 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:29:25.615331 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:25.615289 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:29:39.359432 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.359389 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz"] Jul 07 16:29:39.362122 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.362101 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:39.365305 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.365277 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-crwj9\"" Jul 07 16:29:39.365710 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.365690 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 16:29:39.366285 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.366264 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 16:29:39.366440 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.366418 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 07 16:29:39.376960 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.376932 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz"] Jul 07 16:29:39.466022 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.465980 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96f0c349-7844-4c25-97dd-129fdfe22066-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-thcvz\" (UID: \"96f0c349-7844-4c25-97dd-129fdfe22066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:39.466022 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.466022 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q26sf\" (UniqueName: \"kubernetes.io/projected/96f0c349-7844-4c25-97dd-129fdfe22066-kube-api-access-q26sf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-thcvz\" (UID: \"96f0c349-7844-4c25-97dd-129fdfe22066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:39.466253 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.466058 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96f0c349-7844-4c25-97dd-129fdfe22066-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-thcvz\" (UID: \"96f0c349-7844-4c25-97dd-129fdfe22066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:39.466253 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.466087 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96f0c349-7844-4c25-97dd-129fdfe22066-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-thcvz\" (UID: \"96f0c349-7844-4c25-97dd-129fdfe22066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:39.466253 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.466106 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96f0c349-7844-4c25-97dd-129fdfe22066-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-thcvz\" (UID: \"96f0c349-7844-4c25-97dd-129fdfe22066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:39.466253 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.466129 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96f0c349-7844-4c25-97dd-129fdfe22066-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-thcvz\" (UID: \"96f0c349-7844-4c25-97dd-129fdfe22066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:39.567481 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.567441 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96f0c349-7844-4c25-97dd-129fdfe22066-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-thcvz\" (UID: \"96f0c349-7844-4c25-97dd-129fdfe22066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:39.568134 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.567499 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q26sf\" (UniqueName: \"kubernetes.io/projected/96f0c349-7844-4c25-97dd-129fdfe22066-kube-api-access-q26sf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-thcvz\" (UID: \"96f0c349-7844-4c25-97dd-129fdfe22066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:39.568291 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.568170 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96f0c349-7844-4c25-97dd-129fdfe22066-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-thcvz\" (UID: \"96f0c349-7844-4c25-97dd-129fdfe22066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:39.568291 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.568238 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96f0c349-7844-4c25-97dd-129fdfe22066-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-thcvz\" (UID: \"96f0c349-7844-4c25-97dd-129fdfe22066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:39.568291 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.568274 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96f0c349-7844-4c25-97dd-129fdfe22066-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-thcvz\" (UID: \"96f0c349-7844-4c25-97dd-129fdfe22066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:39.568456 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.568304 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96f0c349-7844-4c25-97dd-129fdfe22066-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-thcvz\" (UID: \"96f0c349-7844-4c25-97dd-129fdfe22066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:39.572932 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.569045 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96f0c349-7844-4c25-97dd-129fdfe22066-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-thcvz\" (UID: \"96f0c349-7844-4c25-97dd-129fdfe22066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:39.572932 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.569335 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96f0c349-7844-4c25-97dd-129fdfe22066-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-thcvz\" (UID: \"96f0c349-7844-4c25-97dd-129fdfe22066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:39.572932 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.569538 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96f0c349-7844-4c25-97dd-129fdfe22066-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-thcvz\" (UID: \"96f0c349-7844-4c25-97dd-129fdfe22066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:39.572932 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.571155 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96f0c349-7844-4c25-97dd-129fdfe22066-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-thcvz\" (UID: \"96f0c349-7844-4c25-97dd-129fdfe22066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:39.577346 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.577317 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96f0c349-7844-4c25-97dd-129fdfe22066-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-thcvz\" (UID: \"96f0c349-7844-4c25-97dd-129fdfe22066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:39.585716 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.585679 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q26sf\" (UniqueName: \"kubernetes.io/projected/96f0c349-7844-4c25-97dd-129fdfe22066-kube-api-access-q26sf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-thcvz\" (UID: \"96f0c349-7844-4c25-97dd-129fdfe22066\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:39.673535 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.673444 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:39.826026 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.825995 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz"] Jul 07 16:29:39.826665 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:29:39.826637 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96f0c349_7844_4c25_97dd_129fdfe22066.slice/crio-555839c6f34046808eef5b84e371324b6936f2703cda32a1e246b0de7623fed2 WatchSource:0}: Error finding container 555839c6f34046808eef5b84e371324b6936f2703cda32a1e246b0de7623fed2: Status 404 returned error can't find the container with id 555839c6f34046808eef5b84e371324b6936f2703cda32a1e246b0de7623fed2 Jul 07 16:29:39.828079 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:39.828064 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 16:29:40.659514 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:40.659475 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:29:40.826302 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:40.826266 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" event={"ID":"96f0c349-7844-4c25-97dd-129fdfe22066","Type":"ContainerStarted","Data":"555839c6f34046808eef5b84e371324b6936f2703cda32a1e246b0de7623fed2"} Jul 07 16:29:41.834340 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:41.834307 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" event={"ID":"96f0c349-7844-4c25-97dd-129fdfe22066","Type":"ContainerStarted","Data":"7d7ab59ebbf34c144afe99691eb203e39a016daad1d45300f0843aefb11e1597"} Jul 07 16:29:47.354419 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:47.354355 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" podStartSLOduration=7.322388379 podStartE2EDuration="8.354337751s" totalImagesPullingTime="1.031949372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:29:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:29:41.85464625 +0000 UTC m=+690.016459207" watchObservedRunningTime="2026-07-07 16:29:47.354337751 +0000 UTC m=+695.516150706" Jul 07 16:29:47.356175 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:47.356146 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:29:49.673988 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:49.673933 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:49.674527 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:49.674018 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:49.686460 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:49.686433 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:49.891652 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:49.891618 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-thcvz" Jul 07 16:29:58.966626 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:58.966591 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2"] Jul 07 16:29:58.970537 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:58.970520 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:29:58.973363 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:58.973340 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 07 16:29:58.982196 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:58.982171 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2"] Jul 07 16:29:59.054982 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:59.054935 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18eab8bf-a1d5-4ae7-b6a3-2422d1087bff-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sdbw2\" (UID: \"18eab8bf-a1d5-4ae7-b6a3-2422d1087bff\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:29:59.055190 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:59.054991 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18eab8bf-a1d5-4ae7-b6a3-2422d1087bff-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sdbw2\" (UID: \"18eab8bf-a1d5-4ae7-b6a3-2422d1087bff\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:29:59.055190 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:59.055080 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18eab8bf-a1d5-4ae7-b6a3-2422d1087bff-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sdbw2\" (UID: \"18eab8bf-a1d5-4ae7-b6a3-2422d1087bff\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:29:59.055190 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:59.055140 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18eab8bf-a1d5-4ae7-b6a3-2422d1087bff-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sdbw2\" (UID: \"18eab8bf-a1d5-4ae7-b6a3-2422d1087bff\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:29:59.055190 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:59.055162 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chk9q\" (UniqueName: \"kubernetes.io/projected/18eab8bf-a1d5-4ae7-b6a3-2422d1087bff-kube-api-access-chk9q\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sdbw2\" (UID: \"18eab8bf-a1d5-4ae7-b6a3-2422d1087bff\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:29:59.055190 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:59.055186 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18eab8bf-a1d5-4ae7-b6a3-2422d1087bff-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sdbw2\" (UID: \"18eab8bf-a1d5-4ae7-b6a3-2422d1087bff\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:29:59.156622 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:59.156582 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18eab8bf-a1d5-4ae7-b6a3-2422d1087bff-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sdbw2\" (UID: \"18eab8bf-a1d5-4ae7-b6a3-2422d1087bff\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:29:59.156622 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:59.156624 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chk9q\" (UniqueName: \"kubernetes.io/projected/18eab8bf-a1d5-4ae7-b6a3-2422d1087bff-kube-api-access-chk9q\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sdbw2\" (UID: \"18eab8bf-a1d5-4ae7-b6a3-2422d1087bff\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:29:59.156938 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:59.156647 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18eab8bf-a1d5-4ae7-b6a3-2422d1087bff-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sdbw2\" (UID: \"18eab8bf-a1d5-4ae7-b6a3-2422d1087bff\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:29:59.156938 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:59.156698 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18eab8bf-a1d5-4ae7-b6a3-2422d1087bff-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sdbw2\" (UID: \"18eab8bf-a1d5-4ae7-b6a3-2422d1087bff\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:29:59.156938 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:59.156717 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18eab8bf-a1d5-4ae7-b6a3-2422d1087bff-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sdbw2\" (UID: \"18eab8bf-a1d5-4ae7-b6a3-2422d1087bff\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:29:59.156938 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:59.156750 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18eab8bf-a1d5-4ae7-b6a3-2422d1087bff-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sdbw2\" (UID: \"18eab8bf-a1d5-4ae7-b6a3-2422d1087bff\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:29:59.157159 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:59.157135 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18eab8bf-a1d5-4ae7-b6a3-2422d1087bff-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sdbw2\" (UID: \"18eab8bf-a1d5-4ae7-b6a3-2422d1087bff\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:29:59.157222 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:59.157179 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18eab8bf-a1d5-4ae7-b6a3-2422d1087bff-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sdbw2\" (UID: \"18eab8bf-a1d5-4ae7-b6a3-2422d1087bff\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:29:59.157275 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:59.157234 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18eab8bf-a1d5-4ae7-b6a3-2422d1087bff-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sdbw2\" (UID: \"18eab8bf-a1d5-4ae7-b6a3-2422d1087bff\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:29:59.159255 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:59.159196 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18eab8bf-a1d5-4ae7-b6a3-2422d1087bff-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sdbw2\" (UID: \"18eab8bf-a1d5-4ae7-b6a3-2422d1087bff\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:29:59.159486 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:59.159470 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18eab8bf-a1d5-4ae7-b6a3-2422d1087bff-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sdbw2\" (UID: \"18eab8bf-a1d5-4ae7-b6a3-2422d1087bff\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:29:59.166450 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:59.166428 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chk9q\" (UniqueName: \"kubernetes.io/projected/18eab8bf-a1d5-4ae7-b6a3-2422d1087bff-kube-api-access-chk9q\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sdbw2\" (UID: \"18eab8bf-a1d5-4ae7-b6a3-2422d1087bff\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:29:59.282445 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:59.282355 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:29:59.430241 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:59.429063 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2"] Jul 07 16:29:59.432710 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:29:59.432681 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18eab8bf_a1d5_4ae7_b6a3_2422d1087bff.slice/crio-bb2172be9ffd84a388a9fff08958f031a37c9d7583abece465b1df2e4aca7f6c WatchSource:0}: Error finding container bb2172be9ffd84a388a9fff08958f031a37c9d7583abece465b1df2e4aca7f6c: Status 404 returned error can't find the container with id bb2172be9ffd84a388a9fff08958f031a37c9d7583abece465b1df2e4aca7f6c Jul 07 16:29:59.941634 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:59.941527 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" event={"ID":"18eab8bf-a1d5-4ae7-b6a3-2422d1087bff","Type":"ContainerStarted","Data":"3a947f4edadb0a6482ce45f26d55fc28021c99ac8dd57965b81f77d91e39b045"} Jul 07 16:29:59.941634 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:59.941591 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" event={"ID":"18eab8bf-a1d5-4ae7-b6a3-2422d1087bff","Type":"ContainerStarted","Data":"bb2172be9ffd84a388a9fff08958f031a37c9d7583abece465b1df2e4aca7f6c"} Jul 07 16:29:59.962501 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:29:59.962443 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" podStartSLOduration=1.750530452 podStartE2EDuration="1.962427906s" totalImagesPullingTime="211.897454ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:29:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:29:59.95970938 +0000 UTC m=+708.121522337" watchObservedRunningTime="2026-07-07 16:29:59.962427906 +0000 UTC m=+708.124240861" Jul 07 16:30:01.160108 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:30:01.160073 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:30:07.474091 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:30:07.474001 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:30:09.283449 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:30:09.283405 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:30:09.283994 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:30:09.283512 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:30:09.298511 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:30:09.298473 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:30:10.015861 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:30:10.015827 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sdbw2" Jul 07 16:31:40.064029 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:31:40.063936 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:31:49.949932 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:31:49.949870 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:32:00.654592 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:32:00.654548 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:32:10.654868 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:32:10.654828 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:32:20.656712 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:32:20.656663 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:32:48.853036 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:32:48.853002 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:33:00.260919 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:33:00.260877 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:33:12.359490 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:33:12.359420 2564 scope.go:117] "RemoveContainer" containerID="5a69b09a0e3bf4e5590e2792084eeeff0daa65ac87d6590f2465d7390291fb04" Jul 07 16:33:34.665559 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:33:34.665515 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:33:44.066434 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:33:44.066383 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:33:52.750837 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:33:52.750803 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:34:01.476269 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:34:01.476228 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:34:12.762008 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:34:12.761968 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:34:22.354716 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:34:22.354683 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:34:33.160610 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:34:33.160563 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:34:42.351463 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:34:42.351367 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:35:05.657156 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:35:05.657112 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:35:14.456832 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:35:14.456798 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:35:38.562608 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:35:38.562570 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:35:49.154828 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:35:49.154795 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:36:04.186932 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:36:04.186875 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:36:14.959360 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:36:14.959318 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:36:24.167678 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:36:24.167639 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:36:34.857977 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:36:34.857935 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:36:44.078148 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:36:44.078110 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:37:10.273338 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:37:10.273299 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f"] Jul 07 16:37:10.273864 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:37:10.273507 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f" podUID="8fec340b-6734-419c-9813-8a84ef5481b6" containerName="manager" containerID="cri-o://1047bce0c40670e71573b61240aeea433ee2c9c9dad47366b1262aeff76951c0" gracePeriod=10 Jul 07 16:37:14.330630 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:37:14.330605 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f" Jul 07 16:37:14.430832 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:37:14.430796 2564 generic.go:356] "Generic (PLEG): container finished" podID="8fec340b-6734-419c-9813-8a84ef5481b6" containerID="1047bce0c40670e71573b61240aeea433ee2c9c9dad47366b1262aeff76951c0" exitCode=0 Jul 07 16:37:14.431019 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:37:14.430846 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f" Jul 07 16:37:14.431019 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:37:14.430887 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f" event={"ID":"8fec340b-6734-419c-9813-8a84ef5481b6","Type":"ContainerDied","Data":"1047bce0c40670e71573b61240aeea433ee2c9c9dad47366b1262aeff76951c0"} Jul 07 16:37:14.431019 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:37:14.430970 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f" event={"ID":"8fec340b-6734-419c-9813-8a84ef5481b6","Type":"ContainerDied","Data":"63ea70a530ec5f38df91ccd41e0e20c280633735d95aa2a96d26e3ea147fba69"} Jul 07 16:37:14.431019 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:37:14.431001 2564 scope.go:117] "RemoveContainer" containerID="1047bce0c40670e71573b61240aeea433ee2c9c9dad47366b1262aeff76951c0" Jul 07 16:37:14.437442 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:37:14.437382 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5cj6k\" (UniqueName: \"kubernetes.io/projected/8fec340b-6734-419c-9813-8a84ef5481b6-kube-api-access-5cj6k\") pod \"8fec340b-6734-419c-9813-8a84ef5481b6\" (UID: \"8fec340b-6734-419c-9813-8a84ef5481b6\") " Jul 07 16:37:14.437442 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:37:14.437436 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fec340b-6734-419c-9813-8a84ef5481b6-extensions-socket-volume\") pod \"8fec340b-6734-419c-9813-8a84ef5481b6\" (UID: \"8fec340b-6734-419c-9813-8a84ef5481b6\") " Jul 07 16:37:14.437812 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:37:14.437793 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8fec340b-6734-419c-9813-8a84ef5481b6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8fec340b-6734-419c-9813-8a84ef5481b6" (UID: "8fec340b-6734-419c-9813-8a84ef5481b6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 16:37:14.439461 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:37:14.439434 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fec340b-6734-419c-9813-8a84ef5481b6-kube-api-access-5cj6k" (OuterVolumeSpecName: "kube-api-access-5cj6k") pod "8fec340b-6734-419c-9813-8a84ef5481b6" (UID: "8fec340b-6734-419c-9813-8a84ef5481b6"). InnerVolumeSpecName "kube-api-access-5cj6k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:37:14.440786 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:37:14.440765 2564 scope.go:117] "RemoveContainer" containerID="1047bce0c40670e71573b61240aeea433ee2c9c9dad47366b1262aeff76951c0" Jul 07 16:37:14.441089 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:37:14.441065 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1047bce0c40670e71573b61240aeea433ee2c9c9dad47366b1262aeff76951c0\": container with ID starting with 1047bce0c40670e71573b61240aeea433ee2c9c9dad47366b1262aeff76951c0 not found: ID does not exist" containerID="1047bce0c40670e71573b61240aeea433ee2c9c9dad47366b1262aeff76951c0" Jul 07 16:37:14.441156 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:37:14.441091 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1047bce0c40670e71573b61240aeea433ee2c9c9dad47366b1262aeff76951c0"} err="failed to get container status \"1047bce0c40670e71573b61240aeea433ee2c9c9dad47366b1262aeff76951c0\": rpc error: code = NotFound desc = could not find container \"1047bce0c40670e71573b61240aeea433ee2c9c9dad47366b1262aeff76951c0\": container with ID starting with 1047bce0c40670e71573b61240aeea433ee2c9c9dad47366b1262aeff76951c0 not found: ID does not exist" Jul 07 16:37:14.539083 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:37:14.539041 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5cj6k\" (UniqueName: \"kubernetes.io/projected/8fec340b-6734-419c-9813-8a84ef5481b6-kube-api-access-5cj6k\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:37:14.539083 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:37:14.539075 2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8fec340b-6734-419c-9813-8a84ef5481b6-extensions-socket-volume\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:37:14.760836 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:37:14.760801 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f"] Jul 07 16:37:14.765623 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:37:14.765595 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9d97f"] Jul 07 16:37:16.350297 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:37:16.350264 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fec340b-6734-419c-9813-8a84ef5481b6" path="/var/lib/kubelet/pods/8fec340b-6734-419c-9813-8a84ef5481b6/volumes" Jul 07 16:38:11.421782 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:38:11.421741 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mvwfq"] Jul 07 16:38:11.422302 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:38:11.422189 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fec340b-6734-419c-9813-8a84ef5481b6" containerName="manager" Jul 07 16:38:11.422302 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:38:11.422202 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fec340b-6734-419c-9813-8a84ef5481b6" containerName="manager" Jul 07 16:38:11.422302 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:38:11.422271 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fec340b-6734-419c-9813-8a84ef5481b6" containerName="manager" Jul 07 16:38:11.425339 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:38:11.425322 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mvwfq" Jul 07 16:38:11.428170 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:38:11.428145 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kbc57\"" Jul 07 16:38:11.438205 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:38:11.438173 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mvwfq"] Jul 07 16:38:11.559461 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:38:11.559425 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsg22\" (UniqueName: \"kubernetes.io/projected/db9445d4-e406-477c-9219-2990654c0438-kube-api-access-tsg22\") pod \"kuadrant-operator-controller-manager-86878c769f-mvwfq\" (UID: \"db9445d4-e406-477c-9219-2990654c0438\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mvwfq" Jul 07 16:38:11.559653 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:38:11.559475 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db9445d4-e406-477c-9219-2990654c0438-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mvwfq\" (UID: \"db9445d4-e406-477c-9219-2990654c0438\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mvwfq" Jul 07 16:38:11.660120 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:38:11.660077 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsg22\" (UniqueName: \"kubernetes.io/projected/db9445d4-e406-477c-9219-2990654c0438-kube-api-access-tsg22\") pod \"kuadrant-operator-controller-manager-86878c769f-mvwfq\" (UID: \"db9445d4-e406-477c-9219-2990654c0438\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mvwfq" Jul 07 16:38:11.660120 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:38:11.660134 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db9445d4-e406-477c-9219-2990654c0438-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mvwfq\" (UID: \"db9445d4-e406-477c-9219-2990654c0438\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mvwfq" Jul 07 16:38:11.660474 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:38:11.660458 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/db9445d4-e406-477c-9219-2990654c0438-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-mvwfq\" (UID: \"db9445d4-e406-477c-9219-2990654c0438\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mvwfq" Jul 07 16:38:11.669567 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:38:11.669539 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsg22\" (UniqueName: \"kubernetes.io/projected/db9445d4-e406-477c-9219-2990654c0438-kube-api-access-tsg22\") pod \"kuadrant-operator-controller-manager-86878c769f-mvwfq\" (UID: \"db9445d4-e406-477c-9219-2990654c0438\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mvwfq" Jul 07 16:38:11.735607 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:38:11.735508 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mvwfq" Jul 07 16:38:11.895742 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:38:11.895705 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mvwfq"] Jul 07 16:38:11.896779 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:38:11.896753 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb9445d4_e406_477c_9219_2990654c0438.slice/crio-f839202a4c91246b0cf68726c1b56f8c0dc7f0132057f175c87245bd5377d702 WatchSource:0}: Error finding container f839202a4c91246b0cf68726c1b56f8c0dc7f0132057f175c87245bd5377d702: Status 404 returned error can't find the container with id f839202a4c91246b0cf68726c1b56f8c0dc7f0132057f175c87245bd5377d702 Jul 07 16:38:12.765348 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:38:12.765255 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mvwfq" event={"ID":"db9445d4-e406-477c-9219-2990654c0438","Type":"ContainerStarted","Data":"6d86bc625d5bf739a7bbaf16aee860d393cbf672cfc3704e51049d5c4cb95e3c"} Jul 07 16:38:12.765348 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:38:12.765315 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mvwfq" event={"ID":"db9445d4-e406-477c-9219-2990654c0438","Type":"ContainerStarted","Data":"f839202a4c91246b0cf68726c1b56f8c0dc7f0132057f175c87245bd5377d702"} Jul 07 16:38:12.765348 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:38:12.765348 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mvwfq" Jul 07 16:38:12.787198 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:38:12.787144 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mvwfq" podStartSLOduration=1.787126778 podStartE2EDuration="1.787126778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:38:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:38:12.785408841 +0000 UTC m=+1200.947221821" watchObservedRunningTime="2026-07-07 16:38:12.787126778 +0000 UTC m=+1200.948939733" Jul 07 16:38:23.773735 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:38:23.773703 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-mvwfq" Jul 07 16:38:31.652818 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:38:31.652782 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:38:40.464290 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:38:40.464247 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:38:52.756916 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:38:52.756876 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:39:04.652578 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:39:04.652491 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:39:16.453171 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:39:16.453134 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:39:25.447845 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:39:25.447808 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:39:35.750954 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:39:35.750917 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:39:45.251562 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:39:45.251522 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:39:57.054114 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:39:57.054078 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:40:06.472570 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:40:06.472528 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:40:17.465135 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:40:17.465097 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:40:43.860675 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:40:43.860583 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:40:51.077450 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:40:51.077409 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:41:00.754816 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:41:00.754776 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:41:09.956455 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:41:09.956415 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:41:16.550113 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:41:16.550077 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:41:31.152815 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:41:31.152770 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:41:39.961546 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:41:39.961499 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:41:48.853006 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:41:48.852964 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:41:58.558863 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:41:58.558823 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:42:07.967743 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:42:07.967658 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:42:16.053375 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:42:16.053334 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:42:30.252384 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:42:30.252346 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:42:38.868265 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:42:38.868232 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:42:49.469989 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:42:49.469949 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:42:59.465337 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:42:59.465291 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:43:11.162123 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:43:11.162069 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:43:21.361323 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:43:21.361281 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:43:33.976869 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:43:33.976823 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:43:45.952115 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:43:45.952028 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:43:57.959234 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:43:57.959196 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:44:10.764518 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:44:10.764480 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:44:22.669457 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:44:22.669419 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:44:32.760720 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:44:32.760682 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:44:41.454742 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:44:41.454706 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:46:35.059244 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:35.059150 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:46:44.456339 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:44.456298 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:46:47.366328 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.366280 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-765c67986-d496s"] Jul 07 16:46:47.370316 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.370288 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:47.394010 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.393971 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-765c67986-d496s"] Jul 07 16:46:47.467610 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.467573 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f2cc7df5-7159-481c-97ea-9017799b6a9d-console-serving-cert\") pod \"console-765c67986-d496s\" (UID: \"f2cc7df5-7159-481c-97ea-9017799b6a9d\") " pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:47.467610 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.467614 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f2cc7df5-7159-481c-97ea-9017799b6a9d-oauth-serving-cert\") pod \"console-765c67986-d496s\" (UID: \"f2cc7df5-7159-481c-97ea-9017799b6a9d\") " pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:47.467889 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.467640 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f2cc7df5-7159-481c-97ea-9017799b6a9d-console-oauth-config\") pod \"console-765c67986-d496s\" (UID: \"f2cc7df5-7159-481c-97ea-9017799b6a9d\") " pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:47.467889 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.467682 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f2cc7df5-7159-481c-97ea-9017799b6a9d-service-ca\") pod \"console-765c67986-d496s\" (UID: \"f2cc7df5-7159-481c-97ea-9017799b6a9d\") " pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:47.467889 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.467746 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2cc7df5-7159-481c-97ea-9017799b6a9d-trusted-ca-bundle\") pod \"console-765c67986-d496s\" (UID: \"f2cc7df5-7159-481c-97ea-9017799b6a9d\") " pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:47.467889 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.467832 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f2cc7df5-7159-481c-97ea-9017799b6a9d-console-config\") pod \"console-765c67986-d496s\" (UID: \"f2cc7df5-7159-481c-97ea-9017799b6a9d\") " pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:47.467889 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.467862 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6j4c\" (UniqueName: \"kubernetes.io/projected/f2cc7df5-7159-481c-97ea-9017799b6a9d-kube-api-access-t6j4c\") pod \"console-765c67986-d496s\" (UID: \"f2cc7df5-7159-481c-97ea-9017799b6a9d\") " pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:47.569080 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.569036 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f2cc7df5-7159-481c-97ea-9017799b6a9d-console-serving-cert\") pod \"console-765c67986-d496s\" (UID: \"f2cc7df5-7159-481c-97ea-9017799b6a9d\") " pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:47.569080 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.569082 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f2cc7df5-7159-481c-97ea-9017799b6a9d-oauth-serving-cert\") pod \"console-765c67986-d496s\" (UID: \"f2cc7df5-7159-481c-97ea-9017799b6a9d\") " pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:47.569344 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.569105 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f2cc7df5-7159-481c-97ea-9017799b6a9d-console-oauth-config\") pod \"console-765c67986-d496s\" (UID: \"f2cc7df5-7159-481c-97ea-9017799b6a9d\") " pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:47.569344 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.569122 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f2cc7df5-7159-481c-97ea-9017799b6a9d-service-ca\") pod \"console-765c67986-d496s\" (UID: \"f2cc7df5-7159-481c-97ea-9017799b6a9d\") " pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:47.569344 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.569153 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2cc7df5-7159-481c-97ea-9017799b6a9d-trusted-ca-bundle\") pod \"console-765c67986-d496s\" (UID: \"f2cc7df5-7159-481c-97ea-9017799b6a9d\") " pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:47.569344 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.569194 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f2cc7df5-7159-481c-97ea-9017799b6a9d-console-config\") pod \"console-765c67986-d496s\" (UID: \"f2cc7df5-7159-481c-97ea-9017799b6a9d\") " pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:47.569344 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.569216 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6j4c\" (UniqueName: \"kubernetes.io/projected/f2cc7df5-7159-481c-97ea-9017799b6a9d-kube-api-access-t6j4c\") pod \"console-765c67986-d496s\" (UID: \"f2cc7df5-7159-481c-97ea-9017799b6a9d\") " pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:47.569895 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.569873 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f2cc7df5-7159-481c-97ea-9017799b6a9d-oauth-serving-cert\") pod \"console-765c67986-d496s\" (UID: \"f2cc7df5-7159-481c-97ea-9017799b6a9d\") " pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:47.570110 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.570083 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f2cc7df5-7159-481c-97ea-9017799b6a9d-console-config\") pod \"console-765c67986-d496s\" (UID: \"f2cc7df5-7159-481c-97ea-9017799b6a9d\") " pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:47.570222 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.570122 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f2cc7df5-7159-481c-97ea-9017799b6a9d-service-ca\") pod \"console-765c67986-d496s\" (UID: \"f2cc7df5-7159-481c-97ea-9017799b6a9d\") " pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:47.570439 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.570415 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f2cc7df5-7159-481c-97ea-9017799b6a9d-trusted-ca-bundle\") pod \"console-765c67986-d496s\" (UID: \"f2cc7df5-7159-481c-97ea-9017799b6a9d\") " pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:47.571732 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.571707 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f2cc7df5-7159-481c-97ea-9017799b6a9d-console-oauth-config\") pod \"console-765c67986-d496s\" (UID: \"f2cc7df5-7159-481c-97ea-9017799b6a9d\") " pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:47.571921 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.571889 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f2cc7df5-7159-481c-97ea-9017799b6a9d-console-serving-cert\") pod \"console-765c67986-d496s\" (UID: \"f2cc7df5-7159-481c-97ea-9017799b6a9d\") " pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:47.578375 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.578357 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6j4c\" (UniqueName: \"kubernetes.io/projected/f2cc7df5-7159-481c-97ea-9017799b6a9d-kube-api-access-t6j4c\") pod \"console-765c67986-d496s\" (UID: \"f2cc7df5-7159-481c-97ea-9017799b6a9d\") " pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:47.679661 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.679562 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:47.824536 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:47.824508 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-765c67986-d496s"] Jul 07 16:46:47.826469 ip-10-0-129-174 kubenswrapper[2564]: W0707 16:46:47.826440 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2cc7df5_7159_481c_97ea_9017799b6a9d.slice/crio-6035272d64c3c22ffbbacb699564e6825517db373f788d111fe38939317ac400 WatchSource:0}: Error finding container 6035272d64c3c22ffbbacb699564e6825517db373f788d111fe38939317ac400: Status 404 returned error can't find the container with id 6035272d64c3c22ffbbacb699564e6825517db373f788d111fe38939317ac400 Jul 07 16:46:48.712406 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:48.712368 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-765c67986-d496s" event={"ID":"f2cc7df5-7159-481c-97ea-9017799b6a9d","Type":"ContainerStarted","Data":"6c1c5c2f28ff7d78daaca02f317a5dba53c2ecf4d62554688173bb1de1ee9ae6"} Jul 07 16:46:48.712801 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:48.712419 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-765c67986-d496s" event={"ID":"f2cc7df5-7159-481c-97ea-9017799b6a9d","Type":"ContainerStarted","Data":"6035272d64c3c22ffbbacb699564e6825517db373f788d111fe38939317ac400"} Jul 07 16:46:48.730635 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:48.730576 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-765c67986-d496s" podStartSLOduration=1.730561163 podStartE2EDuration="1.730561163s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 16:46:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 16:46:48.730145098 +0000 UTC m=+1716.891958055" watchObservedRunningTime="2026-07-07 16:46:48.730561163 +0000 UTC m=+1716.892374096" Jul 07 16:46:56.064976 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:56.064938 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:46:57.680348 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:57.680259 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:57.680348 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:57.680322 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:57.685247 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:57.685225 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:57.768110 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:57.768081 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-765c67986-d496s" Jul 07 16:46:57.829793 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:46:57.829756 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5989d5bcc8-cw2hh"] Jul 07 16:47:07.186148 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:07.186106 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:47:22.851722 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:22.851659 2564 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5989d5bcc8-cw2hh" podUID="f57359a5-a9b6-4704-bd36-b7efa1f573b4" containerName="console" containerID="cri-o://91f8a071f9debadb05785f47a82cdbce2636a0c1c5770f38d07459520909ba87" gracePeriod=15 Jul 07 16:47:23.118141 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.118106 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:47:23.207296 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.207251 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-trusted-ca-bundle\") pod \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " Jul 07 16:47:23.207296 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.207290 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-service-ca\") pod \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " Jul 07 16:47:23.207554 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.207313 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f57359a5-a9b6-4704-bd36-b7efa1f573b4-console-oauth-config\") pod \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " Jul 07 16:47:23.207554 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.207359 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f57359a5-a9b6-4704-bd36-b7efa1f573b4-console-serving-cert\") pod \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " Jul 07 16:47:23.207554 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.207420 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-console-config\") pod \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " Jul 07 16:47:23.207554 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.207458 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6b7x6\" (UniqueName: \"kubernetes.io/projected/f57359a5-a9b6-4704-bd36-b7efa1f573b4-kube-api-access-6b7x6\") pod \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " Jul 07 16:47:23.207791 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.207565 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-oauth-serving-cert\") pod \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\" (UID: \"f57359a5-a9b6-4704-bd36-b7efa1f573b4\") " Jul 07 16:47:23.207853 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.207682 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-service-ca" (OuterVolumeSpecName: "service-ca") pod "f57359a5-a9b6-4704-bd36-b7efa1f573b4" (UID: "f57359a5-a9b6-4704-bd36-b7efa1f573b4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:47:23.207945 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.207878 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-console-config" (OuterVolumeSpecName: "console-config") pod "f57359a5-a9b6-4704-bd36-b7efa1f573b4" (UID: "f57359a5-a9b6-4704-bd36-b7efa1f573b4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:47:23.208016 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.207972 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-service-ca\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:47:23.208092 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.208059 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f57359a5-a9b6-4704-bd36-b7efa1f573b4" (UID: "f57359a5-a9b6-4704-bd36-b7efa1f573b4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:47:23.208186 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.208152 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f57359a5-a9b6-4704-bd36-b7efa1f573b4" (UID: "f57359a5-a9b6-4704-bd36-b7efa1f573b4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 16:47:23.209761 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.209739 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f57359a5-a9b6-4704-bd36-b7efa1f573b4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f57359a5-a9b6-4704-bd36-b7efa1f573b4" (UID: "f57359a5-a9b6-4704-bd36-b7efa1f573b4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:47:23.210067 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.210050 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f57359a5-a9b6-4704-bd36-b7efa1f573b4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f57359a5-a9b6-4704-bd36-b7efa1f573b4" (UID: "f57359a5-a9b6-4704-bd36-b7efa1f573b4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 16:47:23.210174 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.210115 2564 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f57359a5-a9b6-4704-bd36-b7efa1f573b4-kube-api-access-6b7x6" (OuterVolumeSpecName: "kube-api-access-6b7x6") pod "f57359a5-a9b6-4704-bd36-b7efa1f573b4" (UID: "f57359a5-a9b6-4704-bd36-b7efa1f573b4"). InnerVolumeSpecName "kube-api-access-6b7x6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 16:47:23.308653 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.308615 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f57359a5-a9b6-4704-bd36-b7efa1f573b4-console-serving-cert\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:47:23.308653 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.308647 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-console-config\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:47:23.308653 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.308657 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6b7x6\" (UniqueName: \"kubernetes.io/projected/f57359a5-a9b6-4704-bd36-b7efa1f573b4-kube-api-access-6b7x6\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:47:23.308901 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.308666 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-oauth-serving-cert\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:47:23.308901 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.308675 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f57359a5-a9b6-4704-bd36-b7efa1f573b4-trusted-ca-bundle\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:47:23.308901 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.308683 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f57359a5-a9b6-4704-bd36-b7efa1f573b4-console-oauth-config\") on node \"ip-10-0-129-174.ec2.internal\" DevicePath \"\"" Jul 07 16:47:23.924949 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.924898 2564 generic.go:356] "Generic (PLEG): container finished" podID="f57359a5-a9b6-4704-bd36-b7efa1f573b4" containerID="91f8a071f9debadb05785f47a82cdbce2636a0c1c5770f38d07459520909ba87" exitCode=0 Jul 07 16:47:23.925397 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.924957 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5989d5bcc8-cw2hh" Jul 07 16:47:23.925397 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.924978 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5989d5bcc8-cw2hh" event={"ID":"f57359a5-a9b6-4704-bd36-b7efa1f573b4","Type":"ContainerDied","Data":"91f8a071f9debadb05785f47a82cdbce2636a0c1c5770f38d07459520909ba87"} Jul 07 16:47:23.925397 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.925029 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5989d5bcc8-cw2hh" event={"ID":"f57359a5-a9b6-4704-bd36-b7efa1f573b4","Type":"ContainerDied","Data":"828d8b05524e054b2388f38da62fd391a327a9bc481344912a7f9aaa33bff4d3"} Jul 07 16:47:23.925397 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.925050 2564 scope.go:117] "RemoveContainer" containerID="91f8a071f9debadb05785f47a82cdbce2636a0c1c5770f38d07459520909ba87" Jul 07 16:47:23.935314 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.935291 2564 scope.go:117] "RemoveContainer" containerID="91f8a071f9debadb05785f47a82cdbce2636a0c1c5770f38d07459520909ba87" Jul 07 16:47:23.935620 ip-10-0-129-174 kubenswrapper[2564]: E0707 16:47:23.935596 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91f8a071f9debadb05785f47a82cdbce2636a0c1c5770f38d07459520909ba87\": container with ID starting with 91f8a071f9debadb05785f47a82cdbce2636a0c1c5770f38d07459520909ba87 not found: ID does not exist" containerID="91f8a071f9debadb05785f47a82cdbce2636a0c1c5770f38d07459520909ba87" Jul 07 16:47:23.935706 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.935628 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91f8a071f9debadb05785f47a82cdbce2636a0c1c5770f38d07459520909ba87"} err="failed to get container status \"91f8a071f9debadb05785f47a82cdbce2636a0c1c5770f38d07459520909ba87\": rpc error: code = NotFound desc = could not find container \"91f8a071f9debadb05785f47a82cdbce2636a0c1c5770f38d07459520909ba87\": container with ID starting with 91f8a071f9debadb05785f47a82cdbce2636a0c1c5770f38d07459520909ba87 not found: ID does not exist" Jul 07 16:47:23.948027 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.947991 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5989d5bcc8-cw2hh"] Jul 07 16:47:23.959076 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:23.959044 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5989d5bcc8-cw2hh"] Jul 07 16:47:24.350183 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:24.350090 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f57359a5-a9b6-4704-bd36-b7efa1f573b4" path="/var/lib/kubelet/pods/f57359a5-a9b6-4704-bd36-b7efa1f573b4/volumes" Jul 07 16:47:38.263725 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:38.263681 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:47:51.852174 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:47:51.852135 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:48:09.153796 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:48:09.153709 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:48:21.162192 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:48:21.162155 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:49:00.859707 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:49:00.859668 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:49:09.976342 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:49:09.976308 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:49:24.371466 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:49:24.371434 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:50:12.774303 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:50:12.774264 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:50:23.154749 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:50:23.154708 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:50:32.904638 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:50:32.904598 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:50:59.256487 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:50:59.256449 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:51:15.958096 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:51:15.958002 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:51:29.867404 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:51:29.867368 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:52:04.508024 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:52:04.507985 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:52:18.176263 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:52:18.172772 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:52:30.866195 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:52:30.866158 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:52:44.760049 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:52:44.759965 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:52:56.055982 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:52:56.055939 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:53:04.980078 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:53:04.980042 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:53:18.864853 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:53:18.864816 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:53:44.670713 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:53:44.670672 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:54:07.161930 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:54:07.161827 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:54:24.063673 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:54:24.063634 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:54:41.260433 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:54:41.260396 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:54:54.650086 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:54:54.650045 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:55:11.155102 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:55:11.155059 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:55:26.564271 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:55:26.564239 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:55:43.380768 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:55:43.377045 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:56:00.265212 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:56:00.265172 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:56:24.657162 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:56:24.657126 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:56:47.659320 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:56:47.659279 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:57:03.059527 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:57:03.059483 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:57:22.271557 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:57:22.271515 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:57:53.762862 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:57:53.762826 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:58:30.986459 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:58:30.986416 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:58:43.983950 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:58:43.983856 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:58:57.783486 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:58:57.783446 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:59:39.474105 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:39.474060 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-4dwvn"] Jul 07 16:59:47.793013 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:47.792973 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xcq85_5abd0a46-2e65-49fd-a06d-30ee5cd38791/manager/0.log" Jul 07 16:59:47.925084 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:47.925048 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-lnck9_8492c6fd-ea82-4bb1-ab8e-4455c75dbe6d/manager/0.log" Jul 07 16:59:48.658065 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:48.658035 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-6gzws_bce89f98-6a41-4485-a4c7-f1bb3368e25a/manager/0.log" Jul 07 16:59:48.775793 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:48.775762 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f64778974-22mbb_bdb26e74-5407-41be-a50f-c9ecfbd8836f/copy-manifests/0.log" Jul 07 16:59:48.783452 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:48.783427 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f64778974-22mbb_bdb26e74-5407-41be-a50f-c9ecfbd8836f/manager/0.log" Jul 07 16:59:49.931422 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:49.931392 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp_54d4386c-0b67-4cac-a953-23cc4175999b/pull/0.log" Jul 07 16:59:49.938443 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:49.938419 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp_54d4386c-0b67-4cac-a953-23cc4175999b/extract/0.log" Jul 07 16:59:49.945700 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:49.945678 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp_54d4386c-0b67-4cac-a953-23cc4175999b/util/0.log" Jul 07 16:59:50.060346 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:50.060313 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v_35b8f5f2-9e59-4c68-a767-a6e6d45ba798/extract/0.log" Jul 07 16:59:50.068127 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:50.068102 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v_35b8f5f2-9e59-4c68-a767-a6e6d45ba798/util/0.log" Jul 07 16:59:50.076247 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:50.076226 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v_35b8f5f2-9e59-4c68-a767-a6e6d45ba798/pull/0.log" Jul 07 16:59:50.191258 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:50.191180 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4_1b330d35-9651-498e-abaa-3ae7b4831c6a/util/0.log" Jul 07 16:59:50.200823 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:50.200799 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4_1b330d35-9651-498e-abaa-3ae7b4831c6a/pull/0.log" Jul 07 16:59:50.207185 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:50.207165 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4_1b330d35-9651-498e-abaa-3ae7b4831c6a/extract/0.log" Jul 07 16:59:50.314575 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:50.314547 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf_372fdb21-39c4-4378-af66-8c3728384e8c/util/0.log" Jul 07 16:59:50.324200 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:50.324172 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf_372fdb21-39c4-4378-af66-8c3728384e8c/pull/0.log" Jul 07 16:59:50.332493 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:50.332473 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf_372fdb21-39c4-4378-af66-8c3728384e8c/extract/0.log" Jul 07 16:59:50.688412 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:50.688386 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-hf9ll_eb40afc6-0746-4753-bec1-7088e65076af/manager/0.log" Jul 07 16:59:51.030306 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:51.030228 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-mvwfq_db9445d4-e406-477c-9219-2990654c0438/manager/0.log" Jul 07 16:59:51.147404 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:51.147378 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-4dwvn_d50601fa-a47c-4404-8e35-871b6a2e593d/limitador/0.log" Jul 07 16:59:51.762045 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:51.762017 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-pdhmz_2cbd34f1-8649-41be-9595-798f5e903723/discovery/0.log" Jul 07 16:59:51.990387 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:51.990355 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cbbdb79bb-lbrks_d6f791a8-4e75-4f89-9b8f-c65fdd57aa21/kube-auth-proxy/0.log" Jul 07 16:59:52.106024 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:52.105998 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9rftp_d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f/istio-proxy/1.log" Jul 07 16:59:53.009678 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:53.009652 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-thcvz_96f0c349-7844-4c25-97dd-129fdfe22066/main/0.log" Jul 07 16:59:53.248897 ip-10-0-129-174 kubenswrapper[2564]: I0707 16:59:53.248850 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-sdbw2_18eab8bf-a1d5-4ae7-b6a3-2422d1087bff/main/0.log" Jul 07 17:00:00.429448 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:00.429416 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-868j2_093da19e-76ae-42f9-b23a-5ac90eb50fb1/global-pull-secret-syncer/0.log" Jul 07 17:00:00.655298 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:00.655271 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mdnxp_459203a6-cfbe-4739-892a-f6802b684809/konnectivity-agent/0.log" Jul 07 17:00:00.719585 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:00.719504 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-174.ec2.internal_e0639c46a9cdb652f922e26862f5abff/haproxy/0.log" Jul 07 17:00:04.966657 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:04.966577 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp_54d4386c-0b67-4cac-a953-23cc4175999b/extract/0.log" Jul 07 17:00:04.998451 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:04.998415 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp_54d4386c-0b67-4cac-a953-23cc4175999b/util/0.log" Jul 07 17:00:05.030124 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:05.030086 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759t8gfp_54d4386c-0b67-4cac-a953-23cc4175999b/pull/0.log" Jul 07 17:00:05.065625 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:05.065597 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v_35b8f5f2-9e59-4c68-a767-a6e6d45ba798/extract/0.log" Jul 07 17:00:05.095000 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:05.094971 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v_35b8f5f2-9e59-4c68-a767-a6e6d45ba798/util/0.log" Jul 07 17:00:05.125926 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:05.125879 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e06cd2v_35b8f5f2-9e59-4c68-a767-a6e6d45ba798/pull/0.log" Jul 07 17:00:05.158355 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:05.158331 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4_1b330d35-9651-498e-abaa-3ae7b4831c6a/extract/0.log" Jul 07 17:00:05.185934 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:05.185876 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4_1b330d35-9651-498e-abaa-3ae7b4831c6a/util/0.log" Jul 07 17:00:05.216274 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:05.216248 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73b2hm4_1b330d35-9651-498e-abaa-3ae7b4831c6a/pull/0.log" Jul 07 17:00:05.255843 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:05.255763 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf_372fdb21-39c4-4378-af66-8c3728384e8c/extract/0.log" Jul 07 17:00:05.285424 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:05.285395 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf_372fdb21-39c4-4378-af66-8c3728384e8c/util/0.log" Jul 07 17:00:05.315631 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:05.315607 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tw7hf_372fdb21-39c4-4378-af66-8c3728384e8c/pull/0.log" Jul 07 17:00:05.801039 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:05.801011 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-hf9ll_eb40afc6-0746-4753-bec1-7088e65076af/manager/0.log" Jul 07 17:00:05.986387 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:05.986353 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-mvwfq_db9445d4-e406-477c-9219-2990654c0438/manager/0.log" Jul 07 17:00:06.017552 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:06.017519 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-4dwvn_d50601fa-a47c-4404-8e35-871b6a2e593d/limitador/0.log" Jul 07 17:00:08.120594 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:08.120565 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-7r79m_698c0fe4-2b45-4ffd-b498-54dc14b4df02/cluster-monitoring-operator/0.log" Jul 07 17:00:08.296250 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:08.296219 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-cwp65_68f7ee89-78c7-43d1-af0d-d40573c762dd/monitoring-plugin/0.log" Jul 07 17:00:08.643798 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:08.643771 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d89dt_3b83dad7-acd2-43fe-8b45-b549bb50b2c3/node-exporter/0.log" Jul 07 17:00:08.718819 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:08.718793 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d89dt_3b83dad7-acd2-43fe-8b45-b549bb50b2c3/kube-rbac-proxy/0.log" Jul 07 17:00:08.775735 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:08.775707 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-d89dt_3b83dad7-acd2-43fe-8b45-b549bb50b2c3/init-textfile/0.log" Jul 07 17:00:08.825902 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:08.825875 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-ptnpp_cdb29d62-6eb7-406a-b6fb-249a2bd04e4f/kube-rbac-proxy-main/0.log" Jul 07 17:00:08.867395 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:08.867369 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-ptnpp_cdb29d62-6eb7-406a-b6fb-249a2bd04e4f/kube-rbac-proxy-self/0.log" Jul 07 17:00:08.908073 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:08.907997 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-ptnpp_cdb29d62-6eb7-406a-b6fb-249a2bd04e4f/openshift-state-metrics/0.log" Jul 07 17:00:09.452729 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:09.452701 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-f65687bb8-pr5pw_f3274426-8f98-4f7c-be9c-b3a57b88c414/telemeter-client/0.log" Jul 07 17:00:09.485131 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:09.485095 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-f65687bb8-pr5pw_f3274426-8f98-4f7c-be9c-b3a57b88c414/reload/0.log" Jul 07 17:00:09.523760 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:09.523736 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-f65687bb8-pr5pw_f3274426-8f98-4f7c-be9c-b3a57b88c414/kube-rbac-proxy/0.log" Jul 07 17:00:09.580229 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:09.580192 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c4c696cc7-kk4k6_ba804cd6-8e99-4485-a928-a8cd6e0ff350/thanos-query/0.log" Jul 07 17:00:09.609968 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:09.609941 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c4c696cc7-kk4k6_ba804cd6-8e99-4485-a928-a8cd6e0ff350/kube-rbac-proxy-web/0.log" Jul 07 17:00:09.646487 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:09.646458 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c4c696cc7-kk4k6_ba804cd6-8e99-4485-a928-a8cd6e0ff350/kube-rbac-proxy/0.log" Jul 07 17:00:09.682896 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:09.682855 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c4c696cc7-kk4k6_ba804cd6-8e99-4485-a928-a8cd6e0ff350/prom-label-proxy/0.log" Jul 07 17:00:09.724004 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:09.723899 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c4c696cc7-kk4k6_ba804cd6-8e99-4485-a928-a8cd6e0ff350/kube-rbac-proxy-rules/0.log" Jul 07 17:00:09.758852 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:09.758818 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c4c696cc7-kk4k6_ba804cd6-8e99-4485-a928-a8cd6e0ff350/kube-rbac-proxy-metrics/0.log" Jul 07 17:00:10.366458 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.366419 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85"] Jul 07 17:00:10.366853 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.366838 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f57359a5-a9b6-4704-bd36-b7efa1f573b4" containerName="console" Jul 07 17:00:10.366853 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.366854 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f57359a5-a9b6-4704-bd36-b7efa1f573b4" containerName="console" Jul 07 17:00:10.366970 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.366937 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f57359a5-a9b6-4704-bd36-b7efa1f573b4" containerName="console" Jul 07 17:00:10.370279 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.370263 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85" Jul 07 17:00:10.373944 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.373923 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-948xd\"/\"openshift-service-ca.crt\"" Jul 07 17:00:10.373944 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.373931 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-948xd\"/\"default-dockercfg-6wmwk\"" Jul 07 17:00:10.375395 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.375377 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-948xd\"/\"kube-root-ca.crt\"" Jul 07 17:00:10.382099 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.382075 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85"] Jul 07 17:00:10.443421 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.443385 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e58e34d2-4c22-4993-ba87-137c1522c022-proc\") pod \"perf-node-gather-daemonset-gxs85\" (UID: \"e58e34d2-4c22-4993-ba87-137c1522c022\") " pod="openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85" Jul 07 17:00:10.443627 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.443447 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e58e34d2-4c22-4993-ba87-137c1522c022-lib-modules\") pod \"perf-node-gather-daemonset-gxs85\" (UID: \"e58e34d2-4c22-4993-ba87-137c1522c022\") " pod="openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85" Jul 07 17:00:10.443627 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.443471 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e58e34d2-4c22-4993-ba87-137c1522c022-sys\") pod \"perf-node-gather-daemonset-gxs85\" (UID: \"e58e34d2-4c22-4993-ba87-137c1522c022\") " pod="openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85" Jul 07 17:00:10.443627 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.443508 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e58e34d2-4c22-4993-ba87-137c1522c022-podres\") pod \"perf-node-gather-daemonset-gxs85\" (UID: \"e58e34d2-4c22-4993-ba87-137c1522c022\") " pod="openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85" Jul 07 17:00:10.443627 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.443581 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brcl7\" (UniqueName: \"kubernetes.io/projected/e58e34d2-4c22-4993-ba87-137c1522c022-kube-api-access-brcl7\") pod \"perf-node-gather-daemonset-gxs85\" (UID: \"e58e34d2-4c22-4993-ba87-137c1522c022\") " pod="openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85" Jul 07 17:00:10.544454 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.544414 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e58e34d2-4c22-4993-ba87-137c1522c022-lib-modules\") pod \"perf-node-gather-daemonset-gxs85\" (UID: \"e58e34d2-4c22-4993-ba87-137c1522c022\") " pod="openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85" Jul 07 17:00:10.544454 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.544459 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e58e34d2-4c22-4993-ba87-137c1522c022-sys\") pod \"perf-node-gather-daemonset-gxs85\" (UID: \"e58e34d2-4c22-4993-ba87-137c1522c022\") " pod="openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85" Jul 07 17:00:10.544958 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.544491 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e58e34d2-4c22-4993-ba87-137c1522c022-podres\") pod \"perf-node-gather-daemonset-gxs85\" (UID: \"e58e34d2-4c22-4993-ba87-137c1522c022\") " pod="openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85" Jul 07 17:00:10.544958 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.544537 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brcl7\" (UniqueName: \"kubernetes.io/projected/e58e34d2-4c22-4993-ba87-137c1522c022-kube-api-access-brcl7\") pod \"perf-node-gather-daemonset-gxs85\" (UID: \"e58e34d2-4c22-4993-ba87-137c1522c022\") " pod="openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85" Jul 07 17:00:10.544958 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.544558 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e58e34d2-4c22-4993-ba87-137c1522c022-sys\") pod \"perf-node-gather-daemonset-gxs85\" (UID: \"e58e34d2-4c22-4993-ba87-137c1522c022\") " pod="openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85" Jul 07 17:00:10.544958 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.544566 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e58e34d2-4c22-4993-ba87-137c1522c022-proc\") pod \"perf-node-gather-daemonset-gxs85\" (UID: \"e58e34d2-4c22-4993-ba87-137c1522c022\") " pod="openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85" Jul 07 17:00:10.544958 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.544632 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e58e34d2-4c22-4993-ba87-137c1522c022-lib-modules\") pod \"perf-node-gather-daemonset-gxs85\" (UID: \"e58e34d2-4c22-4993-ba87-137c1522c022\") " pod="openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85" Jul 07 17:00:10.544958 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.544649 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e58e34d2-4c22-4993-ba87-137c1522c022-podres\") pod \"perf-node-gather-daemonset-gxs85\" (UID: \"e58e34d2-4c22-4993-ba87-137c1522c022\") " pod="openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85" Jul 07 17:00:10.544958 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.544663 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e58e34d2-4c22-4993-ba87-137c1522c022-proc\") pod \"perf-node-gather-daemonset-gxs85\" (UID: \"e58e34d2-4c22-4993-ba87-137c1522c022\") " pod="openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85" Jul 07 17:00:10.555158 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.555131 2564 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-brcl7\" (UniqueName: \"kubernetes.io/projected/e58e34d2-4c22-4993-ba87-137c1522c022-kube-api-access-brcl7\") pod \"perf-node-gather-daemonset-gxs85\" (UID: \"e58e34d2-4c22-4993-ba87-137c1522c022\") " pod="openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85" Jul 07 17:00:10.680972 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.680832 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85" Jul 07 17:00:10.836376 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:10.836347 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85"] Jul 07 17:00:10.838288 ip-10-0-129-174 kubenswrapper[2564]: W0707 17:00:10.838262 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode58e34d2_4c22_4993_ba87_137c1522c022.slice/crio-1ad884100f84011f5eb5622b4142eae93d2c43d6c893b26263300cb1b7591f7c WatchSource:0}: Error finding container 1ad884100f84011f5eb5622b4142eae93d2c43d6c893b26263300cb1b7591f7c: Status 404 returned error can't find the container with id 1ad884100f84011f5eb5622b4142eae93d2c43d6c893b26263300cb1b7591f7c Jul 07 17:00:11.325338 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:11.325299 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85" event={"ID":"e58e34d2-4c22-4993-ba87-137c1522c022","Type":"ContainerStarted","Data":"3f07d3768b1780421fece0b7b4bf2debc72a2fdced12487b87ecedf878323d57"} Jul 07 17:00:11.325555 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:11.325446 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85" Jul 07 17:00:11.325555 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:11.325492 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85" event={"ID":"e58e34d2-4c22-4993-ba87-137c1522c022","Type":"ContainerStarted","Data":"1ad884100f84011f5eb5622b4142eae93d2c43d6c893b26263300cb1b7591f7c"} Jul 07 17:00:11.344603 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:11.344550 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85" podStartSLOduration=1.344534306 podStartE2EDuration="1.344534306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 17:00:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 17:00:11.342579266 +0000 UTC m=+2519.504392224" watchObservedRunningTime="2026-07-07 17:00:11.344534306 +0000 UTC m=+2519.506347260" Jul 07 17:00:12.024045 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:12.024016 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-765c67986-d496s_f2cc7df5-7159-481c-97ea-9017799b6a9d/console/0.log" Jul 07 17:00:12.059812 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:12.059783 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-t2wmq_68959a9c-22fd-4804-9b45-0eb23f396983/download-server/0.log" Jul 07 17:00:12.634112 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:12.634080 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-rhsmp_6284964a-74fe-4d50-9a06-b13ba29d2d98/volume-data-source-validator/0.log" Jul 07 17:00:13.744020 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:13.743993 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gpl7j_e68eced5-8d40-432a-95f7-3f5ec13e0b25/dns/0.log" Jul 07 17:00:13.767303 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:13.767280 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gpl7j_e68eced5-8d40-432a-95f7-3f5ec13e0b25/kube-rbac-proxy/0.log" Jul 07 17:00:13.879578 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:13.879549 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-lxccs_83dce208-f3dc-4e5c-ba27-e2d6c9472a68/dns-node-resolver/0.log" Jul 07 17:00:14.575256 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:14.575225 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vjgrf_709a00b4-4252-4273-ad18-72ac558f394c/node-ca/0.log" Jul 07 17:00:15.681734 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:15.681703 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-pdhmz_2cbd34f1-8649-41be-9595-798f5e903723/discovery/0.log" Jul 07 17:00:15.738576 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:15.738548 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cbbdb79bb-lbrks_d6f791a8-4e75-4f89-9b8f-c65fdd57aa21/kube-auth-proxy/0.log" Jul 07 17:00:15.771023 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:15.770995 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9rftp_d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f/istio-proxy/0.log" Jul 07 17:00:15.797436 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:15.797403 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9rftp_d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f/istio-proxy/1.log" Jul 07 17:00:16.609769 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:16.609739 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-8hr9f_ebc29602-88aa-4ce8-aa11-341d20706083/serve-healthcheck-canary/0.log" Jul 07 17:00:17.184626 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:17.184591 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-mmrgp_40c9cc16-aae3-4bb5-b11a-7723051bb9c0/insights-operator/0.log" Jul 07 17:00:17.185038 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:17.184723 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-mmrgp_40c9cc16-aae3-4bb5-b11a-7723051bb9c0/insights-operator/1.log" Jul 07 17:00:17.313762 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:17.313735 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n78cx_f029375b-5a3c-4990-9712-27f99db223ae/kube-rbac-proxy/0.log" Jul 07 17:00:17.340988 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:17.340964 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n78cx_f029375b-5a3c-4990-9712-27f99db223ae/exporter/0.log" Jul 07 17:00:17.341830 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:17.341815 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-948xd/perf-node-gather-daemonset-gxs85" Jul 07 17:00:17.369557 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:17.369521 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n78cx_f029375b-5a3c-4990-9712-27f99db223ae/extractor/0.log" Jul 07 17:00:20.219496 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:20.219468 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-xcq85_5abd0a46-2e65-49fd-a06d-30ee5cd38791/manager/0.log" Jul 07 17:00:20.351466 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:20.351438 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-lnck9_8492c6fd-ea82-4bb1-ab8e-4455c75dbe6d/manager/0.log" Jul 07 17:00:20.710570 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:20.710533 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-6gzws_bce89f98-6a41-4485-a4c7-f1bb3368e25a/manager/0.log" Jul 07 17:00:20.739404 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:20.739369 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f64778974-22mbb_bdb26e74-5407-41be-a50f-c9ecfbd8836f/manager/0.log" Jul 07 17:00:20.768528 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:20.768499 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6f64778974-22mbb_bdb26e74-5407-41be-a50f-c9ecfbd8836f/copy-manifests/0.log" Jul 07 17:00:21.363590 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:21.363559 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-pdhmz_2cbd34f1-8649-41be-9595-798f5e903723/discovery/0.log" Jul 07 17:00:21.392360 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:21.392337 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cbbdb79bb-lbrks_d6f791a8-4e75-4f89-9b8f-c65fdd57aa21/kube-auth-proxy/0.log" Jul 07 17:00:21.412755 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:21.412724 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9rftp_d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f/istio-proxy/0.log" Jul 07 17:00:21.442734 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:21.442696 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9rftp_d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f/istio-proxy/1.log" Jul 07 17:00:22.449159 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:22.449127 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-tpvdp_1a1f1bb1-aa88-4eac-878c-d1f63c329a16/openshift-lws-operator/0.log" Jul 07 17:00:22.798269 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:22.798167 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-pdhmz_2cbd34f1-8649-41be-9595-798f5e903723/discovery/0.log" Jul 07 17:00:22.839983 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:22.839950 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cbbdb79bb-lbrks_d6f791a8-4e75-4f89-9b8f-c65fdd57aa21/kube-auth-proxy/0.log" Jul 07 17:00:22.862473 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:22.862445 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9rftp_d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f/istio-proxy/0.log" Jul 07 17:00:22.889218 ip-10-0-129-174 kubenswrapper[2564]: I0707 17:00:22.889171 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-9rftp_d85ae2f7-2c71-4c67-820d-6fcdd23b9d6f/istio-proxy/1.log"